STATING UNCERTAINTIES IN INTELLIGENCE ESTIMATES

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP80B01495R000700090003-6
• Release Decision: RIPPUB
• Original Classification: C
• Document Page Count: 10
• Document Creation Date: December 19, 2016
• Document Release Date: November 16, 2005
• Sequence Number: 3
• Publication Date: August 8, 1975
• Content Type: MF

Approved For Relebod 2005/11aIUi -IT80B01495R00100090003-6
8 August 1975
MEMORANDUM FOR: Samuel V. Wilson, Lieutenant General, USA
Deputy to the DCI for the Int.elligency Community
SUBJECT: Stating Uncertainties in Intelligence Estimates
1. This is in response to your letter to Dr. Proctor of
25 July 1975. As you have noted, the problem of stating more
explicitly the uncertainties in political judgments has been with
us for a. long time. Sherman Kent, despite heroic efforts, was
not entirely successful in winning acceptance for his scheme of
attributing mathematical odds to verbal expressions of probability.
Perhaps the best exposition of Sherman's ideas is an article in
Studies in Intelligence, Vol. 8, No. 4, Fall 1964. Sherman himself
sponsored a fairly ambitious project to find, out what the producers
and-the customers of intelligence estimates meant -- in numerical
terms -- in the use of common estimative words. This project,
carried out byl is described in an article in the same
issue of Studies cited above.
2. The 0 and Kent articles illustrate the inherent diffi-
culties in using numerical equivalents of estimative words to
convey a more precise meaning of uncertainty. Doubtless there
have been other efforts in the past ten years to tackle the problem,
and one could review this activity and summarize the results.
It would, however, be a pretty sterile exercise, ending with
conclusions not very different from Kent and
.3. We would suggest instead that some new techniques be brought
to bear on the problem of more explicit statements of uncertainty.
The Analytical Techniques Group of OPR as part of its work in
the development of forecasting techniques and predictive measures
has dealt with the need to reduce ambiguity in intelligence assess-
ments. The various Bayesian analyses conducted by this Group con-
stitute one approach to improved clarity in expressions of uncertainty.
The experience of the Analytical Techniques Group, in the use of
quantitative measures and group assessments suggests that the
problem of uncertainty extends back to the point at which the
intelligence problem is defined. To come up with more explicit
intelligence assessments it is necessary to articulate the problem
more carefully and to design the analysis more systematically --
or more imaginatively -- than is normally done.
25X1
Approved For Release 2005/12 FQfL80B01495R000700090003-6
CON
 CONFIDENTIAL
Approved For Release 2005/12/24: CIA-RDP80B01495R0000090003-6
4. The Analytical Techniques Group is now testing procedures
by which forecasting and estimative techniques currently used by
industrial and consulting firms can be applied to the intelligence
process. Of particular interest is the use of subjective -- but
explicitly formulated -- expert opinion to assess the impact of
a variety of contingencies upon a given state of affairs. Should
these experiments yield positive results,.'the.new. techniques would
enable a working group of intelligence analysts to identify more
clearly the areas and levels of uncertainty within a particular
intelligence question, and to express the uncertainty with greater
clarity. The use of such an approach would tend to generate estimates
of a range of possible outcomes or future scenarios at various levels
of probability.
5. This project is still in an exploratory phase in OPR.
Our previous experiences with the adaptation of techniques and
methods from academic or business applications to intelligence
suggest that it generally takes more time and sweat than originally
thought. Nonetheless, the various facets of forecasting so far
examined show considerable promise and may help; directly to make
explicit the uncertainjles inherent in political judgments.
25X1
kEW.' S J. LAPHAN
Director; Political Research
CONFIDENTIAL
Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
 Approved For Relea 20059 -~PIA..J k B01495R0004W'D090003-6
THE DIRECTOR OF CENTRAL INTELLIGENCE
WASHINGTON, D. C. 20505
Intelligence Community Staff
Dr. Edward W. Proctor
Deputy Director for Intelligence
Central Intelligence Agency
Washington, D. C. 20505
25 JUL 1975
During recent meetings of the NSCIC Working Group, Ambassador
Ellsworth and other members have pressed for the production of a
paper on the problem of how to explicitly state uncertainties in intel-
ligence estimates. I would like to bring you up to date on this matter,
and request your assistance.
This is not exactly a new problem, as anyone familiar with
Sherman Kent's efforts to wrestle with it will remember. Yet it is
clear that high-level users of intelligence are still seized with it.
Accordingly I have seen to it that briefings to the NSCIC Working
Group address points related to probabilities and confidence levels
on topics such as SALT, MBFR, and Warning.
Members of the Working Group welcomed these responses but
suggested that together they provide only 113040 percent" of the
answer to the broad question. The Group wishes us now to cover
the problem of expressing uncertainties in the "softer" areas--e. g.,
in political judgments. I have agreed to see what could be done.
In this connection, I would like to draw on the expertise in the
DDI. There would be value??I believe--in a study that reviews the
various attacks on the problem over the years and draws some
general conclusions on where we now stand.
25X1
Approved For Release 2005/12/f ID CtTa 80Bd1495R000700090003-6
 U 1DENT1AL
Approved For Release 2005 . CIA-RDP80BO1495R00QW0090003-6
has informally consulted Lew Lapham about this
matter and Lew said he would be glad to give the matter some
thought. Please let me know how you react to the idea.
Sincerely,
25X1
Sam Z V ilson
Lieute nt eneral, USA
Depu to t e DCI for the
Intell Bence Community
cc: Director of Political
Research
Approved For Release 2005/12/2~OWI MM EW495R000700090003-6
 Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
MEMORANDUM FOR: W (info VW/HR/DK)
Sam Wilson would like your reaction to the preparation
(by OPR?) of a study of the problem of expressing uncer-
tainties in political judgments in estimates.
Suggest giving action to Lew Lapham. Copies to OCI
and OSR.
25X1
25X1
5.75 FORM 101 U ESE PREVIOUS
yI-I I_aI V I WI \ V1 1CC 1
Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
 File: OPR
Approved For Relea12005/12/24 : CIA-RDP80BO1495R00070090003-6
OFFICE OF THE IDD/i
22 July 1975
NOTE F'OR: Director of Political Research
I would like to hold off any action on
this proposal for a while for several reasons:
-- We are losing conference room space
in other parts of the building, and we
all are concerned about whether what
is left will be sufficient.
I don't think that putting 6 people in an
area of 375 square feet is very good for
analytical work.
We are working with the Logistics people;
to straighten out our whole space prob-
lem. They have come up with a new
plan which is now under review in the
DDA. To make your suggested move
now might complicate this situation.
Even if we put your order in now, work
would not be started until the end of the
summer. By that time your 6 summer
interns will. have left. We expect to
discuss the new Logistics plan within a
week or two. If it doesn't satisfy us, we
can
reconsider converting your confer-
ence room.
// Ed
Ed Proctor
cc: DDI Management Staff
Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
 OFFICE OF THE Dh
Approved For Release 2005/12/24: CSR P80BO1495R000700
21 July 1975
SUBJECT : OPR Space
I think you ought to ask Lew to hold off
before he converts his conference room:
a. OER is getting rid of one of
its conference rooms and they both
are used almost constantly.
b. Putting 6 people into an area
of 375 sq. feet doesn't do much for
analysis.
c. Logistics people have come
up with a plan which they say will take
care of our immediate needs. It has
been briefed to John McMahon who
endorses, and Blake is getting briefed
this week.
d. Even if Lew converted, it wouldn't
be done until the end of summer, by which
time 6 summer interns would have left.
So I would hope that he could somehow
or other get by until the Logistics plan
can get started.
UN
0
0
0
25X1
Approved For Release 2005/12/24: CIA 80BO1495R000700
cc:
 Approved For Rele06$FTI$~-CK60N~4
06 00zaO090003~
18 July 1975
MEMORANDUM FOR: Deputy Director for Intelligence
SUBJECT : OPR Space
1. This memorandum is prepared in order to seek your assistance
in resolving OPR space problems in a manner least disruptive to the
creative research process.
2. OPR is unable to accommodate its FY 76 T/O of 80 people and
a handful of part-timers within its present assigned space. I _J 25X1
was advised sometime ago that OPR would require space for about four
people before the end of the summer and that later in the year addi-
tional space would be required, and warned me that such space would
not be optimum for people in analytical and reflective work.
3. True to his promise (and his warning), about 200 square feet
have been identified in a very large CRS bull pen on the ground floor.
This space is set off with half partitions and is presently occupied
by two Security Officers. It would be possible to use the space for
three people. There is, very naturally, a substantial amount of
diversified activity going on within the bull pen, people walking
back and forth in the corridors discussing their problems, a great
deal of typing and other office machines operating; in short, researchers
attempting to work there would be surrounded by noisy distractions.
4. Last week I showed the area to each one of the OPR Staff
Chiefs. They were unanimous in their feeling that the area was
simply an unsuitable place for an OPR analyst to do quality work and
that the consequences on both morale and production in assigning any
analyst there, either newly hired or experienced, would be unacceptable.
5. Consequently, we reviewed the options of utilizing the space
under our own control. The options are few, and all of them are to
some extent distasteful. We finally concluded that we should give
up our conference room (375 square feet) and partition it for office
use. We are most reluctant to do this, for we make a good deal of
use of this facility for ourselves, and it is heavily used by other
Approved For ReleADM N STRATIV-CINTERNAL USE4~~l 9OO700090003-6
 Approved For ReleJlaWA721(EfCAB8440gtZ)600Zp0090003-6
SUBJECT: OPR Space
offices when not engaged for OPR purposes. It is my understanding
that conference room space is at a premium in the building, and
that Logistics will oppose reconfiguration of conference room
space. This is understandable. I am sure that OPR is going to
have difficulty locating conference space when it is needed. It
is far preferable, however, to the alternative we have been offered.
6. I am therefore seeking your approval to reconfigure the
OPR conference room for office use and your support with Logistics
to implement the decision. We believe that we can use this conference
room space for six analysts. Later in the year when additional space
is required, we're simply going to have to crowd more people into the
space we have in order to house them all unless circumstances change
in the Headquarters building. The OPR Staff Chiefs are, as I have
said, unanimous in their conviction that this is preferable to the
utilization of semi-partitioned bull pen space.
7. It is my intention to continue an active recruitment program
to identify high-quality candidates to fill our projected FY 77 T/O
of 88 when and if it is approved. However, I do not plan to make
any hiring commitments until we have some assurance that suitable
space will become available. For as long as the space situation
remains unchanged, we will simply have to live with our present T/O.
8. I want to emphasize that nothing in this memorandum is to
be construed as implying criticism of DDI Management Staff. I fully
appreciate that the problem is not of their making. Our objective
is simply to come up with the least undesirable means for creating an
environment as conducive as possible to creative research.
25X1
Director, Polit\ l Research
cc: Chief, DDI Management Staff
-2-
Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
ADMINISTRATIVE-INTERNAL USE ONLY
 Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6
TRANSMITTAL SLIP
DATE
18 July 1975
TO.
ROOM NO.
BUILDING
7E44 I
Hqs.
REMARKS:
el,
~
V h
pis
FROM: D/PR
ROOM NQ.E63 BUILDING Hqs.
FORM 55 24 I REPLACES FORM 36-B
WHICH MAY BE USED.
25X1
Approved For Release 2005/12/24: CIA-RDP80BO1495R000700090003-6