THE VALUE OF INCLUDING ESTIMATES OF PRECISION IN FINISHED INTELLIGENCE

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP83M00171R000800140002-3
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 4
• Document Creation Date: December 16, 2016
• Document Release Date: March 24, 2005
• Sequence Number: 2
• Publication Date: April 1, 1981
• Content Type: PAPER

25X1 Approved For Release 2005/04/18 : CIA-RDP83M00171 R000800140002-3
Approved For Release 2005/04/18 : CIA-RDP83M00171 R000800140002-3
 rpi
AppW~AU9O'A19j:'- 'r 301 R000800140002-3
The Value of Including Estimates of Precision
in Finished Intelligence
It has often been noted in RMS that resource management of the
intelligence ooammunity is organized around the technical collection systems.
There is wide agreement that this is suboptimal and that instead we should
manage resources to accomplish our objective, support to intelligence users.
We do attempt to assess how well collection improvements serve this pur o e t
estimating the increased intelligence information that may be obtained from
improvements to the collection system, but no one pretends that this is c t ir. i
since only those improvements in our support to users that can be obtain-d'
through improvements to the technical collectors are considered. Thou;::
attempts have been made, it is not sufficient to simply extend this e.":. :Y
technical collection initiatives to all collection initiatives since the ?'
useful and cost-effective improvements in support to users may not even b
discovered through this approach. Nor is it sufficient to ask how to
processing, exploitation, dissemination, or production for the saune rea->.;-=
there is no guarantee that all of the most promising opportunities fs
improving support will be considered because the focus of this approac`. --?= x
means rather than ends.
I have come to believe that the principal reason that the ffanaqtiffe n! ?
the Intelligence Community is collector oriented is that there is infor- t S-3
to be had about the benefits of improving the collectors and, more Li xr nt
there is little information available about how to -Ili rove user supPcr=. -_
information that would appear at first glance to be most desirable for
improving intelligence community performance against its objective is dire-':t feedback from the decisionmaker about the usefulness of cur suPPcrt- "`' `
the value of intelligence in providing support is straight ftrward in tt*'
abstract, but very difficult to actually accomplish - partlyaus"- ? ,
is a cci polite of a number of individuals rather than a unitary act
principally because the outcome measure is difficult to define in a t e
way.
Appro~'l $,IR1P3$Mf7lP000800140002-3
 AD df iSTUTjg r---' 7- Y
Approved For Release it/ 4~1{8'!fZD14000800140002-3
The Value of Including Estimates of Precision
in Finished Intelligence
The usefulness of additional intelligence support, c any added
information for that matter, to a decisionmaker can be defined operationally as
changing the assessment of
o consequences: a changed assessment of the consequences to be
expected fran selecting alternative courses of action (costs,
benefits, and risks),
o uncertainty: an increase or decrease in the uncertainty surrounding
the estimates of costs, risks, benefits anticipated fran alternative
courses of action (note that support is improved by a more accurate
portrayal of uncertainty rather than by simply maximizing confidence
by minimizing uncertainty - this may not be evident to the
decisionmaker ),
o alternatives: the addition or elimination of alternative courses of
action fr om the feasible set, or
? o timin : changes in the timing of the decision (finding that it can
be made earlier or later) or changes in the expected timing of the
consequences of selecting an alternative (later or earlier arrival of
costs, benefits, and risks).
We have conducted user surveys to obtain this information but the results have
been disappointing, primarily because there is no clear audit trail available
to define the affect of intelligence information on a decision and also because
there is no unitary actcr to query for informal recollection of these effects.
It is difficult as well to identify intelligence shortfalls because there is a
strong interaction between what can be done f - the decisionmaker and what they
in turn request or expect. Because the demand for intelligence is ephemeral
and easily influenced by expections and changing events, it is difficult to
distinguish highly useful intelligence from barely useful intelligence and
close to impossible to make finer distinctions. Furthermore, to be usefully
precise, demand must be expressed as a result of painstaking internal
comparisons such as those made when operating under a budget. No such
discipline is imposed on the intelligence user and this inevitably diminishes
the value of user surveys.
How then can we develop an integrated criterion for allocating resources
in the community? Moving back from the user one step to the production of
intelligence could be very useful if information, were readily available about
the difficulties in addressing intelligence issues that the shortages or
imprecision of intelligence information have caused. That information is
currently not available. Judging from the intelligence products I have
examined over the past 2-1/2 years, the current production process disguises
Approved {. / R E "P80PA @0 800140002-3
~1 ~~~ ill 1 till. 'i. `:.l
 L - UG LY
ADMINISTRATIVE-I Bi . I
Approved For Release 2005/04/18 : CIA-RDP83M00171 R000800140002-3
Yn value of including Estimates of Prtz_iaion
in Finished Intelligence
engineering could not be used (sual as Po of passible values. In that case
it would be sufficient to describe the range
it would be necessary to include sore discussion of where in that range the
this is so. If there is no possibility of
best estimate lies and why
that result in a usefully nsdoneabye, a
specifying upper and lc*er bounds ified. This is
subjective 9o% upper confidence interval
values about theces i ate that define the ranca
indicating the upp upper and that the analyst expects will include the correct value 90% of the time.
lt a wealth of would information W~xovesupport to
This approach would provide
resource allocation decisions.
users in two ways:
o they would understand much better the uses and limits of the
intelligence they receive and,
o an additional avenue would be provided to the intelligence analyst
for ccrrnunicating concern about events with poorly estimated
probabilities that are i ,crtant nonetheless for decisionmaking.
An alternative implementation of this same approach would be to have the
cif ing the accuracy and precision of intelligence
wouci
asssseessessmen oft s fo assign yed to to s a y separate croup of intelligence analysts. They
an the bias and imprecision of intelligence
be chosen for their familiarity with to estimate the ezrors introduced by the
information as well as their ability
estimating procedures through sensitivity testing and contingency analysis. ern
of establishing a separate group for this function is that the group
advantage a applications most in need of i OVei't
could also be tasked to identify those allocation proposals for providing su:
and to review the potential of resource
improvement.
mitigated by the extensive use of qua i s events. (This is especially
precise statements about highly conditional events poorly known.)
misleading when the likelihood of the conditioning
An approach that would provide substantially more information on which to
base resource decisions better would be to include weer possible, statistica.-
accuses and precision in the final product using,
measures of bias and variation. Where the methods Of ysis int some instances)
certainty that would ordinarily be
that small amount of information about ` ments uncovered thorough
evident to the reader by suppressing d,-~a? by eliminating those aspects o'-
coordination by resorting W vaguer language sion. This is exacerbated by a
the issue that are in dispute from the final ced to disguise differences in
style of Prose that is highly readable but designed
essien of certainty
certainty from one topic to another by
1' f ' giv~s ing a in the the text i and by including
Approved V tR ..83
7A0800140002-3