IMPROVING INTELLIGENCE ANALYSIS

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP86B00985R000100110011-8
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 15
• Document Creation Date: December 19, 2016
• Document Release Date: August 16, 2005
• Sequence Number: 11
• Publication Date: March 7, 1977
• Content Type: MF

Approved For Release 2005/11/28: CIA-RDP86B00985R0000100110011-8
. .r_ J15'
.7 March 1977
MEMORANDUM FOR: Sayre Stevens
Deputy Director for Intelligence
Acting Director, Center for Policy Support
SUBJECT : Improving Intelligence Analysis
1. This relates to our discussion on February-16
about alternative approaches in intelligence analysis.
Four of these--alternative hypothesis analysis, competitive
analysis, devil's advocacy, and alternate conclusions to
a best judgment--are discussed in the-attached memorandum
drafted by
2. Also attached is a companion piece by
that examines one means of institutionalizing such
approaches and augmenting the analytical process. The
memorandum discusses four. frameworks in which new panels
of outside specialists could be constructed for the DDI.
3. These memoranda are preliminary cuts at the issue
of institutionalizing alternative hypothesis analysis in
the Directorate. Your reactions to the paper on panels
will assist, us in developing the paper requested by the
DDCI (see attachment).
Attachments:
As stated
cc: Paul Walsh
Evan Hineman
Karl Weber
ADMINISTRATIVE - INTERNAL USE ONLY
Approved For Release 2005/11/28 : CIA-RDP86B00985R000100110011-8
STAT
STAT
 Approved For R%L96962005/11/28: CIA-140086800985 R@W100110011-8
SUBJECT Improving the Quality of Intelligence
Analysis
Introduction
1. Intelligence analysts usually work with incom-
plete data and the deceptive areas of human perceptions
and intentions. Because of the uncertainties, different
conclusions can sometimes be drawn from the same data.
The problem for the intelligence analyst is to ensure that
the data are rigorously examined, the uncertainties are
clearly stated and the implications of alternative con-
clusions are clearly drawn. The utility of intelligence
analysis for the policy oriented intelligence consumer
is in direct proportion to how the analysis meets these
criteria.
2. Four analytical techniques. that appear to have
merit for intelligence analysis include (1) alternative
hypothesis analysis (2) competitive analysis (3) a devil's
Approved For Release 2005/11/28 : CIA-RDP86B00985R000100110011-8
1 jfi~1
 Approved ForIease 2005/11/28: CIA-RDP86B009800100110011-8
advocate or challenging mechanism and (4) alternative
conclusions to a best judgment.
Alternative Hypothesis Analysis
3. Alternative hypothesis is a technique that en-
sures the examination of more than one hypothesis.
Problems that lend themselves to this kind of treatment
include those in which alternative hypotheses already
exist in the community but have not been rigorously
analyzed; those about which enough significant. disagree-
ment.exist:s to develop possible alternative'hypotheses;
and those fraught with so many uncertainties that al-
ternative hypotheses should be developed. Time con-
straints, the realities of human nature, and past
experience indicate that this approach would be most
effective if individual alternative hypotheses were
developed by separate groups of people.
4. A substantively strong project monitor,
preferably one skilled in understanding and working
out the human problems likely to be'generated by such
an'exercise, would be required. He would staff the
groups, serve as an interface between them, establish
procedures, ensure that teams have equal access to
all pertinent data, and generally monitor their progress.
519 nrrl (ft
Approved AAag Ob /T1/SCR Bf +6 5R000100110011-8
-2-
 Approved For' e L 2005/11'/28': elA=f bO86 C1b9 'SW00100110011-8
Competitive Analysis
5. In contrast to the alternative hypothesis method,
the competitive analysis technique does not presume that
alternative hypotheses already exist or, can be developed.
Instead, two or more groups examine the same problem
independently, and they may or may not use different
hypotheses or reach different conclusions. Competitive
analysis already takes place to some extent among the
various government agencies. Policy questions that
generate wide ranges of uncertainties or disagreements
also lend themselves to this kind of treatment.
6. Both competitive groups could be staffed by
Agency people, or one of the groups could be staffed
by outside contractors. The latter seems preferable.
Experience indicates, moreover, that efforts should be
made to choose people with a mix of views. If either
team were dominated by one set of preconceived biases,
significant hypotheses might not be objectively and
rigorously analyzed. Project monitors would also be
required for competitive analysis exercises. Their
.functions and required skills would be the same as
those discussed above.
21 VV Ally
is 14 40
~3-
Approved For Release 2005/11/28 : CIA-RDP86B00985R000100110011-8
 Approve'd4o d9W.i464 b05/14Yy t:'61A b?8 66981WO0100110011-8
Devil's Advocate/Challenging Mechanism
7. Establishing a devil's advocate or challenging
function would be a third possible way of improving
intelligence analysis. Instead of 'facilitating a positive
? exercise of pursuing various analytical.
paths, a devil's
advocate function would serve more as a critique mecha-
nism. It could be reserved for those analytical products
that could profit from an occasional intense probing to
determine whether or not analytical methodologies used
were sound, whether sufficient alternative hypotheses
were surfaced, and whether any of the latter had been
discarded too early in the analytical process. Taking
an analytical product through such a rigorous test,
either during this formative stage or after its completion,
should give analysts and consumers more confidence in
the product.
8. This function could be performed by appropriate
Agency personnel or outside specialists.. Selecting the
latter would have the advantages of keeping the communi-
cation =Lines with outside research specialists open and
guarding agianst an Agency mind-set predominating.
Specialists could be called in either on a regular basis
or as needed.
~ t
AdmiPPis~rod, BYO - Luc ly
Approved For Release 2005/11/28 : CIA-RDP861300985R000100110011-8
 Approvee o? ie~ase 2005/1j{4$.; cIA,3:QP86B9098i 000100110011-8
Alternative Conclusions to a Best Judgment
9. Presenting alternative conclusions to a best
judgment is another method of, focusing adequate attention
on alternative hypotheses, and it is fairly self-explana-
tory. When analyzing data that could easily lend itself
to various interpretations, an analyst could systematically
set forth the possible alternate coi fusions with their
individual policy implications. One person could under-
take this type of approach alone, or several analysts
could work together, each being tasked to develop one
or more of the possible conclusions.
Utility to Managers
10. The results of any of the'above approaches
would bie of some use to all levels of Agency managers.
They would reap the benefits of an additional quality
control system on their products, knowing that the key
judgments and assumptions of their analysis have been
rigorously tested. There would also be the advantage
of being able to inform policy oriented consumers of
unresolved uncertainties and the implications of
any significant alternative hypotheses. Managers would
profit by having been alerted in advance to possible
11iJ r 1111Gf1f.o Vl3V MY
-5-
Approved For Release 2005/11/28 : CIA-RDP86B00985R000100110011-8
 Approve
05/1 f28r-r; FA.P .. 8.66 98IM00100110011-8
.10
arguments against best judgments and by being better
prepared to defend them. Additional data-or new metho-
dologies might also be surfaced as a result of any of
these approaches.
11. The advantages discussed above would have to
be weighed against several disadvantages. Overseeing
any of these techniques would require considerable
additional time and energy from Agency managers. A
professional atmosphere would have to be maintained
in which. the focus would not be on' the data or the
conclusions, but on the analytical process. All
participants would have to appreciate that our analytical
work seldom produces a single, unassailable conclusion.
12. These approaches also would require additional
personnel and financial resources. Because such efforts
would be expensive, they should be focused selectively
on only those analytical problems that have serious
policy implications.
liA1'L10II'1Qri9lL*,4r.-A M w=.....~,P~ QO ~~
i.ilisuiSiu-hiGt:ttri~iErttJ`