SENIOR REVIEW PANEL RESPONSES TO QUESTIONS FROM THE SENATE SELECT COMMITTEE FOR INTELLIGENCE (SSCI)

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP98S00099R000400800006-2
• Release Decision: RIPPUB
• Original Classification: S
• Document Page Count: 5
• Document Creation Date: December 23, 2016
• Document Release Date: December 4, 2012
• Sequence Number: 6
• Publication Date: July 24, 1985
• Content Type: MEMO

r-, 05X1
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
./ .... / _ / v Z-tTy%-:b IV - (1 LO ,~
Q)
-
ors c ,))
THE DIRECTOR OF CENTRAL INTELLIGENCE
NIC 03713-85
Senior Review Panel 24 J u l y 1985
MEMORANDUM FOR: Associate Deputy Director for
Intelligence
Chief, Intelligence Producers Council
Staff
SUBJECT: Senior Review Panel Responses to
Questions from the Senate Select
Committee for Intelligence (SSCI)
Attached are the subject responses to questions
raised by the SSCI upon the occasion of the DCI's
testimony at the hearing on National Intelligence
Strategy, 19 June 1985. As requested, the answer to
Question 5 was coordinated with the Product Evaluation
Staff.
Attachment
ODCI/SRP/HCR:jsb
Distribution:
1 - Each Addressee
1 - SRP File
1 - SRP Chrono
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
 Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
C
C
NIC 03713-85
24 July 1985
Senior Review Panel Responses to Questions Raised
By the Senate Select Committee on Intelligence
On 19 June 1985
5. Generally speaking, what are the criteria -by which you judge
the quality of intelligence? Please be specific. Are these
criteria weighted differently in different situations?
Given these criteria, how well is the Intelligence Community
measuring up? Have you conveyed these specific criteria to
the CIA's Product Evaluation Staff? Does the Product
Evaluation Staff use these criteria in its own work?
Answer
In its assessments of Community Production the Senior Review
Panel has established a number of criteria for judging
intelligence quality and utility. Except under special
circumstances these criteria are not given specific
quantitative weights and no attempt is made to attach
numberical grades to individual intelligence products. The
Panel's judgments are collegial, after its members apply
these criteria individually, independently, and subjec-
tively, and alter their relative weight assignments
(subjective and approximate) in accordance with the nature
and purpose of the intended intelligence publication. The
Panel reviews proposals for new National Estimates in their
initial stage (Concept Papers and Terms of Reference) and in
one or more successive drafts generally from the standpoint
of adequacy to policymaker needs. The following basic
criteria form a first filter for quality:
(a) Relevance to an identified policy issue or an emerging
trend of significance for US policy.
(b) Timeliness in providing an input to a policy decision
or a review of existing US policy.
(c) Conciseness in expressing its judgments.
(d) Clarity in its presentation.
(e) Completeness in covering the principal policy-relevant
issues, addressing the main intelligence questions
bearing on them, and using all available intelligence.
(f) Logic and Consistency in its development of arguments
from basic intelligence data to final judgments.
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
 Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
Additionally, several further criteria are used to
distinguish the excellent from the acceptable study:
(a) The extent to which the level of confidence in its
judgments is clearly communicated.
(b) The degree to which alternative, reasonably likel ,
outcomes are examined, the developments which would
lead to them identified, and the odds on their
occurrence estimated.
(c) The provision of a set of intelligence indicators
which can be used to follow the continuing unroll ing
of events, and of intelligence gaps.
(d) The balance between speculation and hard fact in the
analysis, and the extent to which the two are
distinguished.
(e) The clear exposition of the assumptions used.
(f) The adequacy with which dissenting views are
presented, their bases explained, di ering evidence
or interpretations set out, and their implications
projected.
The Panel believes that on the whole the jointly produced
products of the Intelligence Community (NIEs, SNIEs, IIMs,
IIAs) measure up well against these criteria. The quality
of these products is at its highest point in recent years,
if not in the entire history of the Intelligence Community.
The Production Evaluation Staff is aware of the criteria
used by the Panel in its review of National Estimates.
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
 Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
/ 11 -n1tl 25X1
6. What evidence does the Intelligence Community actually have
to show that the intelligence it produces today is better
than the intelligence that was produced five years ago?
Better than 20 years ago? Is any record kept on the
consumers' responses to the quality of this intelligence
over these periods?
Answer
The Senior Review Panel reviewed all joint products of the
Intelligence Community (NIEs, SNIEs, IIMs, IIAs) for the
period 1977 through 1984, and compared the quality of those
products issued in the second half of that period with those
produced in the first half, using the criteria described in
the answer to Question 5, above. The results of that study
were presented to the DCI in a memorandum dated 24 May 1985
(copy provided). The overall findings are best encapsulated
by the concluding paragraph of that study.
"19. Summing up our inquiry into the Phase I and
Phase II periods, we conclude that, while further
improvements remain to be made, the national estimates
underwent far-reaching and significant change,
amounting to a virtual transformation of the
product. In Phase II, coverage was broader, more
consistently applied, better timed and coordinated
with decision points, and more attuned to the entire
range of critical issues facing policymakers.
Alterations in format and style much improved the
quality of communication between analysis and
consumer. The anticipatory and predictive content of
estimates markedly increased; reactive and retrospec-
tive analysis receded. And a new generation of
broadly-ranging, strategic estimates, without
precedent in Phase I, opened newer perspectives for
national intelligence support of the policymaking
process."
No specifically comparative study covering a 20-year period
has been carried out, although the review referred to in the
answer to Question 7 bears on the problem. The Panel does
not maintain a record of consumers' responses to the quality
of this intelligence for these periods.
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
 Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2
In your testimony, you said that you had the Senior Review
Panel study "how and why the Intelligence Community had over
the last two decades failed to call (and) identify critical
turning points accurately in Cuba, in Vietnam, in Iran, in
Nicaragua, in Afghanistan before the invasion at the time of
the Communist takeover and a couple of other kinds of
critical turning points."
a. We would appreciate your furnishing this study to the
Coinmi ttee.
b. Have you commissioned similar studies by the Senior
Review Panel to investigate any intelligence failures
that may have occurred since 1980? If so, we would
appreciate copies of these studies as well. If such
studies have not been commissioned, then why?
Answers
a. Copy furnished.
b. The Panel's 24 May 1985 report is responsive to this
question.
S E A K L I 25X1
Declassified in Part - Sanitized Copy Approved for Release 2012/12/04: CIA-RDP98S00099R000400800006-2