Study Suggestions

Document Type: 
Document Number (FOIA) /ESDN (CREST): 
Release Decision: 
Original Classification: 
Document Page Count: 
Document Creation Date: 
December 9, 2016
Document Release Date: 
August 12, 2000
Sequence Number: 
Case Number: 
Publication Date: 
August 9, 1972
Content Type: 
PDF icon CIA-RDP84B00506R000100050014-5.pdf287.71 KB
Approved For Relee 2001/08/14: CIA-RDP84B00506R0W-00050014-5 NATIONAL SECURITY COUNCIL WASHINGTON, D.C. 20506 SECRET August 9, 1972 MEMORANDUM FOR RAY S. CLINE!, FROM: A. W. MARSHALL SUBJECT: Study Suggestions I have reviewed your three suggested studies (Chile, Yugoslavia, NSSM 108) and have discussed them with interested members of the NSC staff. Based upon the information now before me, I would prefer to see review efforts conducted towards Yugoslavia., Chile and NSSM 108 in that order. _ Ranking the three studies is a. difficult task because they all have potential. However, I do not favor starting with NSSM 108. Even if restricted to reviewing the management process, it would still be a major undertaking. Activities surrounding its development took nearly a year's time. To assess the management process would require investigating not only the link between policymakers and the community, but also the managerial efforts within the community to bring together analysts on a number of nations to develop a comprehensive picture of Latin America. Chile and Yugoslavia seem more manageable and relevant to present needs. Chile and Allende obviously are important influences on U. S. policies to- wards Latin. America; analysis better attuned to policymakers' needs would be helpful. Should the NSCIC Working Group chose this study, I recommend the time frame be moved back to January 1970 (or thereabouts) in order to evaluate the analytical output surrounding Allende's rise to power. The Yugoslavia study seems marginally more important than Chile because of the distinct possibility of political turmoil there, in the near future, which could be exploited by other powers. Without going into detail on any one proposal, I have some general sugges- tions concerning the objectives, methodologies and the analytical or management issues to be addressed that may be of some use to ycu: SECRET *NSC Declassification/Release Instructions on File* Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5 Approved For Rele 2001/08/14: CIA-RDP84B00506R0@00050014-5 SECRET OBJECTIVES The objectives of a study should be focused on presenting the NSCIC with lessons learned from reviewing the topic and developing specific guidance for the NSCIC to consider passing to the community. Detailed historical records will normally have to be compiled to carry out these studies, but this effort should not be allowed to overwhelm the study groups. Where necessary, sampling of products could be used. Also, as one of your studies suggests, comparing two years separated by four to five years might reduce the workload and focus on the question; --Have we improved over the last several years? What are the trends in key indicators of pro- duct quality/ or analysis staff quality, etc.? What is important is (a) the evaluation of the intelligence process and products during the period, and (b) the presentation of a rigorously developed set of conclusions, general and specific recommendations, and appraisals of the likely impact of these recommendations on the quality of production and the internal functioning of the community. All three studies have a distinct management orientation in that they propose to examine the interplay among output, guidance received and resources available. Qualitative evaluation of a particular product apparently would center on asking whether it clearly made its points and whether these points were relevant to the needs of policymakers at the time. Little work will be performed directly to evaluate the intrinsic quality of the product - i. e. , whether it represented an exhaustive perceptive analysis -- except as this falls out from the study. This approach does not disturb me. I find it perfectly appropriate to target the study efforts on the management issues as well as on analytical ones. However, for some selected products an assessment of the quality of the intelligence analysis, especially as regards political and governmental behavior, would be of interest. What is impor- tant is to insure that everyone participating in the study understands the approach. METHODS I envision that the role of the working level participants in a study would go beyond that of recognizing problems and making general recommendations. SECRET Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5 SECRETved For Rele 2001/08/14: CIA-RDP84B00506R0QQ,100050014-5 Since these individuals will be the ones most familiar with the intelligence effort surrounding the topic, and will have spent the greatest time thinking about it, they should have some valuable thoughts about both the very broad changes needed and the specific adjustments which could be made in the present system of managing the intelligence process. Fully exploiting their insights is essential to drawing maximum value from the work which goes into studies. The working group can still be the final judge of the weight of the case the study group makes. I suggest the NSCIC Working Group select one of the topics to begin immedi- ate work on. You should guide the study and I would be delighted to help you in any way I can. The remaining two could be postponed until later in the year when present studies will be nearing completion. At that time these topics could be considered along with any others that have been submitted. Past studies have experimented with panels, individual analysis and group reports, and some of these methods may not be appropriate to the proposed three topics. For example, a. panel may be of little value in evaluating a management process if its members are unfamiliar with the community; however, an appropriately selected panel could be very helpful in judging whether a product presented a, clear message to the policymaker. If'the management process is to be a key area, a working group should also consider using management oriented individuals (such as OMB personnel) who have no vested interests. MANAGEMENT AND ANALYTICAL ISSUES While the issues outlined in the scope of each of your proposals give a broad view of each study's direction, a more detailed list of issues should be for- mulated before a study actually begins. The following are some suggestions. Study Execution Prior to beginning the compilation of information I think a study group should reach some tentative ideas about the scope of their effort. Any pre-study theories should be discussed to decide how to test for them. The time period and the actual intelligence products to be covered (e. g. , NIE's, daily briefings, etc.) should be laid out with special attention to what is being ignored and why. A general format for the report should be discussed. SECRET Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5 Approved For ReI Fe 2001/08/14: CIA-RDP84B00506R0f 00050014-5 SECRET Management Issues An excellent first step to understanding the management of intelligence during a period would be to study the linkage along which information flowed within the community and between the community and the policy- makers. Both the informal and formal aspects of these links could be examined. Attention could be given to the flow of information, feedback and guidance, and to how the flow was affected by the perceptions of dif- ferent participants (either agencies or individuals), by time constraints, and by procedures. However, I recognize that to develop such a detailed perspective is extremely difficult when events occurred months in the past, people have changed, etc. A particularly interesting issue would be to identify the implicit standards which were set for intelligence output (e. g., number of reports, response time to questions, etc.) and how the community measured up to those criteria. Furthermore, who set the criteria and why? Consideration should be given to how the output of the community could have been improved through better management. In the short term this includes examining the role of middle managers in motivating analysts, in communicating needs to analysts, and in matching the analysts' effort with the needs of the consumers. The role of higher level intelligence managers should-be subjected to the same review. (In the long term the working group may become interested in such questions as how training influenced the roles of both analyst and manager and how the system of personnel management and motivation affect their behavior.) Analytical Issues The "quality's of analysis is a frequently discussed topic. In addressing it an effort must be made to define what we mean by quality -- is it a rigorous analysis, one that combines all viewpoints, one that reads well, etc. The criteria of quality may vary markedly between individuals and agencies; in fact the lack of consensus could well be a major part of the problem. Examining the analytical output requires an appreciation for the raw data being gathered aad the validity being implicitly given to it (simply because we collect it covertly does not mean it is accurate, regardless of the SECRET Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5 Approved For RelQpe 2001/08/14: CIA-RDP84B00506R04g:100050014-5 SECRET collection device). The modification in the raw data as it is processed may merit review in order to see how it influences the analyst. The analyst's' work is then perhaps the key to the quality of the product. How does he assemble data? What methods does he use to analyze it -- does he just assemble data and make obvious inferences; does he test different analytical frameworks against the data; what data does he dismiss? In looking at the final written product one needs to consider whether major points are clear, whether they respond to consumer needs, and how the analysts try to communicate the appropriate level of uncertainty. SECRET Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5