What’s Next? Recent Works on Improving Intelligence Analysis
Sarah Beebe and Randolph Pherson, Cases in Intelligence Analysis: Structured Analytic Techniques in Action (Washington, DC: CQ Press, 2012), 241 pp.
National Research Council, Baruch Fischhoff and Cherie Chauvin, eds., Intelligence Analysis: Behavioral and Social Scientific Foundations (Washington, DC: The National Academies Press, 2012), 350 pp.
Stephen Marrin, Improving Intelligence Analysis: Bridging the Gap Between Scholarship and Practice (London: Routledge Press, 2011), 192 pp.
Reviewed by Jason Manosevitz
Improving intelligence analysis has become something of a cottage industry during the past decade. The three books reviewed here cover the range of work by scholars and former intelligence officers offering ideas about how to do analysis better. The key questions to ask as this flood of books crests are: what value does each new work add, and does it focus on the right issues?
Sarah Beebe and Randolph Pherson provide a welcome addition to the work on structured analytic techniques in Cases in Intelligence Analysis. Beebe and Pherson both had careers with CIA and have a good sense of intelligence analysis and the pitfalls new analysts are likely to encounter. Cases is aimed at new analysts and strives to teach tools for tackling different kinds of problems. Their book goes beyond similar works focused on structured analytic techniques (SATs) by providing case studies to demonstrate how and when to use different kinds of analytic methods for evaluating information and approaching intelligence problems.
Beebe and Pherson's book is straightforward. Each case provides step-by-step instructions on how to execute one or more SATs relevant to each case. The authors also give an overview of each case's policy context, hammering home the purpose of analysis. Their book will be of most value to those teaching new analysts and analysts wanting to brush up on methods used in the Intelligence Community (IC). Each of the 12 case studies is self-contained, and the examples range across recent domestic and international events. This should appeal to analysts from across the IC, especially those working in agencies with substantial analytical components specifically, CIA, DIA, the Department of Homeland Security, and the FBI.
Some of the names the authors assign to the SATs are over the top, however. For example, starbursting is nothing more than asking basic questions such as who, what, where, when, how, and why. Beebe and Pherson provide a useful chart at the beginning of the book that lists the cases along with the applicable SATs so one can quickly focus on a specific technique. They also provide helpful cross-references to earlier work by Pherson and Richards Heuer, which provides additional information on the various SATs. Many of the chapters, e.g., Who Poisoned Karinna Moskalenko? Is Wen Ho Lee a Spy? Who Murdered Jonathan Luna? The Assassination of Benazir Bhutto, The Atlanta Olympic Bombing, and The DC Sniper seem geared toward political analysts and have a bit of a whodunit feel. Techniques and case studies appealing to economic and leadership analysts would be a welcome addition.
At the opposite end of the spectrum from Beebe and Pherson's how-to guide is Intelligence Analysis, the National Research Council's (NRC) response to a request by the Office of the Director of National Intelligence (ODNI) to leverage academic thinking about how social science disciplines can improve intelligence analysis. The book groups stand-alone chapters by leading academics addressing analytic methods, analysts and their mindsets, and organizations. It provides a handy reference to the range of social science work that could be useful for improving analysis. Most intelligence practitioners and policymakers probably will find the chapters overly theoretical, but scholars studying how to improve intelligence analysis will get a lot out of this volume because it represents the state of academic debate on the subject.
Former Deputy Director of National Intelligence for Intelligence Analysis Thomas Fingar provides the volume's introductory chapter. He surveys the IC and discusses its demographic problems, highlights difficulties intelligence analysts face, and describes a heady world in which intelligence analysts try to meet the demands of busy policymakers.
The authors of each of the remaining chapters cite the foundational literature in their fields and spur some debate among themselves about improving intelligence analysis. Chapters by Edward Kaplan, Bruce Bueno de Mesquita, and Gary McClelland provide rich overviews of organizational research design, game theory, and signal detection, respectively. The main thrust here is to highlight analytic tools, without consideration of the analyst using them.
Bringing analysts and their work groups into the picture, Barbara Spellman, Hal Arkes, James Kajdasz, Reid Hastie, and Catherine Tinsley deliver chapters on cognitive processes and social dynamics. The authors draw heavily from psychology and sociology to address how information interpretation, the desire for more and more information, expertise, and social group dynamics affect analysis.
Baruch Fischhoff, Philip Tetlock, Barbara Mellers, Steve Kozlowski, and Amy Zegart conclude the volume by grappling with the IC from an organizational perspective. They address organizational communication, accountability, human resource management, macro learning behaviors, and difficulties with organizational change. As an example of the academic debates among the NRC's contributors, Zegart squares off against Kozlowski to argue that studies of successful private sector companies should not be applied to public sector institutions (i.e., IC agencies) because those studies suffer from massive selection bias. In other words, studying best practices of certain successful companies may not be useful for improving intelligence because these studies show only what has worked, not why some practices have worked or why others failed.
Stephen Marrin's book, Improving Intelligence Analysis, falls somewhere between the work of Beebe and Pherson and the NRC volume. Most students of intelligence analysis these days are familiar with Marrin's work because he has published a number of articles and contributed to book chapters about improving analysis, and he has been an active leader in the intelligence section of the International Studies Association. Until recently, Marrin taught at Brunel University in the United Kingdom. He also worked at the CIA for about three years in the late 1990s.
Improving Intelligence Analysis is Marrin's first book. It is a combination of previously published articles and new material. The book spans classic topics such as the gap between intelligence scholarship and practice, whether analysis is an art or a science, the training and education of analysts, and the debate about intelligence analysis as a craft or a profession. Marrin delivers an overarching and terse missive to both practitioners and academics: academic scholarship can improve analysis, but only if intelligence analysts know and understand the academic literature. To bring this argument forward, most of the book's chapters read like literature reviews, with long citations from key articles about intelligence analysis written in the last 50 years, with little added value from Marrin.
Intelligence practitioners probably will scoff at much of what Marrin presents. His critiques of analysis focus almost exclusively on the CIA and, in many cases, he refers to dated intelligence products, technology, and processes. This rightfully raises questions about how well he understands how intelligence analysis is done. Academics might complain the book lacks a theoretical thread tying the chapters together. Despite these problems, Marrin's book covers a wide swath of intelligence literature. Marrin also sprinkles throughout his book a key point that one wishes he had developed further. Indeed, it is something of shot across the bow to works like the others reviewed in this article. Marrin points out that no one really knows if the use of structured analytic techniques actually produces better intelligence analysis. He explains there is simply a paucity of data on and study of the results.
Having some method for doing analysis is better than having none, and striving to improve analysis is an important task. All three books add some value in this regard, though Beebe and Pherson's book and the NCR's volume probably provide more substance than Marrin's work. Future research should concentrate on two key areas. Picking up on Marrin's point assessing whether social science methods, and SATs in particular, improve analysis is essential. McClelland's chapter in the NCR volume provides a good starting point with ideas about evaluating analytic performance and tradecraft. This will be a tough task for academics, since all but a small part of the IC s work is classified and even those, like Pherson and Beebe, who teach these methods are unlikely to gain sufficient access to published products to make solid judgments.
Another potentially fruitful area for investigation is how intelligence analysts and analysis are managed. Intelligence managers have virtually no hand in applying SATs or initially drafting intelligence products, but they play a vital role in finalizing products and evaluating analysts. As a result, managerial influence in the analytic process bears examination, but it has been left out of nearly all the recent work.
[Top of page]
All statements of fact, opinion, or analysis expressed in this journal are those of the authors. Nothing in any of the articles should be construed as asserting or implying US government endorsement of their factual statements and interpretations. Articles by non-US government employees are copyrighted and so marked in the table of contents.