What To Do When Traditional Models Fail
The Coming Revolution in Intelligence Analysis
What To Do When Traditional Models Fail
Editor’s Note: This article is designed to stimulate debate. Written and circulated within government circles in 2001, it is presented here for consideration by a wider audience. CIA officer Steven Ward joins the debate with a counterpoint article of this issue.
* * *
The great challenge facing analysts and managers in the Directorate of Intelligence (DI) is providing real insight to smart policymakers. Meeting this challenge is hard, but intelligence officers have long believed that careful attention to the tradecraft of intelligence analysis would lead to work that added value to the information available to policymakers. During its 50-plus years, the CIA, we believed, evolved a model that needed only successful execution to produce quality intelligence analysis. When we faltered, we blamed the analysts (or the collectors), but not the model.
What if the failing, however, lies not with the analysts but with the model they are asked to follow? Customer needs and preferences are changing rapidly, as is the environment in which intelligence analysis operates. Yet the DI’s approach to analysis has hardly changed over the years. A DI analyst from decades ago would recognize most of what a typical analyst does today, from reading traffic to preparing finished intelligence. Stability is often comforting, but in the DI’s case change may be what is most needed.
The Current Model
On the CIA’s public internet website, the DI defines its mission as the provision of timely, accurate, and objective intelligence analysis on the full range of national security threats and foreign policy issues facing the United States. The website outlines the different types of analytic support that might be useful to a customer at any given time. DI officers provide analysis that helps officials work through their policy agendas by: addressing day-to-day events; apprising consumers of developments and providing related background information; assessing the significance of developments and warning of near-term consequences; and signaling potentially dangerous situations in the future.
A key aspect of this model is that it focuses first on developments. In fact, the analysts’ work process is structured around developments. They spend the first quarter or more of their workday reading through the “overnight traffic” to determine what is new. They report what is new to their colleagues and superiors and then often to the policymaking community. The “new thing” may be an event—the death of a world leader or the precipitous decline of an Asian currency. Or it may be an item of intelligence reporting on a situation of interest—from signals, imagery, human-source, open-source, or other type of collection. This basic model has guided the DI’s work for decades.
More recently, DI managers have realized that the specific interests of customers must have greater weight in determining what to do on any given day. As a result, the model has acquired an additional step—understanding customer feedback to determine policymaker interests. This new step, however, merely supplements the pivot around which the analytic work turns—identification of the new development.
Critical, sometimes unstated, assumptions underpin this tradecraft model:
Assumption 1: Policymakers need a service that tells them what is going on in the world or in their particular area of concern.
Assumption 2: Policymakers need help in determining what an event means.
Assumption 3: The CIA and specifically the DI have unique information about what is happening.
Assumption 4: DI analysts are particularly insightful about what these developments may mean.
When Models Fail
Models work only as long as they suit the environment in which they operate. If reality changes, then it is a good bet that the model needs to evolve as well. The DI’s tradecraft model was developed during the 1960s and 1970s and optimized against the characteristics of that period. It was an era of information scarcity— truth about the world’s many closed societies was a rare commodity. Communicating across borders and with other governments was hard—government leaders rarely talked to each other on the phone and summits among world leaders were unusual events. Ideology was a key driver in international relations—it was always important to know how far left or right a government would tack. These traits do not describe today’s environment.
Analysts today have to add value in an era of information abundance. The policymaker, an intelligence consumer, has many more ways of staying informed about recent developments, intelligence-related or not. The responses to a survey of customers of the Senior Executive Intelligence Bulletin (SEIB) conducted in late 2000 are illustrative. When asked to identify the unclassified information sources they relied on, 85 percent of the respondents picked all four of the following sources: foreign newspapers and weekly periodicals; US newspapers and weekly periodicals; their professional networks; and official, informal communications, such as e-mail.
Policymakers today also read raw intelligence reports on a regular basis. Twenty to thirty years ago, analysts in the DI had the fastest access to incoming intelligence information and could count on seeing particularly criticalcables before policymakers. Today, thanks to information technology, policymakers often read the raw traffic at the same time as, if not before, analysts. In a 1998-1999 survey, SEIB customers were asked, “What other sources of daily intelligence do you read?” Almost one-half of the respondents volunteered that they often read raw traffic. Given that “raw traffic” was not offered as a specific choice, the real percentage was almost certainly higher than the write-in responses indicated.
Analysts today have to dig deep to surpass the analytic abilities of their customers. Modern communication technologies and evolving diplomatic practices now allow government leaders to communicate with each other freely and often. US officials even talk to opposition party leaders. This makes it much easier for policymakers to be their own analysts—to gain insights into the intentions of other governments and decipher what developments may mean. The DI has probably always underestimated the extent to which policymakers serve as their own analysts. Arguably, policymakers have never needed the DI to tell them that riots undermine governments or that currency crises shake investor confidence. Today, however, they no longer even need much help deconflicting signals from other governments.
Analysts today have to reach beyond political analysis, an area in which it is particularly hard to provide value to policymakers. The ideological orientation of governments is no longer the important issue in international relations; it has been replaced by a growing list of non-traditional issues that tend to defy ideological definition. In the DI, however, political analysis is still king. We want to follow the ins and outs of political activity in any number of countries even though the audience for this type of analysis is not as broad as it once was. A recent study of articles in the SEIB, for example, revealed that 70 percent dealt mostly with analysis of political developments. In contrast, a much wider variety of issues was covered in memos written directly in response to questions from senior customers. Only about one-third of those memos—whose topics presumably matched what was most on the policymakers’ minds—covered political matters, and many of those discussed the behavior and attitudes of foreign leaders, a sub-category of political analysis that remains of high interest to senior policymakers.
The move toward non-traditional issues is already underway, evidenced by the creation of specialized Centers to deal with terrorism, weapons proliferation, and narcotics and crime. Nonetheless, too many of our flagship products still reflect a political analysis bias. We need to do a better job aligning our publishing strategies with emerging realities.
Analysis in some other conventional areas can still provide value-added, but, like political analysis, the challenge is greater than before. Economic analysis faces daunting competition from the open-source world and those analysts need either to serve consumers who are not economic specialists or to identify niche substantive areas where the Agency can still provide unique support. Scientific and military analyses are borderline issues that defy easy solutions. A number of our senior customers, particularly in civilian agencies, cannot serve as their own experts on technical topics, so there is more room for the intelligence analyst to provide value-added. The issue for military analysis, however, is which agency should be primarily responsible. This is now a crowded field, occupied not only by the DI and the Defense Intelligence Agency, but, increasingly more to the point, by the strong intelligence centers at the unified military commands. The DI is still in the process of defining its comparative advantage in military analysis.
Analysis that Fits the New Environment
So, how does the DI, or anyone, do intelligence analysis in an era of information abundance, wellconnected policymakers, and non‑traditional issues? First, we need new assumptions:
New Assumption 1: Most of the time, policymakers have a good sense of what is going on in their areas of concern.
New Assumption 2: Policymakers frequently understand the direct consequences of events and their immediate significance.
New Assumption 3: The CIA—and particularly the DI—often lacks unique information about developments, especially in the political and economic spheres. Raw intelligence is ubiquitous and can get to policymakers before it reaches the analysts.
New Assumption 4: Policymakers need the greatest help understanding non-traditional intelligence issues. There is still a market for political analysis and certainly for related leadership analysis, but to be successful in traditional areas the DI must generate unique insights into relatively well-understood problems.
A DI optimized against these assumptions would understand current developments, but only as the necessary foundation for its real contribution to policymakers. Analysts would specialize in complex analysis of the most difficult problems. They would focus on the policymakers’ hardest questions. Their goals would include identifying new opportunities for policymaking and warning first of discontinuities that could spell danger.
What does this mean in practical terms? How would the practice of intelligence analysis change?
Analysts must focus on the customer. For many analysts, particularly those involved in political work, the focus would shift from tracking developments in their particular accounts to addressing the specific, hard questions of policymakers. An analyst, for example, would often start her day by reviewing feedback and tasking from customers, instead of first reading the morning traffic. We need to use technology and a network of high-caliber representatives at policy agencies to create stronger links between analysts and customers.
Analysts must concentrate on ideas, not intelligence. Because the DI has no monopoly over the dissemination of intelligence reporting, synthesizing it for others is a poor investment of its time and talent. This particularly applies to political and economic analysis; policymakers do in fact often need help deciphering technical reports on such issues as proliferation and information warfare. In many substantive fields, the DI can best serve the policymaker by tackling the hard questions and trying to develop more reliable ways of identifying and understanding emerging issues. To do this kind of work well, the DI will need keen critical thinkers open to unconventional ideas, perhaps even more than it will need regional experts. Customers are actually pretty good at letting us know what issues keep them up at night; we have to stop dismissing these questions as either too hard or not intelligence-related.
To free analysts to do this work, we will need to de-emphasize products that largely describe what has just happened. This will be hard because there are customers who want such products, which are seen as convenient, free goods. But if our relatively painless experience last year with the elimination of the Economic Intelligence Weekly, a decades-old publication that reviewed economic developments, is any guide, policymaker demand for such products is shallow at best.
Analysts must think beyond finished intelligence. Analysts are schooled in the need to produce validated, finished intelligence—“finished” meaning that it has been carefully considered, officially reviewed, coordinated with colleagues, and sent out under official cover. The main problem is that such products often cannot keep pace with events or even with information sources. DI officers who deal frequently with customers—including those who carry the President’s Daily Brief to the most senior officials—report that many products short of finished intelligence often satisfy the needs of policymakers. These include annotated raw intelligence, quick answers to specific questions, informal trip reports, and memoranda of conversation. Too many intelligence analysts and managers remain fixated on formal products even as policymakers move further away from them in their own work. As anyone who has done a recent tour at a US Embassy knows, most of the real scoop on world events is now exchanged in informal e-mails and telephone calls. Our adherence to the increasingly outdated concept of finished intelligence is what makes the DI wary of such informal intelligence practices as electronic “chat rooms” and other collaborative venues.1
Analysts must look to the Centers as models. If you sit long enough on a DI career service panel, you will still hear some managers say that certain analysts in the Counterterrorism Center or the Crime and Narcotics Center are not doing real DI work. They are producing little in the way of finished intelligence, and they are spending a lot of time doing individual tasks that meet very specific customer needs. Instead of being perceived as outside the DI mainstream, the Centers should be recognized as early adapters of the new model. Their focus on customer requirements, collaborative work, and less formal products speaks to the future.
Now for Something Completely Heretical
As policymakers continue to raise the standards for intelligence analysis, we may need to change more than just our assumptions and work habits. The fundamental characteristics of intelligence analysis, carefully developed during the last half of the twentieth century, may in fact need to be completely rewritten. The transition might look something like the box at the right.
The qualities of “old analysis” are familiar to any intelligence professional. We pride ourselves on carefully basing our judgments on fact, on our expertise, on our ability to warn, and on our neutrality. Some might argue that these are clearly the analytic qualities that must persist under any scenario, regardless of whether we have addressed the needs of our customers.
Perhaps not. To really help smart policymakers, we may need to adopt new practices, new habits of thinking, and new ways of communicating our analysis.
To tell a policymaker something he does not already know, we have to be prepared to take risks in our thinking, to “go to print” with new, adventurous analytic lines before anyone else. This is not always our current style. Almost everything an analyst learns teaches her to be conservative: do not jump to conclusions, consider all sources, coordinate your views with colleagues. At best, an analyst will occasionally lean forward, when in fact she must strive to be several steps ahead of the policymaker on a regular basis.
It is difficult to generate new ideas when you have to stay close to the facts. New ideas are often intuitive, based on one or two stray bits of information that coalesce into new insight. Analysts in the 21st century will not only have to develop their intuition, they—and their managers—will also have to trust it.
Analysts today spend considerable time identifying patterns in recent events and then projecting them onto the future. This is trend analysis. Unfortunately, policymakers who are smart—and most are—can easily do this for themselves. The analysts’ real value increasingly will lie in identifying discontinuities that shatter precedents and trends.
Analysts are often good at identifying what is not likely to work in a given situation; however, policymakers are usually more interested in figuring out what can work. While courses in the Intelligence Community teach analysts how to warn, there are no handbooks on how to identify new opportunities for policymakers.
The most controversial contention may be that 21st century analysts will need to become less independent and neutral in favor of greater tailoring to customer needs. Some critics have already noted that our customer focus in recent years is eroding our detachment from policymaking. The usual answer is to assert that customer focus and neutrality are compatible; but in truth they are not completely. The more we care, as we should, that we have an impact on the policymaking community, the less neutral we become, in the sense that we select our topics based on customer interests and we analyze those aspects that are most relevant to policymakers. Analysts understandably are confused by this new direction. They were taught, they say, to produce intelligence analysis that focuses on events and developments, not customers. It is not their job to worry about whether or not it has impact.
This is the most significant and difficult consequence of working in an information-rich era lacking in significant ideological conflict. Analytic detachment and neutrality are values bred of the Cold War,when foreign policy observers often compensated for lack of information with ideologically based assertions. Intelligence analysts correctly tried not to do that—they were reliably objective.
Being completely neutral and independent in the future, however, may only gain us irrelevance. We need, of course, integrity in our analysis—we must be willing to say things that are uncomfortable for the Pentagon or the State Department and that are not compatible with the goals of policymakers. But we should not pretend that integrity and neutralityare the same thing or that they are dependent on each other. Neutrality impliesdistance from the customer and some near mystical ability to parse the truth completely free from bias or prejudice. Integrity, on the other hand, rests on professional standards and the willingness to provide the most complete answer to a customer’s question, even if it is not the answer he wants to hear. Neutralitycannot be used to justify analytic celibacy and disengagement from the customer. If forced to choose between analytic detachment and impact on policymaking, the 21st century analyst must choose the latter.
1The need to escape the constraints of finished intelligence was highlighted more than five years ago by Carol Dumaine, a DI officer currently leading the Directorate’s Global Futures Partnership, who has written extensively on new models for intelligence analysis. In 1996, for example, in a submission to an in-house electronic discussion database, she noted that the future intelligence officer would “produce unfinished intelligence—all of it on line, interactive, iterative, multidimensional, an interdisciplinary fabric of specialist contributions, and available 24 hours a day to trusted consumers.”
Carmen A. Medina serves in the Office of Policy Support in the Directorate of Intelligence.