Evolution Beats Revolution in Analysis

Counterpoint to "The Coming Revolution in Intelligence Analysis"

Steven R. Ward

Editor’s Note: This article responds to the views on intelligence analysis in the 21st century advanced by Carmen Medina of this issue.

* * *

The recommendations presented in Carmen Medina’s article, “The Coming Revolution in Intelligence Analysis: What To Do When Traditional Models Fail,” come across as solutions in search of a problem. Her description of the Directorate of Intelligence (DI) fails to take into account many of the changes that the DI has made in the past decade. Constructive self-assessments are important to any organization, but this critique of the Directorate’s current practices lapses from examination to undeserved flagellation. The author is well known for having been thoughtfully engaged in seeking ways to improve the DI for many years; therefore, the suspicion arises that she crafted her article with the intent to provoke as much discussion as possible.

In brief, Medina argues that the DI’s current model of intelligence analysis is failing to respond to rapid changes in consumer needs and preferences. She contends that the DI is insufficiently focused on the consumer and is devoting its resources to increasingly useless syntheses of intelligence reporting. She recommends what she calls a revolutionary model that would shift analysis from a focus on day-to-day “developments” to forward-leaning, conceptual thinking that is “less independent and neutral” and more tailored to the policymakers’ specific needs.

The article’s main failing is that its primary contentions fly in the face of history and recent feedback from our consumers and Agency leadership. This essay joins the debate by commenting in general on Medina’s thesis and defending, in particular, the DI’s focus on “develop-ments” as the basis of any successful model of intelligence analysis.


Continuity or Stagnation?

Medina’s description of the DI as stagnating in comfortable stability is demonstrably false. Change has been a constant feature of life in the Directorate in recent decades. The DI has undergone numerous reorganizations and adopted new approaches to the training of analysts to improve products and services. Analysts have developed closer working relationships with their counterparts in the Directorate of Operations, other agencies, and the military. The past decade, in particular, has been rife with change, resulting in greater emphasis on serving our primary consumers, a goal that Medina identifies with her “new” model for intelligence in the future. We have steadily increased our emphasis on the so-called “non-traditional” issues (counterterrorism, counter proliferation, and crime and narcotics), which Medina claims is lacking. Now, in the 21st century, we continue to experiment with ways to do our job and serve our consumers better, particularly at the Issue Group level and below.1 Admittedly, some analytic tradecraft and management practices at the office level and above may appear static, but the existence of some weak pockets does not prevent the continual rise of new ideas for ways to better serve our consumers with both “finished” intelligence (coordinated, reviewed, and polished publications) and less formal products and services.

In asserting that DI work procedures need to change, Medina appears to confuse continuity and reliance on fundamental processes with stagnation. To underscore her point that not enough has changed, she writes that a DI analyst from decades ago would recognize our work processes today. This should be no more surprising, however, than the contention that a carpenter from centuries ago, while probably amazed by power saws, nail guns, laser levels, and new construction materials, would still recognize the basic work performed by today’s homebuilders and cabinetmakers. Some tasks and procedures are best completed in well-established ways. This includes the handling of intelligence by the DI, which, it is probably fair to say, George Washington and possibly Julius Caesar would recognize and understand as consistent with the proper processing of information used for statecraft and military operations. Many fundamental aspects of what Medina describes as the “traditional model” of analysis preceded the creation of the CIA and hold up as solid approaches to intelligence analysis even as our tools change.


Old and New Assumptions

Medina’s breakdown of various assumptions about DI tradecraft into “old” and “new” categories is unconvincing.2 None of the four “old” assumptions—that policymakers need a service to update them on developments and that the DI has unique insights to tell them what events mean—can be dismissed outright as outmoded, although the degree of their applicability may be changing. Her critique also fails to take into account the wide variety of consumers that the DI serves, ranging from the Executive Branch and the Congress to the military and foreign intelligence partners. The old assumptions may no longer apply well to some policymakers at high levels, but they are still valid for many around the policy community. Most of the so-called “new” assumptions—that policymakers already know about events, understand their significance, and have their own access to raw intelligence—have been in play since at least Robert Gates’s tenure as Deputy Director for Intelligence (DDI) in the early 1980s. Again, the degree of application has varied over time, but they are not new, whether referring to our current focus on the President as “First Customer” or our longstanding engagement with lower-level policymakers. To the DI’s credit, it has constantly striven to “optimize” itself, to use the author’s term, against the goals implicit in these assumptions by the changes made over the past decade.

The author’s call for certain analysts to specialize in “complex analysis of the most difficult problems” seems to ignore signal efforts along this line that the DI has already undertaken, such as DDI Douglas MacEachin’s advocacy of structured argumentation through “linchpin analysis” or the creation two years ago of the Senior Analytic Service to promote in-depth work on multifaceted issues. Jack Davis and others have led efforts since the late 1980s to encourage creative thinking and strengthen the Directorate’s customer focus through “alternative analysis,” which uses various methods to deal with uncertainty, and “opportunity analysis,” which identifies levers for manipulation by US policy. Our success, or lack thereof, in these fields may be open to debate, but the concepts are neither new nor exclusive to 21st century analysis.

As for Medina’s recommendation that more attention be given to “non-traditional intelligence issues,” one can only remark that the Counterterrorist Center recently celebrated its 15th anniversary and the Crime and Narcotics Center was created in 1989. Moreover, our analysts were supporting arms control and analyzing arms transfers long before the term “counterproliferation” came into vogue.

Medina’s article proposes that “as policymakers continue to raise the standards for intelligence analysis, we may need to change more than just our assumptions and work habits,” completely revamping “the fundamental characteristics of intelligence analysis carefully developed during the last half of the twentieth century.”3 The suggestions that Medina presents as “heretical” seem, when stripped of dramatic descriptors, to be a mix of the current best practices in the DI. Assigning more positive adjectives to the proposed 21st century model than to the traditional model—“bold,” “complex,” precedent-shattering” versus “cautious,” “hierarchical,” “precedent-based”—may reflect desirable areas of growth for the Directorate, but adds nothing substantial to the case for abandoning our current model.


Revolution or Evolution?

Claims of dramatic shifts in large systems, whether the environment, a national economy, or a US government agency, always need to be viewed with some skepticism. Systems do not change overnight, especially those affected by some of the more immutable traits of human nature. Medina’s claims about a new environment of information abundance radically altering policymaker needs are overstated. They echo much of the “new economy” thinking that, as good as it sounds, is increasingly unconvincing as it has been put into practice. Not too long ago, The Washington Post ran a series of articles on “The Rise and Fall of Michael Saylor,” the Microstrategy chief who became a multi-billionaire, then watched his wealth and his company collapse after bad accounting practices took the luster off his vision of how to handle the new environment of information abundance.4 The series reminds us that untested theories, especially when presented in glowing terms to excite the imagination of investors and managers, often promise more than can be delivered and more than, in practice, anyone wants.5 The DI, like many corporations, already has a good and useful product. When consultants and others come to us saying that everything has changed and so must we, the proper response before investing significant resources ought to be “prove it.”

Undeniably, the author is correct in asserting that more information is moving through more channels of communication today than ever before. As was discussed on the “Friends of Analysis” electronic database in the late 1980s, the DI has been losing its comparative advantage as an information supplier ever since the White House Situation Room was established under President Kennedy. Today’s data abundance needs to be viewed realistically, however: much of the information is redundant, and much of it, to be blunt, is garbage. The DI serves as a useful filter for helping policymakers deal with the infor-mation torrent. No matter how many channels of information exist, a policymaker has at most two eyes, two ears, and 24 hours in a day.

No one disagrees that policymakers benefit from intelligence tailored to their needs. This is not new—debates have raged in the Directorate about how to harness new technology to better serve the policymaker for decades. The issue about the right mix of more policy-linked “opportunity analysis” versus more fact-based “objective analysis”—never truly an either/or proposition—goes back to the Sherman Kent-Willmoore Kendall debate of 1949.6 Moreover, while concepts and ideas that put facts to use are more valuable than facts alone, we have to question whether policymakers will extend a warm welcome to analysts seeking closer involvement in the policy process. Can we be sure that most policymakers want the proximity proposed in Medina’s article? Human nature being what it is, might they not become more jealous and aloof when dealing with policy-related ideas from analysts (secrecy and surprise being valued as much by policymakers in interagency battles as by generals in wartime)?

It is not this article’s intent simply to gainsay everything Medina puts forward. Perhaps a general critique of the article could be summed up as: more proof needs to be shown that the traditional model has failed and that significant change, much less a revolution, in the DI is needed. As I see it, the main problem with the traditional model is that it occasionally—albeit still too often—is inconsistently or poorly applied. Medina’s specific contention that the current model wrongly focuses on events and developments at the expense of consumers, however, calls for a strong counter-argument.


Importance of Focusing on Developments

Policymakers’ intelligence demands in the wake of 11 September 2001 show the enduring value of analysts providing support that, at least in part, responds to the “old” assumptions laid out by Medina: Senior officials needed help keeping current on, and interpreting, “developments” related to global terrorism. In any crisis, our consumers’ need for intelligence support is intensified. Historically, the DI’s first response has been to establish a task force to provide a continuous watch on developments and to report on them in once- or twice-daily Situation Reports. These steps have generally been well received—and often vigorously demanded—by our traditional consumers and the inevitable new users among the military commands and civilian agencies. Task force activities do not reduce other intelligence support. Instead, the assignment of some analysts to focus on breaking events frees other analysts—often those with in-depth expertise—to concentrate on the broader picture, the “hard questions,” and direct responses to policymaker requests.

Between October 2001 and April 2002, analysts working on Afghanistan were inundated with requests from senior policymakers to track developments in the region. The resulting high-level, event-driven “current intelligence” was shown to be an essential, but not necessarily the essential part of a successful model of intelligence analysis. Although details cannot be provided in an unclassified article, the demand for DI updates and interpretation was strong. One can argue that crisis support is sufficiently different from routine support to policymakers that it only temporarily tips the balance in favor of a focus on developments at the expense of the provision of more conceptual, policy-relevant products. In my experience, however, policymaker requests support the view that a focus on developments is fundamental to intelligence analysis under any conditions in any century.


Myth of the Well-Informed Policymaker

As Medina notes, most policymakers are knowledgeable about their areas and are well read on issues relevant to their responsibilities. They also are very busy people, however, who have constant demands on their time and attention. In an era of information abundance, it follows that policymakers will be hard pressed to review and synthesize for themselves all of the available worthwhile information. If, as Medina says, analysts require up to two hours each day to read through the overnight traffic, can busy policymakers cover the same ubiquitous raw intelligence and still read domestic and foreign newspapers and periodicals and work their professional and informal information sources without any assistance from the DI or other Intelligence Community (IC) analysts? Raw intelligence, especially material with special controls, is not widely available to policymakers outside the National Security Council. Can the many policymakers who lack secure storage facilities adequately analyze raw intelligence reports that they may see only briefly and recall and incorporate related information without some help? Finally, can policymakers who travel regularly come up to speed easily upon their return?/p>

For the exceptional policymaker, the answer to the questions above may be “yes,” but most are likely to need help and probably value the ability to draw on DI analysts for updates. Moreover, while the Director for Near East Affairs at the NSC might not read an article in the IC-supported Senior Executive Intelligence Brief (SEIB) except when catching up on developments while he was away from the office, a director covering transnational issues probably would welcome a SEIB article that alerts her to the need to shift her attention to another country within her broad account.7 The CIA’s White House briefers and intelligence representatives assigned to various policy agencies should be specifically asked how often they rely on fact-based, descriptive background notes that provide details about events and their historical context to answer questions by high-level policymakers during their morning briefings. Such metrics are likely to confirm that knowledge about developments is critical to providingstrong intelligence support downtown.

To maintain credibility, an analyst would be wise to be as conversant in developments and disseminated raw intelligence as the policymaker being served. Few things undermine a policymaker’s confidence as rapidly as an analyst’s lack of familiarity with widely available information about developments in countries and issues relevant to ongoing policy discussions. Moreover, keeping current on events provides the foundation for identifying the discontinuities in foreign happenings that the DI needs to highlight to policymakers as potential warning signs of political/economic/military shifts inimical to US interests.


The Case for an Event-Focused Publication

As the only IC-coordinated current-intelligence product, the SEIB tries to serve a wide audience, drawing primarily on raw intelligence of routine sensitivity. Medina accepts that analysts should understand current developments, but she seems to view the recording and tracking of developments in a written publication as unworthy of the DI’s energy and resources. The SEIB is heavy with political analysis, the type of analysis that policymakers need least, in her view, because it is too general and does not carry the policy relevance of memos written in direct response to consumer questions. Her description of the SEIB—and its predecessor, the National Intelligence Daily (NID)—is on the mark, but this is not necessarily a weakness. Because political activity worldwide tends to be more dynamic than economic or military activity, it is logical that a publi-cation operating under the SEIB’s guidelines would be general and political in its focus. To criticize it for not being as specific as requested memos is unfair and overlooks the SEIB’s role in prompting many of those policymaker requests in the first place. In fact, we should welcome such an iterative process of engaging with policymakers.

But is such a broadly focused publication useful? Or is it superfluous to the policy process. Over the years, I have asked policymakers about the value of the SEIB/NID. Virtually no one has contended that analysis presented in the daily publication has directly affected a policy decision. More than a few, however, have observed that the daily publication has often contributed to the tone of policy debate—primarily by capturing developments and recording IC thinking, thus providing a common base of information. Many policy meetings involve large groups of interested officials, ranging from those intimately involved with the specific issue under discussion to those with broad or secondary responsibilities. The SEIB serves as a widely available point of reference on foreign events relevant to the discussion. Seen in this light, the daily intelligence publication seems a lot less dis-pensable than it might look to an individual senior policymaker. A general point of reference such as the SEIB is worth keeping, even as we attempt to achieve more ambitious objectives for the publication under the current or the proposed analytic models.8


Getting From Here to There

No serious student of intelligence analysis can take issue with the contention that analysts should aim to focus first on the customer, concentrate on ideas, and think beyond finished intelligence. The question is how to reach this stage of excellence. The quality of new DI analysts has probably never been higher, but it is still a rare young analyst who has the ability to perform at the level required to support Medina’s proposed model for analysis.

In recent years, the DI’s increased emphasis on training reflected a corporate recognition of the need to build and maintain basic tradecraft skills. Current-intelligence publications, which tend to be event focused, are one of the few remaining ways for analysts to develop such skills and build the foundations for creative thinking. If the Directorate wants all of its analysts to be able to answer policymakers’ “hard questions,” it needs to give them opportunities to learn to trust their “intuition,” which grows out of mastering the details of their accounts and polishing writing and presentational skills. Once seasoned, analysts can turn from devoting time to current intelligence to addressing specific consumer needs. Current intelligence prepares analysts to respond quickly to policymaker requests, easing the jump from the descriptive and general analysis published in the SEIB to the more sophisticated thinking that goes into memorandums, e-mails, and phone responses directly to policymakers.

Devoting time to other serial publications also should be seen as an investment in the future, not a distraction. Top-level policymakers may not miss the developments covered in the Economic Intelligence Weekly or some of the other discontinued serial publications, as Medina points out; however, most branch chiefs and analysts who remember these types of publications regret their passing—or should, if they do not. They were a valuable tool to develop young analysts through self-directed research and peer and manager review. Many of us remember them as valued outlets for exploring ideas and floating new lines of analysis. Like current intelligence, serial publications were primarily descriptive, and thus were also a great repository of reference material. It is not beyond our collective wit in the DI to devise a process that serves our consumers well while allowing our analysts to benefit from the exercise of their tradecraft and, dare we dream, revive some of the fun of writing for serial publications, either as part of the current intelligence process or through the use of serial flyers and other products.


The Traditional Model Can Work

The goals expressed in “The Coming Revolution in Intelligence Analysis” are laudable, but they smack of an unattainable perfection, especially when so-called “revolutionary” changes are the solution. The article distorts our current practices and overlooks the diversity of the DI in an effort to legitimize the proposed new model. This is a weak foundation on which to advance solutions whose unexpected costs could include the neglect or abandonment of useful processes and institutional destabilization. These potential dangers are not, by themselves, reason to reject a revolutionary approach to analysis, but they should impose caution on those calling for radical change.9 For now, the evolution of the Directorate appears to be working well and should not be abandoned for an alleged speedier process without better cause than that presented in Medina’s article.

At its heart, the article is criticizing the DI’s office-based culture, where considerable autonomy can result in components choosing different paths. Many would argue that the problem is that the current model for analysis has not been applied consistently across the Directorate. Many of the shortcomings that Medina lists are the result of this uneven application and the failure to solidify the corporate foundation of basic tradecraft skills. This raises doubts that a new model would be any easier to adopt than the current one. Successful change in any organization requires either a dramatic and widely accepted shift in basic principles or years of sustained attention to shaping processes and values. In both cases, senior leadership needs to demonstrate what it truly values by using the full gamut of its abilities to promote and reward the desired behaviors.

Rather than trying to jumpstart the process of altering the Directorate’s office-based culture with another round of disruptive changes, the DI would be better served by continuing to seek improvements at the margins. Some Issue Groups and Teams have shown that it is possible to achieve an optimal balance between the attention they pay to current developments and customer service, between building analysts’ skills and providing timely and valuable responses to policymakers, and between maintaining their analytic integrity and tailoring support to meet policymakers’ needs. In short, we know what needs to be done and how to do it. The challenge is for our senior leadership to show through its actions that to achieve the most ambitious goals of responsiveness and relevance across the Directorate it will enforce a high corporate tradecraft standard and solidify our foundation of analytic and managerial skills through training, opportunity, and accountability.





1 In late 1996, the DI underwent a major reorganization, which entailed combining several office-level components, converting division-level units into Issue Groups, and calling subordinate branches, “teams.”

2 See pp. 24-25 of Carmen Medina’s article, “The Coming Revolution in Intelligence Analysis: What To Do When Traditional Models Fail,” in this issue (vol. 46, no. 3) of Studies in Intelligence.

3 See p. 27 of Medina’s article in this issue.

4 See the four-part series by Mark Leibovitch, “Microstrategy’s CEO Sped To the Brink,” The Washington Post, p. A01, 6-9 January 2002.

5 A similar example is the case of Priceline, an internet website where customers “named their own price” for airline tickets and other services. Clever marketing and an ability to sell investors on the promise of the internet and the new economy allowed the company to reach a market capitalization worth more than the entire US airline industry! Ultimately, Priceline crashed to earth when investors determined that its business model was not profitable. More recently, a slew of business failures—Worldcom, Enron, and others—have been traced to unsound business, debt, and accounting practices that were hidden within what analysts and the business press lauded as revolutionary.

6 See Jack Davis, “The Kent-Kendall Debate of 1949,” Studies in Intelligence, vol. 35, no. 2 (Summer 1991), pp. 37-50.

7 This example was provided to the author by a former Director of Near East Affairs at the NSC and matches the author’s own experience while serving at the NSC and covering global issues.

8 The SEIB also has value as an intelligence record. In responding to policymaker questions and to taskings to brief congressional and military customers, many an analyst has been thankful for the ability to go back through SEIB articles to recall noteworthy events and speed preparations for these unavoidable diversions from focusing on “ideas” and “hard questions,” as described by Medina.

9 Richard Betts recently provided a useful discussion on the costs and benefits of various approaches to reforming intelligence. See “Fixing Intelligence,” Foreign Affairs, January-February 2002.



Steven R. Ward serves in the Directorate of Intelligence.



Historical Document
Posted: Apr 14, 2007 07:45 PM
Last Updated: Jun 27, 2008 07:13 AM