Library

 

Summary

Summary

Observations

A wide range of problems has contributed to the unease currently pervading the Intelligence Community;[1] a significant number of the most serious result from shortcomings in intelligence analysis rather than from defects in collection, organization, or management.[2] The obvious and very public failures exemplified by the surprise attacks of 11 September 2001 and by the flawed National Intelligence Estimate (NIE) of 2002 on Iraqi weapons of mass destruction (WMD) have resulted in a series of investigations and reports that have attempted to identify the causes of those failures and to recommend corrective actions.[3] These recommendations have usually emphasized the need for significant modifications in the organizational structure of the Intelligence Community and for substantial enhancements of centralized authorities in order to better control and coordinate the priorities and funding of community entities. The Intelligence Reform and Terrorism Prevention Act (IRTPA) of 2004, which created the office of Director of National Intelligence (DNI), was based on such foundations.[4]

The logic of this study differs from most of those recommendations with respect to both causes and corrective measures. The key observations in the original “Analytic Pathologies” briefing point in a fundamentally different direction for the root causes of the failures and for fixing the manifest problems. Most importantly, these observations lead to the conclusion that the serious shortcomings—with particular focus on analytic failures—stem from dysfunctional behaviors and practices within the individual agencies and are not likely to be remedied either by structural changes in the organization of the community as a whole or by increased authorities for centralized community managers. Those key observations, which follow, provide the conceptual foundation for this study.


1. There has been a series of serious strategic intelligence failures.
Intelligence support to military operations (SMO) has been reasonably successful in meeting the challenges on the tactical battlefield of locating, identifying, and targeting adversary units for main force engagements. Similar progress in supporting counterterrorism operations has been claimed.[5] At the same time, however, other military and national users have been far less well served by the Intelligence Community across a range of functions. There have been significant shortfalls in support to post-conflict security and stabilization operations and reconstruction efforts in Iraq. Analytic support has also come up short both in accurately capturing adversary thinking and intentions and in providing intelligence that identifies and characterizes developing strategic challenges, such as WMD.[6]

Moreover, within the past decade and a half, a series of intelligence failures at the strategic level, including serious failures in operational and strategic warning, have highlighted real weaknesses at this level and undercut the confidence of principal national users in the community’s capabilities against important intelligence targets. These failures include Iraqi WMD developments (1991 onward), the global black-market in WMD, strategic terrorism (beginning with the attack on the World Trade Center in 1993), the North Korean nuclear program (1994), the emergence of globally-networked Islamic fundamentalism (1996 onward), the Indian and Pakistani nuclear programs (1998),[7] the 9/11 attacks (2001), and Iran‘s WMD programs (2002). Similar failures, as well as an apparent inability to provide accurate assessments and estimates on other important issues, such as the nuclear forces and strategies of China and Russia, affect national users at the highest levels and outweigh any increases in effectiveness at the tactical level.

Indeed, as a bottom-line assessment, this study contends that the Intelligence Community has been least successful in serving the key users and meeting the primary purposes for which the central intelligence coordinating apparatus was created under the National Security Act of 1947.[8] These principal officials are the president and his cadre of senior national security policymakers, not the departmental and battlefield users. As a senior intelligence official recently reminded us, those objectives were two-fold: not only to provide “strategic warning” in order to prevent another surprise such as Pearl Harbor, but also to help head off long-term challenges through a better understanding of the emerging strategic environment.[9]


2. These failures each have particular causes, but the numerous individual problems are
interrelated. These failures did not have a single locus—they occurred in technical collection, human source reporting, and analysis, among other critical functions—but neither do they reflect a series of discrete, idiosyncratic problems. Instead, they resulted from deep-seated, closely-linked, interrelated “systemic pathologies” that have prevented the Intelligence Community from providing effective analytic support to national users, especially effective anticipatory intelligence and warning.[10] The Intelligence Community’s complicated organizational structure and the accreted practices of its analysts have combined to create what Charles Perrow calls “error-inducing systems” that cannot even recognize, much less correct their own errors.[11]


3. The Intelligence Community still relies on the same
collection paradigm created for “denied areas.” Remote technical collection and targeted human access were appropriate means of penetrating denied areas and obtaining critical intelligence against a bureaucratized, centralized, and rigid superpower adversary that exhibited strongly patterned behavior. The problem presented by many of the new threats, whether from transnational terrorist groups or from non-traditional nation-state adversaries, however, is not that of accessing denied areas but of penetrating “denied minds”—and not just those of a few recognized leaders, but of groups, social networks, and entire cultures. Unfortunately, information for intelligence is still treated within the old “hierarchy of privilege” that emphasized “secrets” and was more appropriate for a bureaucratized superpower adversary who threatened us with large military forces and advanced weapons systems.[12] Without refocusing its energies, the Intelligence Community will continue to do better against things than against people.


4. Analytic
methods also have not been updated from those used to fight the Cold War. There were intelligence failures during the Cold War, but the United States and its allies managed to stay on top of the challenge presented by our principal adversary. A relatively stable threat (and consistent single target) allowed the Intelligence Community to foster in-depth expertise by exploiting a very dense information environment, much of which the opponent himself created. That “Industrial Age” intelligence production model—organized for efficiency in high-volume operations and fed by large-scale, focused, multiple-source collection efforts conducted mostly with episodic “snapshot” remote systems that were very good at big fixed targets—built a solid foundation of evidence. This knowledge base allowed analysts to cross-check and corroborate individual pieces of evidence, make judgments consistent with the highest professional standards, and appreciate and communicate any uncertainties (both in evidence and inference) to users. In particular, this dense information fabric allowed analysts to place sensitive intelligence gathered from human sources or by technical means within a stable context that enabled confirmation or disconfirmation of individual reports. As national security challenges evolved during the years following the collapse of the Soviet Union, however, continued reliance on the Cold War intelligence paradigm permitted serious analytic shortfalls to develop.


5. The Intelligence Community presently lacks many of the scientific community's self-correcting features.
Among the most significant of these features are the creative tension between “evidence-based” experimentalists and hypothesis-based theoreticians, a strong tradition of “investigator-initiated” research, real “horizontal” peer review, and “proof” by independent replication.[13] Moreover, neither the community as a whole nor its individual analysts usually possess the ingrained habits of systematic self-examination, including conducting “after action reviews” as part of a continual lessons-learned process, necessary to appreciate the changes required to fix existing problems or to address new challenges.[14]


6. Intelligence analysis remains a “craft culture,” operating within a guild structure and relying on an apprenticeship model that it cannot sustain.
[15] Like a guild, each intelligence discipline recruits its own members, trains them in its particular craft, and inculcates in them its rituals and arcana. These guilds cooperate, but they remain distinct entities. Such a culture builds pragmatically on practices that were successful in the past, but it lacks the strong formal epistemology of a true discipline and remains reliant on the transmission, often implicit, of expertise and domain knowledge from experts to novices. Unfortunately, the US Intelligence Community has too few experts—either analytic “masters” or journeymen—left in the ranks of working analysts to properly instruct and mentor the new apprentices in either practice or values.

 

Conclusions

The Intelligence Community is not normally self-reflective and usually avoids deep self-examination, but recognition and acceptance of the seriousness of its problems by all levels of the community is a necessary prerequisite for true change, including significant modifications to current organizational cultures and ethos. Agreement on the basic diagnosis must, therefore, precede detailed propositions about effective remedies. I suggest that the following six premises, first articulated in the “Analytic Pathologies” briefing, summarize the most important conclusions to be drawn from the preceding discussion of the current enfeebled state of the Intelligence Community.


1. The dysfunctional practices and processes that have evolved within the culture of intelligence analysis go well beyond the classic impediments highlighted by Richards Heuer in
The Psychology of Intelligence Analysis.[16] A more effective analytic paradigm must be built that incorporates the best analytic methods from modern cognitive science and employs useful and easily usable supporting tools to overcome these impediments and prevent them from combining into systemic pathologies.


2. More corrosively, the individual impediments form interrelated, tightly-linked, amplifying networks that result in extremely dysfunctional analytic pathologies and pervasive failure.
A thorough reconceptualization of the overall analysis process itself is needed. The new approach would incorporate a better connected, more interactive, and more collaborative series of networks of intelligence producers and users. In addition, it must be designed to detect and correct errors within routine procedures, instead of leaving them to be found by post-dissemination review.


3. The new problems and circumstances call for fundamentally different approaches in both collection and analysis, as well as in the processing and dissemination practices and procedures that support them.
It is clear that serious problems in the existing organizational structure of the Intelligence Community are reflected in poor prioritization, direction, and coordination of critical collection and analysis activities. However, many problems that are more fundamental and deep-seated exist inside the organizational “boxes” and within the component elements of the intelligence agencies themselves. Fixing these—dysfunctional processes, ineffective methods, and ingrained cultures—is not solely a matter of increased authorities, tighter budgetary control, or better management. A strategic vision that addresses the systemic pathologies, leadership that understands how key functions ought to be improved, and a sustained long-term commitment to rebuilding professional expertise and ethos will be essential.


4. Accurate diagnosis of the root causes of problems “inside the boxes” is required; otherwise remedies will be merely “band-aids.”
For example, the analytic problems occur at and among four organizational levels: 1) individual analysts; 2) analytic units, including their processes, practices, and cultures; 3) the individual intelligence agencies; and 4) the overall national security apparatus, which includes the entire Intelligence Community in addition to the executive bodies responsible for making policy. Solving problems at all four of these interlocking levels requires an integrated attack that includes solutions addressed to the right level and tailored for each problem element.


5. The Intelligence Community must bring more perspectives to bear on its work and create more effective “proof” and validation methods in constructing its knowledge.
It should, in particular, adopt proven practices from science, law, and medicine, including more open communication and self-reflection.


6. Whatever the details of structures or authorities, the new Director of National Intelligence (DNI) leadership must assure that the corrective measures are implemented within each agency and across the community
. Moreover, all this should be done in the knowledge that change will be continual and that there will be no static resting place where the “right” solutions have been found; organizational structures and processes must be designed to evolve with and adapt to that realization.[17]

 

Recommendations

Curing the flaws in intelligence analysis will require a sustained emphasis on rebuilding analytic capabilities, refocusing on human cognitive strengths enhanced by innovative support tools, and restoring professional standards and ethos among the analysts themselves. Most of the recent reform recommendations notwithstanding, more guidelines and tighter management oversight are no substitute for analytic expertise, deep understanding, and self-imposed professional discipline—all achieved not only by formal education and training, but also through assimilation from following experienced mentors. Moreover, neither curiosity nor expertise on the part of the individual analysts can be restored by directives from the top; they must come from an appropriate recruiting profile, effective training, continual mentoring at all levels, time to learn and practice the craft of analysis—both individually and collaboratively—and constraining the “tyranny of the taskings” that prevents analysts from exercising curiosity and pondering more than the obvious answer.[18]

To ensure that the Intelligence Community can provide more effective capabilities to meet the increasingly complex challenges of 21st-century security issues, this study recommends rebuilding the overall paradigm of intelligence analysis from its foundations. The essential components of this effort are:

1. A revamped analytic process;

2. An entirely revised process for recruiting, educating, training, and ensuring the professional development of analysts (including the essential aspect of mentoring);

3. Effective mechanisms for interactions between intelligence analysts and users;

4. A proper process for “proof,” validation, and review of analytic products and services;

5. An institutionalized lessons-learned process;

6. Meaningful processes for collaboration within the Intelligence Community.

Furthermore, although implementing each of these processes separately would produce significant improvements in the quality of analysis, a more effective approach would be to mount a broad-gauged, systematic, and integrated effort to deal with the entire analysis process.

 


Footnotes:

[1]See The 9/11 Commission Report: Final Report of the National Commission on Terrorist Attacks Upon the United States (cited as the 9/11 Commission Report) and Report on the U.S. Intelligence Community’s Prewar Intelligence Assessments on Iraq by the US Senate Select Committee on Intelligence, 7 July 2004 (hereinafter cited as SSCI Report).

[2]See Henry A. Kissinger, “Better Intelligence Reform,” Washington Post,16 August 2004: 17.

[3]For a review of the various commissions that have tackled intelligence reform, see Michael Warner and J. Kenneth McDonald, US Intelligence Community Reform Studies Since 1947. A detailed look at the work of one such recent commission is Loch K. Johnson, “The Aspin-Brown Intelligence Inquiry: Behind the Closed Doors of a Blue Ribbon Commission,” Studies in Intelligence 48, no. 3 (2004): 1–20. Still, there is no guarantee that good intelligence will necessarily help decisionmakers reach good judgments or make good decisions, but poor intelligence can clearly corrupt good decision processes and amplify ill-advised tendencies in flawed processes.

[4]Intelligence Reform and Terrorism Prevention Act, PL 108–458, 2004 (hereinafter cited as IRTPA).

[5]See Testimony by Cofer Black, Coordinator for Counterterrorism, US Department of State, before the House International Relations Committee, 19 August 2004.

[6]It appears, for example, that the intelligence needed to support the security and stabilization operations in Iraq with effective “cultural awareness” during the post-conflict “Phase IV” has been far less than adequate. See comments by senior military officers at a conference in Charlottesville, Virginia, sponsored by CIA’s Center for the Study of Intelligence (CSI). Intelligence for a New Era in American Foreign Policy, (hereinafter cited as Charlottesville Conference Report), 3–5.

[7]Perhaps the more serious error in the case of the Indian-Pakistani nuclear tests was not the failure to predict the timing of the catalytic Indian test (which was really more a failure by policymakers); arguably, it was the failure to estimate correctly the scale and status of the Pakistani weapons program, including its links to the global WMD black market.

[8]Michael Warner, “Transformation and Intelligence Liaison,” SAIS Review of International Affairs (hereinafter SAIS Review) 24, no. 1 (Winter-Spring 2004): 77–89.

[9]See Deborah Barger, “It is Time to Transform, Not Reform, U.S. Intelligence,” SAIS Review 24, no. 1 (Winter-Spring 2004): 26–27.

[10]The systemic pathologies are discussed in detail in Chapter Three.

[11]Charles Perrow, as cited in Robert Jervis, “What’s Wrong with the Intelligence Process?” International Journal of Intelligence and Counterintelligence 1, no. 1 (1986): 41. See also Charles Perrow, Normal Accidents: Living with High-Risk Technologies.

[12]Fulton Armstrong, “Ways to Make Analysis Relevant But Not Prescriptive,“ Studies in Intelligence 46, no. 3 (2002): 20.

[13]“Evidenced-based” analysis is essentially inductive; “hypothesis-based” is deductive; they should be seen as complementary approaches, not competitors for ownership of the analytic process.

[14]For an exception, see John Bodnar, Warning Analysis for the Information Age: Rethinking the Intelligence Process. In fact, both the Joint Military Intelligence College (JMIC) and the Center for the Study of Intelligence have programs to create a discipline of intelligence by bringing together intelligence theory and practice. Regrettably, the results of these efforts have not yet penetrated the mainline analytic units.

[15]In fact, the analytic community self-consciously characterizes its practices and procedures as “tradecraft.”

[16]Richards J. Heuer Jr., The Psychology of Intelligence Analysis. Building on the work on cognitive impediments to human judgment and decisionmaking of Daniel Kahneman, Amos Tversky, and others, in addition to his own long experience as a senior intelligence analyst, Heuer highlighted many psychological hindrances to making accurate judgments by individuals and small-groups.

[17]A medical analogy might make this argument clearer. Although a low-cholesterol diet, proper exercise, routine physicals, a low dose of aspirin, and moderate intake of alcohol may be useful over the long-term for preventing heart disease, patients in acute cardiac distress require more forceful intervention to save them. The measures listed above would have been useful before the attack, and they may be appropriate after recovery, but they are not effective during an acute crisis or in the immediate aftermath, when patients must be kept under observation to be certain they are “taking their medicine.”

[18]Professor Jeffrey Pfeffer of the Graduate School of Business at Stanford University is one of several commentators who have emphasized the importance of “slack” to enable collaboration and collective efforts—including discussion, review and comment, professional development, and service to the “community of practice,” as well as pursuing the scent of curiosity.


Historical Document
Posted: Mar 15, 2007 04:11 PM
Last Updated: Jun 27, 2008 09:57 AM