A Holistic Vision for the Analytic Unit
Richard Kerr, Thomas Wolfe, Rebecca Donegan, Aris Pappas
“What is needed
is a vision,
from the bottom up,
of intelligence analysis that focuses
In early 2003, Director of Central Intelligence George Tenet asked Richard Kerr, former deputy director of central intelligence, to organize a small group—the authors of this article— to provide an overall assessment of the intelligence produced before the war in Iraq began that spring. After that report was finished in June 2003, the group produced two additional reports dealing with Iraq: a critique of the National Intelligence Estimate on Weapons of Mass Destruction and a report aimed at identifying systemic problems and issues in the Intelligence Community the group uncovered in working on the preceding two reports. The unclassified version of this last report was published in Studies in Intelligence 49, no. 3 in 2005.
These reports were informed by interviews, documents, and other material, as well as by our background and experience as former managers of intelligence analysis. From these studies and from our own past observations and independent experience, we, under the sponsorship of then-Assistant Director of Central Intelligence for Analysis and Production Mark Lowenthal, took a fresh look at the principal components of the intelligence process: requirements, collection, analysis, product, and dissemination. Although this report was prepared in May 2005 and changes have been taking place at many levels in the Intelligence Community, we continue to believe this vision remains relevant today.
—The Kerr Group
Beginning in the late 1970s, the US military entered an era sometimes referred to as a Revolution in Military Affairs. During this period, the military went through a fundamental reassessment of capabilities, force structure, and operations— a process that some argue is continuing to this day. In contrast, although the Intelligence Community also made a variety of changes following the end of the Cold War, they were incremental in nature. They did not fully address longstanding issues, including analysis and products, nor did they tackle emerging problems creatively.
Over the past several years, proposals for improving intelligence have been many and varied. Most have emphasized the overall structure and management of the Intelligence Community, with recommendations aimed at making top-down changes. This paper argues that what is needed is a vision, from the bottom up, of intelligence analysis that focuses on the working of the basic analytic unit. We examine the analytic process, note problems and issues, and make recommendations to enhance the Intelligence Community's analytic capabilities and products.
The Holistic Analytic Unit
The advent of a Director of National Intelligence and changes mandated by commission reports on the performance of the Intelligence Community present unique opportunities to apply a new framework for intelligence analysis. Herewith is a vision for an approach that creates analytic units with a holistic view of their mission, responsibility, and capability. They will comprise physical units at their core and virtual units with presence throughout their areas of responsibility.
Implementation should begin with a single country and then expand region-wide. Once decided upon, changes should be made quickly, and high-level attention and enhanced resources will be key. The individual steps of the process should be undertaken simultaneously rather than serially.
Identify six to 12 countries or areas of particular importance to the US. Pick one or two, perhaps Iran and North Korea, as test cases. Create analytic units for the test case countries with the following characteristics:
Internal expertise, mixed with strong abilities to identify and use knowledge not resident in the unit. Avoid the myth of “total resident knowledge”
Very senior leadership, with rich resources in personnel and funding, to include significant amounts of external contract money, with contracts developed and approved within the unit
Creativity the key
Responsibility for the “whole.” Units should:
Produce current intelligence and long-term estimates
Identify intelligence requirements
Establish collection priorities
Manage IC funding directed against the target
Non-traditional staffing. Units should include or have close relationships, including formal contracts and informal contacts, with:
Experts without security clearances, including non-US citizens
Private sector firms and Federally Funded Research and Development Corporations for administration and substance
Universities and other seats of knowledge
Self-contained assets for research assistance, contract management, conference organization,
administration, and security
Embedded representatives from key organizations and customers
Strong external presence to ensure that the unit is regarded as a central player in the preparation of dynamic assessments and the application of existing knowledge
Assign personnel to other principal organizations in the area of responsibility, including Defense, State, pertinent Federal and NGOs, academic and private entities
Institute regular conference calls, videoconferences, visits, and other interactions with country teams, chiefs of station, national laboratories, military commands, State desk officers, and collection agencies
Preside over programs sponsoring in-country research, academic exchanges, student programs, conferences, and other efforts
New products and state-of-the-art dissemination systems should produce intelligence on a near-real-time basis keyed to customer interests and designed to provide reference material to support current issues
Intelligence estimates should be short, validated outside the IC, and focused not on single-point outcomes but on the implications of change
Strong, high-level review, accountability, and measurement of performance to ensure against backsliding
[Top of page]
Requirements and Collection
Fundamental to the success of intelligence analysis are robust, flexible collection strategies guided by analyst input. In fact, too often today collection drives analysis rather than the other way around. This is due, at least in part, to the separation of collectors from analysts. Accordingly, collection priorities often do not reflect the true needs of the analysts working important issues.
Collection of information on difficult targets is a core mission of intelligence, and neither clandestine nor technical collection measures are up to the challenges of today. The key issues facing US national security over the next decades include the political, economic, and social strains in key countries and the ability of countries to develop and deliver destructive weapons. Experience in Iraq shows that technical, and even clandestine, reporting provided only superficial information on weapons programs, with little or no insight or understanding of the inner workings and dynamics of the programs. In fact, it can be argued that information from these sources sometimes was as misleading as it was at times valuable.
Such issues raise questions about future investment priorities. It is inevitable there will be intense competition for resources among collection disciplines, and a careful review is needed of SIGINT, IMINT, and HUMINT relative to the resources devoted to them. In addition, the value-added and the relative merit of each source must be examined. The involvement of intelligence analysts in such a review will be key to its success.
A productive relationship between collectors and analysts must still be created, and when it is, it will be fundamental to establishing collection priorities and resource allocations. Currently, however, there is a significant gap between them. Too many analysts do not understand collection capabilities, and many are not even familiar with collection systems. To a significant extent this has resulted from the reduction over the past decade of the professional collection management cadre capable of integrating human, imagery, and signals intelligence capabilities into coherent strategies and closely tied to the analysts. This development has been compounded by the separation of collection professionals from the analytic cadre who had been intimately involved in identifying and ranking collection gaps and developing collection strategies.
Although many analysts have contacts with collectors, it is not at a level that furthers their knowledge of collection capabilities or what collectors are collecting or not collecting. Moreover, analysts generally are not very adept at anticipating collection needs. They tend to be reactive, focusing on existing issues rather than identifying emerging issues or those likely to emerge down the road. In part this results from the absence of coherent research programs to help stimulate sound collection strategies. Many issues can be anticipated and collection requirements and strategies established before issues become the focus of policymakers’ attention.
One response to the analyst/collection problem is to have collectors embedded in analytic units. Although this has been tried in various ways over the years, it has been haphazard and with only a collector or two for short periods. Although not all collection platforms are right for all issues, expert representatives from appropriate collection entities (NSA, NGA, OSC, etc.) should be permanently integrated, on a rotating basis, into key analytic units. Understanding that finite personnel resources would preclude this from being done for every analytic unit, it could be done for at least high profile issues, for example, Iran.
For targets and issues of lower priority, an embedded collection generalist could substitute for the several experts representing particular collection agencies/capabilities in high profile-units. That individual should be familiar with all collection systems and not just the one in which his/her expertise resides. Moreover, regardless of the issue’s priority, the collectors must work intimately with analysts on developing strategies and filling gaps, as well as on educating analysts on how system capabilities can or cannot contribute to the questions raised by the issues at hand.
Too often, however, collectors argue that dispersal of personnel to analytic units diminishes the benefits obtained from organizational purity, i.e., the interaction among personnel working the same issues. This argument may have some validity, but organizational purity cannot carry the day if the Intelligence Community is to avoid the weak and unimaginative collection strategies that prevailed in the lead-up to the Iraq war.
Source descriptions and reliability remain serious problems, and only a closer association of analysts and clandestine services will resolve them. With respect to US clandestine reporting in particular, more incisive analyst involvement in establishing the reliability of sources is essential if analytic products are to reflect the actual quality of intelligence information and evidence. Although collection itself is a problem, analysts often must rely on reporting whose sourcing is misleading and even unreliable. US clandestine reporting still too often uses different descriptions for the same source, leading analysts to believe they have corroborating information from more sources than is actually the case. More recently, obliquely worded caveats have been put on reissued reports that do not appear on earlier issuances of the same report, further confusing analysts in determining how much confidence they should place in the reporting.
Analysts and collectors need effective mechanisms for establishing collection strategies, vetting ideas, discussing issue priorities, and identifying emerging problems and likely customer interests and concerns. The Intelligence Community Hard Target Boards seem, for some countries, to fill this role. Recognizing that not all boards are equal, some manage to accomplish these important tasks across agencies. A board’s effectiveness, however, seems to a large extent dependent on the depth and breadth of expertise and leadership capabilities of the chairman, usually the issue manager. The Hard Target Board concept could be strengthened by ensuring that representation is at the right level and that participation is taken seriously. Finally, the chairman must indeed be respected in the community as the foremost expert in the substantive area and have the leadership skills to capably guide the participating agencies.
In addition, new approaches to information collection must be given high priority. A “soft” intelligence collection program should be developed. For example, there is need to better exploit information obtained from a country’s elites—academics, politicians, businessmen, clergy, and the myriad other groups that make up a complicated society. In the case of Iraq, such information would have helped analysts better understand the context in which seemingly threatening developments were unfolding.
A system for collecting, reporting, and disseminating this type of information, similar to that used by the clandestine service, needs to be established, as well as an appropriate assessment process. The State Department, as well as the business and scientific communities, must be involved. This soft intelligence collection effort should not be accomplished separate from the basic analytic unit but should be a direct part of the unit’s activities.
[Top of page]
The Analytic Unit: Business Practices for a High-quality Product
Country and regional analytic units, because of their expertise, need to again lay claim to primary responsibility for all facets of their areas. They have historically been better at integrating disciplines and providing products with perspective and context than have single-issue entities. Moreover, this also would address the difficulty of dealing with the intersection of issues, where the Intelligence Community has not been particularly effective. Regional units are better positioned than single-issue units to anticipate the global impact of regional/intersecting issues, such as the global impact of economic growth in India and China.
Leadership of these units should reside in very senior personnel with well-established expertise. Senior and experienced leadership of regional units must be a high priority. Special salaries must be authorized for key personnel, and incentives for expertise and remaining in a particular substantive area must be better than the incentives to become, for example, a PDB briefer. That means a new reward system, substantial bonuses, and enough backup so that both senior and junior analysts have time to travel, attend conferences, and have opportunities for short assignments in target countries. Much greater emphasis needs to be placed on the management of analysis.
The unit’s resources should be unconstrained, and creativity and high-quality analytic products must be the focus. In other words, much greater emphasis needs to be placed on the supervisor as a driver of analytic quality. Less attention should be given to the mechanics of the production process, such as fitting intelligence pieces into prescribed formats. High-quality, continuous mentoring of analysts is essential to ensure they have the capability and the confidence to reassess comfortable assumptions in the face of contrary evidence. To accomplish this, in addition to capable leadership, the unit must be intimately involved in requirements, priorities, and funding directed against the target.
At the same time, country and regional analytic units must overhaul their business practices, particularly with regard to outreach. They must engage in aggressive and extensive consultation with other intelligence organizations, collection entities, foreign liaison, military commands, academic experts, and chiefs of station, and ensure comprehensive exploitation of open source information. The means of consultation should include conference calls, extensive use of videoconferences, regular face-to-face meetings, frequent exchanges and communications via fax and e-mail, and other means that further the sharing of information and ideas among entities that provide value added to the analysis of priority issues. These activities cannot be casual or episodic, as is now too frequently the case, but must be routine and built into the daily and weekly business practices of each unit.
The objective is not only to create expertise inside the unit but also to educate analysts about where expertise and information reside outside the unit. The unit should not aim to become the repository of all expertise and knowledge. A major push should be undertaken to contract with the academic and private sectors for research and analysis on specific countries, regions and issues. A satellite company outside the classified environment should be established where foreign nationals could be housed to support translations, assist in research projects, and generally support unclassified analysis. Either as part of this organization or separate, it would be important to develop internet and open-source teams to search the net, sample or monitor bloggers, exploit academic and guild journals, and ensure knowledge of a country’s political, social, and economic life. The country or regional analytic unit could profitably make use of one or several private-sector firms or even establish its own federally funded research and development corporation.
Routine administrative chores need to be reduced, with the focus on the business of intelligence— the production of quality analysis. Significant human resources, such as research assistants, contract personnel, conference organizers, and administrative and security officers, should be attached to the unit. External contract money should be made available with contracts developed and approved inside the unit.
[Top of page]
The press of writing for current products and addressing daily customer demands, in other words getting the job done, are the usual reasons cited for not pursuing new business practices and products more aggressively. In addition, the siphoning off of personnel and expertise to staff single-issue centers has further hampered the ability of regional and country analytic units to be the centers of primary responsibility for their areas. The advent of an increase in analytic personnel would seem to reduce the legitimacy of the first claim and mitigate the second problem, providing new personnel are directed to country and regional analytic units and not dispersed elsewhere.
The Intelligence Community’s use of single-issue centers, offices of functional expertise, and crisis-response task forces may satisfy a political or substantive need, but they have a downside for the analytic product. Entities such as those focused on weapons proliferation, drugs, economic crime, and particularly terrorism provide an important focus for analysis, policy development, and action. These issues are most effectively addressed, however, in a country or regional context.
An examination of pre-war intelligence on Iraq revealed systemic analytic problems that resulted from the separation of technical and regional analysis. Intelligence produced on the technical and cultural/political areas was largely distinct and separate, with little attempt to examine the impact of one on the other. In the end, technical analysis came to dominate. Thus, perspective and a comprehensive understanding of the Iraqi target per se were lacking.
Stripping expertise from regional offices to staff these entities, along with the continuing emphasis on current intelligence, diminishes the ability to provide perspective and context for issues and too often leads to analysis narrowly focused on only part of an issue. For example, a piece written by a functional office on Iran’s nuclear capabilities most likely would have little or no political context. Yet, such technical capabilities derive from the country’s political policies, which are developed in a regional, if not an international context. Such narrowly focused analysis forces the policymaker to knit together separate products to provide context and perspective for the issues to be addressed. This is something most policymakers are unwilling or unable to accomplish and, if it is done, it usually results in support of an already established policy objective.
Moreover, fragmentation of intelligence issues creates coordination problems that lead to products that often become watered down to meet the demands of an ever-larger number of components. It also leads to duplication, confusion, and misuse of scarce resources. To wit, the violence in Iraq has been characterized as terrorism by a center and as an insurgency by the Iraq office. The same violence should not be separated into two baskets of responsibility, running the risk of analytic units providing confusing, if not conflicting, analysis to the policymaker.
Functional expertise should be collocated with regional expertise, if not wholly, then at least with some representative experts. Offices that focus solely on functional/technical issues are necessary to the analytic process, but their narrowly focused intelligence should be integrated into pieces with the wider perspective produced by regional or country analytic units. Again, it was the focus on functional/technical intelligence absent the political/cultural/social context that proved so misleading in the Iraq situation.
[Top of page]
The Analytic Product and Dissemination
Country and regional analytic units should have responsibility for all intelligence production, to include current intelligence, research, estimates, and policy support. They should have the staffing and support and aggressively work to establish themselves as experts or to have ready access to experts in all facets of all issues in their areas of responsibility. They should have the necessary resources, editorial capability, and authority to publish without a complicated review process. There should be flexibility in the types of products. Research pieces, even if not published, should be required because they build the depth of expertise that does not come from a focus on current intelligence. Estimates should be short, validated outside the Intelligence Community, and focused not on single-point outcomes but on the implications of changes in conditions or different outcomes.
Emphasis on in-depth research is essential to the development of country and regional expertise, which enables the analyst to provide perspective and context. Although some interviewees claimed that more research is being accomplished than commonly believed, the preponderance of analyst activity continues to be current intelligence and policy support. In fact, some managers described analysts as “action junkies.” That is, locked into the current intelligence process, they know bits and pieces and can answer discrete questions, but they lack sophisticated contextual knowledge.
The fast-paced world of current intelligence leaves little time for careful examination of assumptions, alternatives to accepted lines of analysis, or discussion of sources and evidence. Moreover, quick, rapid-fire responses to policymaker queries often give the impression of certitude about analysis and sources that discourages thoughtful examination of the analytic line. This was one of the chief problems evidenced in the examination of the analysis on Iraq.
Quite apart from content, products intended for policymakers are too rigid in form, format, and function. Some products are required to be written even when there is nothing to say or when the intelligence fails to meet policy thresholds. This diminishes the quality and impact of the published product. If intelligence does not rise to the presidential/VIP level of interest, then the analyst’s time might be better spent on deepening expertise. Lack of an intelligence input will not cause the policymaker to assume that he has been forgotten or that the analysts have disbanded.
Some different perspectives on analytic products need to be developed. A careful review of an approach originally developed by the CIA in the 1980s for a serial publication that focused on political instability would be useful. This publication used specific indicators to monitor subtle changes in the social, political, economic, and military climate in key countries. Produced by a unit dedicated to analytic methodologies and international issues, the quarterly was based on country analysts' assessments of prospects for regime or major policy changes over particular periods of time. It provided regular, systematic assessments of recent developments affecting the stability of those countries. Both policymakers and collectors of intelligence found the indicators approach useful, the former for spotting trends and the latter for tasking assets. Such an approach, combined with a sophisticated set of polls—using those already underway or polls structured inside the country or regional units—would help identify social change and conflict. Combined with attendance at international conferences, in-country research, academic exchanges, or student programs, this type of intelligence could provide valuable insight supported by data.
New forms for disseminating analysis need to be developed. In addition to printed publications, an official but less formal way of communicating with policymakers is warranted. One approach may be an e-mail type system or other means of electronic communication. Although not everyone should be allowed to e-mail policymakers, electronic systems would enable appropriate personnel to send timely, informal messages on important issues.
On a broader level, contracts should be let for the development, or at least a feasibility study, of a near-real-time system with online support to the customer. Such a system should merge current intelligence with direct reference to research products, and it must allow for quick response to customer questions. An intelligence professional should be available for major customers to provide assistance in tailoring the products to their needs.
[Top of page]
Tradecraft and Training
Tradecraft training should play an important role in providing opportunities to examine and try new approaches to intelligence analysis, but rigid products and procedures are often resistant to new ideas. Programs focused specifically on critical thinking skills and analytic tools have recently been introduced into the Intelligence Community training curriculum. Although analysts generally praise the new training programs, they find many of the lessons from courses in analysis, writing, and production difficult, if not impossible, to apply in the real world. Too often the training is rendered irrelevant by inflexible product formats, writing styles, and content requirements that cannot accommodate newer presentational concepts, sophisticated analytic thinking, and alternative analytic approaches. Moreover, some products are packaged or even written by editors whose primary interest is in making the analysis fit the current format. High-level managers, often not exposed to newer training courses, frequently claim that it is risky to apply lessons from training that change the products too radically. Finally, new concepts have traditionally been introduced at the working level with no attempt to hold managers accountable for their implementation or success.
Some training specialists and senior managers argue that methodologists and tradecraft experts should be embedded with key analytic units as a way to promote analytic rigor. They believe that this approach would help mainstream the analytic techniques taught in the classroom and inculcate critical thinking skills into the day-to-day work of the unit. The success of this approach, however, depends on convincing skeptical managers and analysts that such techniques are valuable tools that should be applied routinely to their issues.
Ever watchful for ways to enhance a unit’s capabilities, scholars-in-residence have attained a certain cachet among many managers of analysis. These academics are seen as bringing substantive expertise, broader perspective on issues, and teaching skills, all of which should inform and enable analysts to produce better intelligence. In fact, the experience with these scholars has not been uniformly favorable. It is often difficult to find the right person who understands and can work within the intelligence environment and who is capable of producing actionable intelligence assessments rather than academic treatises. If the right fit can be found, the rewards can be enormously positive. Nonetheless, sending analysts to a university for a course or a semester can be an easier, less expensive, and more effective approach.
In the last analysis, the most important issue with respect to training is the sustainability of the analytic techniques and tradecraft that are taught in the classroom. They must be reinforced in the work place, and first line supervisors must take the lead in doing so.
[Top of page]
The intelligence world is one of ambiguity, nuance, and complexity. Dealing with these elements is difficult in the world intelligence serves, where success or failure is the uncomplicated measure by which the Intelligence Community is judged. Serious shortcomings in collection, inadequate use of outside expertise and knowledge, the lack of exploitation of open source intelligence, and the emphasis on current intelligence have been the result of well-intentioned attempts to do the best analytic job with the resources provided.
The US Intelligence Community is robust, highly capable, and thoroughly motivated and represents an invaluable asset to the nation and its citizens. Nonetheless, the community must be sufficiently mature to both adapt to the changing circumstances and counteract the evolutionary processes that have conspired to threaten its reputation and its ability to successfully perform its mission. The alternative is unacceptable.
All statements of fact, opinion, or analysis expressed in this article are those of the author. Nothing in the article should be construed as asserting or implying US government endorsement of an article’s factual statements and interpretations.
[Top of page]