Preserving Central Intelligence: Assessment and Evaluation in Support of the DCI


after-action assessment unit proposed to support DCI,

Reform of the Intelligence Establishment

Preserving Central Intelligence: Assessment and Evaluation in Support of the DCI

William Nolte

quote Once again, reform of the US intelligence establishment is a prime topic of Washington discussion. We have been here before since the Cold War ended, with a dozen or more studies, official and unofficial, focused on intelligence, its roles, and its functions. These efforts notwithstanding, it can be suggested that the end of the Cold War and the major changes which followed in both the operational and technical environments for intelligence did not produce sufficient momentum to overcome the institutional inertia that favored the status quo in the nature and structure of American intelligence. Moreover, neither the significant (though not as dramatic as some would like to claim) budgetary austerity of the 1990s nor the attacks of 11 September 2001 (remarkably at first glance) produced enough shock to prompt major changes.1

An alternative view is that US intelligence services are significantly changed from their Cold War embodiments, individually and collectively. Future "reform" efforts should take into account the changes that have been made, even as we concede that significant change has not equaled sufficient change, that is to say, change commensurate with the ongoing transformation of the technical and operational environments confronting intelligence. It would be a mistake to enter a process of institutional reassessment on the assumption that intelligence, in its current circumstance, is "all wrong" or that a reform or set of reforms in US intelligence will make things "all right."

Before we choose to abandon, as some have begun to argue, the office of the Director of Central Intelligence, one of the great and most resilient creations of the National Security Act of 1947, we should at least explore options to strengthen and preserve it. This article will focus on suggestions for correcting two related deficiencies in the intelligence establishment, the absence of an effective internal assessment mechanism in service of the DCI and the absence of an equivalent to the US military's "combatant command" structure, which has proven invaluable to the defense establishment over the past half-century.

Developing Lessons Learned: the Truth Shall Set You Free

The propensity for deception, especially self-deception, is a major characteristic of institutionalization, the process by which instruments of policy (or commerce or industry) age and die. Having established a position of leadership in a given market, a once agile and innovative organization begins to get comfortable with that leadership. This phenomenon is a common theme of management literature abundant in airport bookstores all over the world—though its popularization should not obscure its fundamental truth.

Institutional decay, in the private or public sector, need not be a progressive or terminal disease. One remedy for complacency and self-deception used in a variety of organizational and professional settings is a rigorous after-actions or lessons-learned process. Whether in the medical profession (in the morbidity conferences especially common in teaching hospitals) or in the military (through a variety of after-action efforts), critical self-examination can inhibit the tendency to assume that all is well within an organization and can point to possible improvements in process or procedure. This is not a method to be applied only in the event of failure. An effective after-action review can heighten awareness and improve skills even if an operation is successful—and thus turn on its head the old saw, "the surgery was a success, but the patient died."

Most of all, an effective after-action process should be a statement, to those within an organization as well as to those overseeing it from outside, of the essential integrity of the organization and its people. Others may judge us, the process declares, but they do so neither more carefully nor more rigorously than we judge ourselves.

The intelligence establishment has tried this but fitfully. Intelligence agencies, like other public bureaucracies, have inspectors general. But so do the military services, which have to their great credit neither confused inspection regimes with learning processes nor concluded that the existence of one obviated the need for the other. The lessons-learned process may be compatible with and complementary to an inspection process, but the two are not identical. Ideally, lessons-learned processes should be defined as professionals judging themselves as part of a continuous learning credo or the repeated self-application of deeply held standards and values.

The intelligence agencies would not have to look far for after-action models to study and potentially adopt. The US Army's Center for Army Lessons Learned at Fort Leavenworth, Kansas is one, among many in the Department of Defense and the military services. Even without the "lessons learned" label, the war colleges and the various research elements associated with them serve important after-actions functions. In developing its own after-actions capability, the intelligence establishment should not resist drawing on these and other existing institutions for studies, conferences, and other means of internal review.

Where, we need to ask, are the Intelligence Community counterparts to these institutions? Nothing will—or should—stop outside overseers from conducting their own reviews of intelligence performance, either periodically or in light of an event or controversy, but the intelligence agencies would enhance their own standing and integrity by ensuring that the first such study undertaken, especially on issues of great controversy, is its own, conducted rigorously, fairly, and fearlessly.

We have at least considered this in the past, but we have largely found it "too hard to do." The National Intelligence Council (NIC) established a vice chairman for evaluation in the early 1990s. But a decade of experience suggests the creation of the position did not lead to rigorous examination of the organizational and cultural lessons of major intelligence efforts, successful, unsuccessful, or somewhere in between.2

Before-Action Assessment: We Shall Know the Truth . . .

...about ourselves. Effective assessment cannot operate solely as an after-the-fact function. Prospective assessment ("How well do we think we are prepared to deal with ____?") and retrospective or "lessons learned" regimens should be parts of an integrated process. Here the military service model breaks down somewhat. The US Army's Center for Army Lessons Learned is not supplying the chief of staff or the secretary of defense with assessments of the army's ability to perform the missions assigned it in, for example, the Pacific theater. Operational assessment and alignment, in the DOD case, is the role of the combatant commands, another structure without clear analogue in the intelligence establishment.

Just who is the DCI's "go to" official for the Middle East, for Europe, or for counterterrorism? Is the "go to" person for substantive expertise the same as the "go to" person for an assessment of our intelligence readiness on a given area or issue?

In this context, we need to be certain that the question asked is "Who is the responsible official on behalf of the DCI in his capacity as head of the intelligence community?" It is at least possible that the CIA could have a "go to" official and that the DCI would accept that official in all his DCI capacities. It is less certain that the other agencies of the intelligence establishment will accept the "dual hattedness" of a CIA official as the de facto or designated "community" leader in a given area. It is most certainly true that a pattern in which CIA officials assume community responsibility as an ex officio outgrowth of their CIA roles will produce a resistance in other agencies.

The modern US intelligence establishment has struggled with this problem almost from its inception. If the original role of the CIA was to serve as the hub of the wheel of US intelligence, every action by CIA to concurrently develop as a competitive spoke on that same wheel has produced a greater disinclination on the part of other intelligence agencies to accept the integrity of the agency's intended centralizing, coordinating function.

In the area of substantive expertise, one can argue that the NIC, as a body producing coordinated, community-based intelligence estimates came into being in part because by the 1970s it was apparent that the previous Office of National Estimates—as a CIA component—was not accepted by other parts of the community as an "honest broker" in that process. That may only be part of the story, and the NIC may still find it difficult to achieve the status of an honest broker.3 The NIC and the National Intelligence Officers, nonetheless, are and should be part of the DCI's "go to" instrument. They cannot, however, exercise that function in its entirety. The NIOs have traditionally served to bring together the community's analytic expertise. To an increased degree, though, they are now seen as the community's liaison with the academic professions and other experts outside of the community. Is it reasonable to ask them to serve in the additional role of operational "go to" officers? At what point does this look like some old movie describing an attack on a fortress or a castle in which the defenders are constantly scurrying from wall to wall as the enemy storms the ramparts. Though it is at least conceivable that a NIC, expanded in size and authority, could serve all three "go to" roles, for the purposes of this article the NIC would be part of a "go to" network.

An Intelligence Community Assessment Center

What is the solution to the "go to" problem? A step in the right direction would be creation of an Intelligence Community Assessment Center, jointly directed by the Assistant DCI for Collection and the Assistant DCI for Analysis and Production, with the NIC and the Associate DCI for Military Support serving as coordinating partners. The center should have two primary missions—prospective and retrospective assessment—and one secondary mission—conducting research studies directed by the DCI or other senior authorities. Each mission would have a characteristic mode of operation.

Treating the secondary mission first, this should encompass studies that are not especially time sensitive and not connected with an after-action exercise or ongoing oversight of a defined area of mission responsibility. The "study mode" should permit the Intelligence Community to review projects and programs in the same way as the current ADCI/AP staff has supported the development of the National Intelligence Priorities Framework and the Collection Concepts Development Center (CCDC) has explored a number of important issues. Such studies should be undertaken in partnership, wherever possible, with other parts of the community management structure, with the NIC, or with individual agencies. The CCDC and the ADCI/AP evaluation staff should form part of the core of a new Intelligence Community Assessment Center.

Primary Missions: Prospective Assessment

The first primary mission should be assessment of capabilities and resources within various areas of responsibility. Assuming a Pacific AOR, for example, a Pacific Assessments Cell (or staff) should be able to give the DCI and other senior leaders immediate, up-to-date assessments of the strengths and weaknesses of the intelligence establishment's ability to fulfill standing or ad hoc priorities and requirements. Such assessments should cover all "INTs" and the full spectrum of collection and analysis functions. If, for example, the DCI asks "Are we ready to deal with a crisis with North Korea?" the Pacific Assessment Cell should be responsible for the answer. "What are our intelligence strengths?" "Where are we vulnerable?" These and other questions, of course, need to be asked and answered within the context of constantly reviewed requirements and priorities.

The second consulting partner in the assessment center (with the NIC) should be the ADCI/MS, whose responsibilities should be expanded to support the DCI in both his CIA and Intelligence Community roles. While cells in the assessment center would most often be organized to match the areas of responsibility of various combatant commands, not every combatant command should necessarily have an analogous assessment cell. To some degree, distinctive organizational arrangements could even provide different perspectives on an issue or target. But to the degree such organizations are created, zones of coverage need to be explicitly and carefully defined. This is no place for gaps.

Cells answering the readiness question should consider their primary responsibility to be the tracking of shifting policies, priorities, and requirements against available intelligence resources. The heads of the cells, while reporting to the ADCIs and working in cooperation with officials from line and staff elements in CIA and the other agencies, should be indoctrinated with the thought that in the end they are responsible for making the DCI aware of how well US intelligence is equipped and positioned to deal with events in a given AOR. They are the DCI's "go to" officials.

After-Action Assessments

The assessment center should be the DCI's primary in-house capability for lessons learned after a major "deployment" of intelligence resources. What constitutes major? Because the center works for the DCI, with authorities subordinated to the ADCIs, the decision to initiate a lessons-learned effort should come from the ADCIs, with the concurrence of the DCI. In some instances, Operation Iraqi Freedom for example, the amount of preparation required in advance and its clear national security significance would permit the assessment center to begin planning an assessment even before an event takes place. Other events, 9/11 as a tragic example, will not permit such prepositioning of resources or advance thought. The center needs to be prepared to operate in both sets of circumstances.

Employing the assessment center in either of its modes presents significant conceptual and methodological difficulties. It will, moreover, involve the development of capabilities not required of their respective (but separate) counterparts in the military structure. One immediate question is whether the assessment center component responsible for prospective or oversight assessment can evaluate itself in an after-action mode. Would there not be a conflict of interest?

A solution to this difficulty is to have the component or components under review operate as providers of information to a review team drawn from other components, a team that could be augmented by retired or other external personnel. A second guarantee of the integrity of the process should be a senior assessment panel, consisting, ex officio, of the ADCIs, the chairman of the NIC, and the ADCI/MS. Again, the DCI may choose to augment this panel to enhance the credibility of the process. Whatever the membership of the assessment panel, its members should be required to attach their signatures to each major assessment the DCI directs.

The Intelligence Community Assessment Center will not match in size (or rank) the combatant commands. Parity is not the issue here and should not be permitted to become a matter of great concern. The issue is giving the DCI a self-assessment capability that focuses on information—timely, focused, on target—and integrity of process, not one of "my organization is bigger than your organization." In fact, in a networked environment, lack of mass could be an advantage for the Center, fostering agility that can compensate for bulk. The goal is not to build structure; the goal is to enable the DCI to serve the unifying and coordinating function envisioned for his position in the successful vision of national security articulated by the Truman administration at the start of the Cold War.

The vision of 1947 cannot, in all detail, be the vision of 2005 or 2015. But keeping in mind its enduring principles, the vision of an effective national oversight of departmental intelligence is a worthy objective. For all its limitations, the DCI role takes into account the inescapable reality that intelligence must be both departmental and national, a reality made more complex in our day by the need to think of the federal implications of homeland security. Proposals for reform of US intelligence and its leadership need to be examined with this complexity in mind.


1. The end of the Cold War and a subsequent period of relative fiscal austerity forced changes within the intelligence system, masked to some degree by a well-intentioned but sometimes ill-conceived effort to "do more with less." Some of these efforts, consciously or not, had the effect of preserving structures and shielding them from fundamental review. As for September 11, one can argue that it at least postponed some intelligence reform. The war on terror as understood after the attacks of that day was truly going to be a "come as you are" affair, and the slimmed down, incrementally reformed Cold War intelligence establishment responded to the challenge, conscientiously and professionally, but perhaps not efficiently.

2. This staff did provide the basis, in the late 1990s, for a staff for the new Assistant DCI for Analysis and Production, and a vestige of the original survives in the ADCI/AP evaluation staff, which has done enormously important work supporting such programs as the National Intelligence Priorities Framework. It has also been an important contact with the Collection Concepts Development Center. But none of this has taken the form of after-action assessment of intelligence performance, and we need both the will and the capacity to do exactly that.

3. At some point, a future DCI needs to raise the question of whether the NIC or any "community" component, that is to say, any body working for the DCI as head of the community, can be located within the headquarters compound of the CIA. This may be a question of appearances, but appearances count.

William Nolte is Deputy Assistant Director of Central Intelligence for Analysis and Production.


Posted: May 08, 2007 09:01 AM
Last Updated: Aug 03, 2011 02:16 PM