The Challenge of Managing Uncertainty
To Paul Wolfowitz, former Under Secretary of Defense for Policy, the essential challenge for policy officials is to make sound decisions amidst inherent uncertainty about the character of pending threats to and opportunities for US security interests. To succeed in these circumstances, policymakers must become, in effect, the senior analyst on their core accounts. Above all, they must become adept at the analytic techniques for doing battle with incomplete information and contradictory assumptions.
Policymakers need support from intelligence to help deal with uncertainty. Thus, policy officials come to respect and rely on analysts and managers who appreciate this aspect of the decision process. Analysts and their analysis are deemed most useful when they:
- Clarify what is known by laying out the evidence and pointing to cause-and-effect patterns.
- Carefully structure assumptions and argumentation about what is unknown and unknowable.
- Bring expertise to bear for planning and action on important long-shot threats and opportunities.
By the same standard, the heavily engaged policymaker has little use for intelligence products that emphasize prediction over explanation and opinion over evidence. The policymaking process is particularly ill served by assessments that trivialize the challenge of uncertainty by burying honest debate in compromise language and by ignoring high-impact contingencies.
Ambassador Wolfowitz believes effective management of uncertainty and related challenges to sound decisionmaking requires close cooperation between policy and intelligence officers. The analyst and the collector have to know the operational agendas of policymakers and to understand the continuous and largely informal processes by which they come to decision. Similarly, policymakers have to get close enough to intelligence to provide direct guidance to the collection and analytic processes.
Both the policy and the intelligence sides suffer, as does the national interest, whenever principles or practices are allowed to interfere with close professional cooperation.
This article on the views of Amb. Paul D. Wolfowitz is the second in a series by the author on what ranking officials of the administration of President Bush believed worked well in intelligence-policy relations, what did not, and why. The views of Amb. Robert D. Blackwill, Special Assistant to the President and Senior Director for European and Soviet Affairs, National Security Council Staff, were published in "A Policymaker's Perspective on Intelligence," Studies in Intelligence, summer 1994. Those of Amb. Herman J. Cohen, Assistant Secretary of State for Africa, will be published in a forthcoming issue of Studies.
Ambassador Wolfowitz is now Dean of the Paul H. Nitze School of Advanced International Studies, The Johns Hopkins University. In February 1995, President Clinton appointed Ambassador Wolfowitz to the Commission on the Roles and Capabilities of the US Intelligence Community.
The author interviewed Ambassador Wolfowitz in December 1994 and elicited additional views during February-March 1995. The article also reflects informal remarks Ambassador Wolfowitz made on intelligence and policy in two group meetings the author attended during 1994 and a short essay the Ambassador drafted on the issue, also in 1994.(1)
Focusing on Uncertainty
Ambassador Wolfowitz received his bachelor's degree from Cornell University in 1965 in mathematics and the physical sciences, his initial intellectual passions. He soon switched to what he saw as the more challenging field of political science. His graduate studies at the University of Chicago in the late 1960s, under Professor Albert Wohlstetter, focused on decisionmaking in national security affairs. In studying critical decisions made by US presidents, including Lincoln and Truman, he was struck by how much tougher the challenge was when the outcome could not be known than was allowed for by scholars who made judgments on presidential decisions with the benefit of historical perspective.
By the early 1970s, Ambassador Wolfowitz had concluded that the arms control policies of Democratic and Republican administrations alike did not reflect adequate rigor in taking account of uncertainty about Soviet strategic military doctrines and plans. In his view, "systems analysis" and other decision tools had given policy officials and their staffs an exaggerated confidence in their ability to understand and prepare for the Soviet strategic threat.
Next, his service in the Arms Control and Disarmament Agency during 1973-77 led him to question the analytic methodologies of CIA and other Intelligence Community components regarding the USSR's strategic military goals and game plan for various arms control negotiations. On the issues he studied personally, he concluded that intelligence analysts were working with thin evidence about Soviet intentions, were projecting American goals on the Soviets in a process of "mirror imaging," and were not paying adequate attention to the full range of plausible interpretations of Moscow's goals and tactics.
In Ambassador Wolfowitz's view, intelligence analysts dealt with what could not be known about Moscow's strategic intentions by promoting a single interpretation that consciously or unconsciously applied the biases of US policymakers with a political interest in understating the Soviet threat.
In judging Soviet plans for nuclear missile systems, for example, intelligence analysts predicted the retirement of intermediate-range systems similar to ones that the United States had decided to retire from its own inventory as obsolete. As it turned out, the Soviets instead subsequently modernized their intermediate-range missiles and made them a major new threat to US Allies and forces in Europe.
On this and like subjects, Ambassador Wolfowitz believed that US intelligence analysts and decisionmakers faced critical and compound uncertainties requiring carefully structured argumentation of various plausible alternatives. Instead, intelligence analysts submerged the uncertainty into carefully crafted compromise language that supported current US policy.
He compared these practices with the authority assumed by a priesthood to promote certain views and constrain others without suffering any questions about the commandments on their tablets.
In 1976, Ambassador Wolfowitz was selected as a member of the so-called Team B, which challenged the expertise, methods, and judgments of Intelligence Community analysts working on Soviet strategic military objectives (specifically, National Intelligence Estimate 11-3-8 for 1977). Although part of the motivation for the Team-B challenge may be found in ideology and politics, it had the effect, in his view, of forcing analysts to be less casual about uncertainty and policy biases and more self-conscious about their methodology and assumptions.
The B-Team demonstrated that it was possible to construct a sharply different view of Soviet motivation from the consensus view of the analysts and one that provided a much closer fit to the Soviets' observed behavior (and also provided a much better forecast of subsequent behavior up to and through the invasion of Afghanistan). The formal presentation of the competing views in a session out at [CIA headquarters in] Langley also made clear that the enormous experience and expertise of the B-Team as a group were formidable. Unfortunately, the bureaucratic reaction to the whole experience was largely negative and hostile.
Intelligence and Policy
Ambassador Wolfowitz subsequently served as Deputy Assistant Secretary of Defense for Regional Programs (1977-80), State Department Director of Policy Planning (1981-82), Assistant Secretary of State for East Asian and Pacific Affairs (1982-86), Ambassador of the United States to the Republic of Indonesia (1986-89), and Under Secretary of Defense for Policy (1989-93).
In these positions, he continued to hold critical views of what he saw as unhelpful intelligence dogmas and practices. At the same time, he came to appreciate how important properly conducted collection and analysis were to the success of the policymaking process.
Please don't put me down as an intelligence basher. I could not have achieved what I did without the first-rate intelligence support I often received. Even the much-maligned "weekly reader" [National Intelligence Daily] is useful, because it covers events that don't make it into the newspapers.
In distinguishing helpful from unhelpful intelligence analysis, Ambassador Wolfowitz elaborates his views on the challenge of uncertainty in decisionmaking.
Uncertainty about the meaning of events and especially about prospective threats and opportunities complicates every policy decision. On a good day, you deal with 60-40 odds. Most of the time it is much less clear-cut than that.
In his view, moreover, the serious policymaker cannot ignore a 10-percent likelihood that could have a major impact on US security, much less a 40-percent likelihood. Throughout the Cold War, much of US defense expenditures were directed to the highly unlikely prospect of a Soviet nuclear attack. Policymakers, individually and collectively, have to grapple with resource planning and deployments based on a complicated calculus concerning not only odds, but also interests and resources.
Successful policymakers do not delegate this challenge either to intelligence analysts or to their own staffs. According to Ambassador Wolfowitz, the policymaker has to be the analyst of last resort in making assessments for the President and other principals.
That said, the policymaker as assessor of foreign countries and challenges needs all the help he or she can get.
Artificial separation of intelligence and policy, in contrast, serves only to degrade the performance of both systems.
Great harm is done if differences in professional values cause the two groups to avoid close contact.
How to cooperate? Ambassador Wolfowitz believes:
Intelligence production should be driven by the policy process.
In addition to knowing the planning and action agendas of their core policymaking clients, this requires the intelligence professionals to understand the decisionmaking process, including the fact that the process is continuous, mostly informal, and somewhat untidy. To this extent, formal intelligence papers do not have the same impact as informal person-to-person exchanges, during which the policy official can "cross-examine" the intelligence expert. On the policy side, too, formal planning papers do not always carry the importance intelligence analysts ascribe to them.
There is still another requirement for effective relations: Intelligence organizations have to make their own processes transparent to policymakers. Rather than polluting the intelligence ethic, policymakers' understanding of the collection and production processes enables them better to direct the analysts' unique resources to what is most needed in the battle with uncertainty--an objective and lucid examination of the issues causing the most confusion.
Formalized lists of intelligence "requirements," prepared a year or more in advance, cannot substitute for a more active policymaker involvement.
Ambassador Wolfowitz cites the "East Asian Informals" he held as Assistant Secretary of State for East Asian and Pacific Affairs as an example of effective intelligence support of decisionmaking. Analysts and collectors met regularly with key policy officials to exchange information and views. The discussions were "informed, factual, pointed." The intelligence players learned firsthand what issues were on the mind of the policy officials, and on what particular aspects they needed most to help. Policy officials, in turn, learned what information was newly at hand and what research findings would soon be available for their use, as well as what judgments in recent intelligence issuances were based mostly on speculation.
Responsibility for deciding the policy initiatives to recommend to the President and his Cabinet-level advisers rested with the policy officials at the meetings. But otherwise roles were not set in concrete. Policy participants served as both collectors and analysts, bringing their own tidbits of information garnered usually from their foreign counterparts and laying out their own cause-and-effect reasoning. Collectors and analysts helped work through tactical policy alternatives, "by explaining why they would take this or that course of action."
Ambassador Wolfowitz attributes the US success in managing a peaceful transition in the Philippines from the Marcos dictatorship to a democratic government in good part to effective intelligence-policy relations. The fact that the three key policy officials involved in daily management of the challenge--while they constantly argued about means--agreed on US goals also helped.
According to one of the intelligence participants, the "bonding" at the East Asian Informals opened the way for additional opportunities for keeping in direct, informal contact with policy counterparts--via telephone, in hallways, and on airplane trips to the field.
Ambassador Wolfowitz also cites the effective support provided policymakers by the Arms Control Intelligence Staff. Here, intelligence contributed information and insights to meet the policymakers' needs at every phase of the planning, negotiation, and verification processes. He contrasts this customized, continuous, and largely informal support with what he sees as the much less useful intelligence effort put into formal, arms-length papers. He also remarked that agreement on goals among policy principals, a feature of the Philippines success, was not the rule on arms control issues.
What Adds Value and What Does Not
Ambassador Wolfowitz believes inadequately supported judgments continued to undercut the reputation and utility of intelligence analysis during his last years of policy service (as Under Secretary of Defense for Policy, 1989-93).
The notion that the opinions of analysts should be the main product--when often they are not a useful product at all--is a recipe for having analysis ignored.
In other words, predictions by analysts convey little of value to policymakers. Even if analysts have done their homework and studied the available evidence, policymakers learn little from unsubstantiated opinion. Absent the evidence on which analysts' judgments are based, the policymaker has only a bureaucratic interest in intelligence judgments, and that only because other policymakers may "appeal to the authority" of intelligence opinion to lever policy debates when they are short of evidence to make their case.
First-rate analysis, in contrast, lays out all the facts. It may be that these are all facts to which somebody else had access but the policymaker did not necessarily have. Pulling these facts together, structuring them, and setting out the relationships among them is no mean feat. It is important that analytic products lay out the facts, the evidence, and the analysis rather than simply stating conclusions or analytical judgments. One of the most valuable contributions an analyst can make is when he or she puts the facts together in a new and illuminating way.
Ambassador Wolfowitz believes the analyst is most valuable in clarifying the "micro issues" that often are not adequately tended to in formal intelligence products. This is why briefings and other direct interactions are greatly appreciated by the busy policymaker. First, the policy official who calls for a briefing gets the chance to ask questions on issues that are troubling him as he works his way through the decisionmaking process.
When the author-expert is present, we have a gold mine on the real issues.
Second, even when the intelligence side initiates the briefing, the policy official benefits from hearing from the substantive expert who is excited about the findings from his or her latest research. In this context, he referred to the analyst as the "intelligence ferret" who searches out and brings to light what the hands-on policymakers need to execute successfully their operational responsibilities.
Ambassador Wolfowitz, however, adds a caution about briefings of NSC principals. Cabinet-level officials can be spread thin even on important policy issues because of the breadth of their responsibilities. Ambassador Wolfowitz sees it as an abuse of the trust needed in intelligence-policy relations if Agency leaders try to influence policy decisions in the absence of departmental officials who have the action on the issue at hand and who may wish to counter the Agency's views and interpretations.
Debate and Uncertainty
[In the analysts' work] there has to be some real allowance for uncertainty.
Ambassador Wolfowitz believes another major contribution that intelligence can make to help the policymaker grapple with uncertainty is to expose and explain the debates that go on among analysts. Serious policy officials are always interested in disagreements among analysts, because analysts as well as policymakers are constantly grappling with uncertainty. Policymakers benefit when they can take into account what the analysts see as the full range of possible outcomes on a tough issue. Ambassador Wolfowitz cautions about wasting time with debate for its own sake. But if the complexities of an issue naturally lead either to differences of opinion among analysts or to collective agreement that there is more than one useful way of looking at the evidence, these insights should be shared with policymakers.
The idea that somehow you are saving work for the policymaker by eliminating serious debate is wrong. Why not aim, instead, at a document that actually says there are two strongly argued positions on the issue? Here are the facts and evidence supporting one position, and here are the facts and evidence supporting the other, even though that might leave the poor policymakers to make a judgment as to which one they think is correct. I would have found that kind of document useful; unfortunately, it was far too rare.
Analysis as "Tools"
To sum up his views on value added, Ambassador Wolfowitz urges that analysts see intelligence assessments as "tools" to help in the development of a policy decision, and not as "weapons" to determine by fiat the outcome of a policy debate.
Analysts should not usurp the decision role of policymakers by prematurely limiting the options on the table.
If an assessment contains conclusory statements without the full range of supporting evidence, and if it either suppresses or obscures differences of opinion or uncertainties among the analysts, it is more likely to be used as a weapon rather than as a tool.
Analysts must always remember that their job is to inform the policymaker's decision, not to try to supplant it, regardless of how strongly they feel about the issue.
Ambassador Wolfowitz addresses analysts' concerns about policymaker "objectivity" head on. Most national security issues require both analysts and policymakers to go beyond the hard evidence and to rely upon assumptions. Ambassador Wolfowitz says he is "offended" by the analysts' adoption of an unchallengeable claim to objectivity in these circumstances and their attribution of automatic policy bias to the policymaker. In explication of his own commitment to objectivity, he says:
Policymakers are like surgeons. They don't last long if they ignore what they see when they cut an issue open.
In his view, neither camp can completely avoid the impact of policy bias when it comes to dealing with uncertainty. The intelligence side likes to pretend otherwise, but the manner in which it favors certain substantive assumptions over others has predictable implications for US policy debates.
The serious policy official recognizes the power of policy bias and has a powerful incentive to do all he or she can to insure against the influence of bias and wishful thinking during the working out of analytic assumptions.
Policymakers want to succeed and cannot do so without sound assumptions.
"Bad News" and Warning
What if the analyst does his homework and produces an assessment that undercuts an assumption undergirding an established policy? Ambassador Wolfowitz responds that the analyst has to understand the policymaker's intense commitment to the success of his policy. To this extent, challenges to policy assumptions have to be handled carefully. Policymakers will not gladly give up hard-fought premises; yet to succeed they cannot stick with faulty ones.
Ambassador Wolfowitz recommends that the analyst bring the bad news to the office of a policymaker's staff member. Emphasis should be placed on new evidence and findings. Then let the staffer convey the bad news to his boss.
The more factual the better. Explain what is known and how it is known and let the judgment flow from the evidence.
Intelligence should give the policy officials heavily engaged in working an issue some time to adjust to the new findings. Learning about intelligence that contradicts policy assumptions in The Washington Post or in the National Intelligence Daily would be the least desirable circumstances from the policymaker's point of view.
Ambassador Wolfowitz characterizes "warning as first cousin to bad news." A properly executed warning is a serious matter because it requires rethinking of policy, including possible redeployment of resources and the undertaking of risky as well as costly action. Thus:
Warning must lobby for attention. This does not work through anonymous, routine warning reports. A one-page written brief would help get the needed attention; a special briefing would also help.
Ambassador Wolfowitz's views on what works, what does not, and why regarding intelligence support to policymaking are illustrated by his perceptions of the relationship before and during the Persian Gulf crisis touched off by Iraq's invasion of Kuwait.
His story starts in 1977. As Deputy Assistant Secretary of Defense for Regional Programs, his job was to assess the impact of major budget decisions on US long-term strategy for defending US interests in the region. The main threat, fromthe perspective of planning for low-probability/high-impact contingencies, was either a Soviet invasion of Iran or an Iraqi attack against Saudi Arabia or Kuwait. He despaired of receiving useful intelligence support, because the analysts would see both threats as unlikely, if not highly unlikely. So, instead of backing up planning options with an intelligence assessment, he commissioned a member of his staff to draft a historical annex that assessed the circumstances in which the USSR and Iraq had over the decades threatened or committed military aggression in the Gulf region.
When he assumed office as Under Secretary of Defense for Policy in 1989, Ambassador Wolfowitz once again focused on strategic planning for the Gulf. This time, with the USSR undergoing collapse, many foreign policy authorities were arguing that the threat to US interests in the Gulf had disappeared. He argued, to the contrary, that the United States should plan for defense against possible aggression by Iraq against its oil-rich neighbors, even though he did not judge the threat to be as immediate as it turned out to be.
Nevertheless, in light of the magnitude of US interests at stake, the potential Iraqi threat over the long term required serious US attention. Because the argument was persuasive with Secretary of Defense Richard Cheney, plans for rapid deployment of US military forces to the region were brought up to date, and measures to ensure effective deployment were undertaken.
Ambassador Wolfowitz states that the National Intelligence Estimate on Iraqi foreign policy released in November 1989 did not influence the aforementioned policy process one way or the other. He does not fault the analysts for failing to predict the subsequent Iraqi aggression, because even Iraqi dictator Saddam Hussein may not yet have made his plans regarding military action against Kuwait.
But he does underscore his criticism of assessments that place emphasis on the most likely outcome and do not treat seriously important (albeit admittedly unlikely) contingencies--either what could trigger them or what the United States could do to deter or counter them.
Determining which unlikely threats are worthy of serious consideration is something that would require the effective cooperation of policymakers and intelligence analysts.
The Ambassador gives intelligence high marks for having a thick and reliable book on Hussein's aggressive tendencies. He recalls an NSC meeting in the fall of 1989 at which President Bush asked whether there had been any change in Saddam's character (''We know a lot about this guy Saddam Hussein; can the leopard really change his spots''?). Deputy Director of Central Intelligence Dick Kerr laid out the evidence through:
A lengthy recitation of facts--facts not judgments--that were quite overwhelming in the direction they pointed: this leopard was unlikely to change.
As it happened, the President decided for other considerations to go ahead with a policy of seeking to moderate Saddam's behavior.
Ambassador Wolfowitz faults the Intelligence Community for not warning the policy community about the changing character that took place in Saddam's public statements early in 1990. Somebody should have catalogued his increasingly belligerent rhetoric, compared and contrasted his statements to prior formulations, and laid out one or more plausible explanations for the change.
In general, we tend to undervalue unclassified information. For example, because the public speeches of Saddam Hussein and other dictators are often florid and mendacious, we tend to ignore them and not subject them to serious analysis. In this case, at least, that was a mistake.
When signs started to turn up that the projected scenario regarding Iraqi behavior was not unfolding as we wished (that is, Saddam started to make his threat about burning half of Israel, along with a series of other developments), somebody within the Community should have said, "Wait a minute, here are facts that we ought to take some account of." Analysis, in this instance, would have usefully pointed to the fact that events were not going in the direction we had expected or hoped for.
Finally, Ambassador Wolfowitz gives intelligence high praise for the support given to implementing all phases of US policy once the crisis was underway.
I would be the first to say we got very good support from the Intelligence Community.
We made enormous use of intelligence throughout the lead-up to the Gulf war, and during the Gulf war. But it was primarily used to figure out how to implement policy, not to debate policy preferences.
(1) The first meeting, on 7 February 1994, was sponsored by the Working Group on Intelligence Reform of the Consortium for the Study of Intelligence. The second meeting, 28 June 1944, was sponsored by the Product Evaluation Staff of CIA's Directorate of Intelligence. The essay appears as a commentary in Douglas MacEachin, The Tradecraft of Analysis: Challenge and Change in the CIA (Working Group on Intelligence Reform Papers, 1994).
Jack Davis served in the Directorate of Intelligence.