The Tyrannical Numbers

The Tyrannical Numbers

To the extent that the disparaging public comments make reference to actual CIA products, they focus almost entirely on numbers-GNP figures and some unclassified statistics disseminated by the CIA over the years. The HPSCI report, in presenting its findings relating to those areas of CIA's work that merited criticism, opined that if the CIA had refrained from comparative assessments-showing Soviet-US GNP ratios in aggregate and per capita figures or Soviet GNP rankings with other countries-its reports ``might very well have not generated the current controversy.'' 10 There is some validity to this judgment, although just how much less criticism there would have been is an open question.

This monograph is not intended to take on the methodological arguments over what was the correct quantitative measurement of Soviet GNP. Such an undertaking is well beyond the expertise of this author, and a review of the vast amount of literature that has been devoted to the subject raises a question of whether it is resolvable. A major effort in this area is the study by Abraham Becker of RAND Corporation published in late 1994 11. Regardless of whether a reader agrees with Becker's specific conclusions, the presentation encompasses a wide range of diverse viewpoints and is accompanied by an extensive index of pertinent work for those who would seek to delve deeper into the subject. A comprehensive treatment of the issue was recently published by Gertrude Schroeder. 12

The purpose of this monograph is to argue that judgments on CIA's performance on the Soviet Union should be based on a straightforward comparison of the record and the events. If the CIA is to be judged as having failed, it should be because the picture painted in the CIA products was/is incorrect. It is useful, however, to put the GNP arguments in the context of the substantive intelligence questions at issue regarding the Soviet Union.

One of CIA's more vociferous critics, Anders Aslund, referring to the fact that CIA estimated the USSR's average annual GNP growth during the 1980-85 time frame to be nearly 2 percent, said that, ``If the CIA assessments had been reasonably accurate the Soviet economy would be a maturing industrialized economy . . . there would be little need for economic reform; Gorbachev's urgency would be incomprehensible; and most internal criticism in the USSR would be unfounded.'' 13

The differences on this issue ``do not appear to be over whether the Soviet economy was in a dismal state, but over which quantitative GNP calculation was an accurate depiction of the situation on the ground.''

The Soviet economy portrayed in the CIA products described above, however, hardly qualifies as a ``maturing industrialized economy.'' On the contrary, those products-over many years-consistently described the Soviet economy as, for example, ``primitive, grossly unbalanced, and in massive disequilibrium'' with a consumer economy that is ``fourth class when compared to Western economies,'' (1981-Appendix A, reference 8) and as resembling ``a developing economy'' (1989-Appendix A, reference 32):

  • The CIA did, in fact, lay out a strong case on the ``need for economic reform'' in the USSR and described at length the basis for ``internal criticism'' of the Soviet economy that led to Gorbachev's efforts.
  • CIA's analytic products show how frequently the Agency said that eventually the declining economy and stultifying societal conditions would lead to the turmoil in leadership politics that Gorbachev provoked.

Thus, the differences between CIA and Aslund do not appear to be over whether the Soviet economy was in a dismal state, but over what quantitative GNP calculation was an accurate depiction of the situation on the ground.

Much of the criticism of CIA's performance on the Soviet economy falls into this pattern. It entails a substantial amount of intuition. For example, Herbert Meyer, a former economic editor of Fortune whom DCI William Casey brought to the Agency in the early 1980's as a special assistant, is quoted as saying:

Everything I had been able to learn about the Soviet economy, including visiting the place, told me it couldn't be growing at the rate the CIA said it was. . . It simply couldn't be true. I know what an economy looks like when it's growing three percent a year, and that isn't what it looks like [Author's note: Actually, CIA calculated the average for the early 1980s at slightly less than 2 percent per year.]. . . . You cannot have food shortages growing worse, production shortages growing worse, bottlenecks-all those things we knew were going on-and still have an economy growing at the rate the agency said it was-which the U.S. was barely doing at that point. . . .It couldn't be true. 14

Why did the CIA report East German GDP as greater than West German GDP in The World Factbook 1987?

One of the most frequently cited examples of grossly flawed analysis by CIA concerns an erroneous entry in The World Factbook 1987 that stated that in 1986 West German GDP per capita was $100 lower than that of East Germany. This is cited by critics as demonstrating the extent to which CIA overestimated the strength of the Communist economies and failed to see the deterioration that led to the collapse of the Communist regimes.

While the process that led to this is deserving of criticism and has since been corrected, the published figures were not a reflection of CIA's analytical judgment of the relative performance and strength of the West German and East German economies. The discrepancy occurred as a consequence of separate analytic divisions using different methodologies to convert GDP into a dollar figure for inclusion in its World Factbook.

  • The East German analyst, working with an outside contractor hired to devise ways of compensating for the inaccuracies of East German data, used ``purchasing power parity ratios'' (PPPRs) based on UN data to estimate GDP in dollars.
  • The West German analyst applied the dollar-deutsche mark market exchange rate from 1985 to convert the 1986 West German GDP figure (used also by the OECD) into dollars. 
  • The problem from using two different conversion rates was compounded by the fact that in 1985 the dollar was booming compared to the deutsche mark. Thus, using the market exchange rate between the currencies resulted in a significant underestimation of the West German GDP.

The isolated nature of the error is demonstrated by the fact that, in that same year, the Handbook of Economic Statistics published by the Directorate of Intelligence used a common methodology for converting GDP into dollars, reported 1986 West German per capita GDP as being 32 percent greater than the East German figure:

  • While claims have been made that the gap was even wider, such arguments are completely different from allegations that the Agency was blind to the enormous gap between the economic performance of the two Germanys.
  • Moreover, arguments that an even greater gap should have been reported in the Handbook often are based on confusing GDP per capita with per capita income. GDP calculations include the production of unsold goods and spending on defense and other government projects and services that may not directly benefit households. Government spending, wasteful construction projects, and unsold inventories were especially high in Communist countries.

As in the case of Aslund's comments, the disagreement was not over all the dismal things "we knew were going on,'' the divergence was over whether ``those things'' were possible in an economy that was growing at an average of nearly 2 percent a year. The CIA argued that this was possible because GNP merely measured gross output without regard to use, quality, or contribution to welfare; it included, for example, the military production and raw quantities of wasteful output. (US calculations of its own GNP as an indication of the public welfare recently have come in for similar criticism.) Others, such as Meyer, found the numbers ``counterintuitive"-inconsistent with what they saw-and looked for lower numbers they believed were more compatible with the dismal conditions that everyone agreed existed.

Given the nature of the analytic problem posed by the Soviet system, the analysts preparing the numbers anticipated that some of their numbers would be open to question. CIA participants in this analytic effort would welcome an objective public debate on the numbers issue. This would provide a forum in which CIA's numbers and those offered by others could be subjected to a common examination of sourcing and methodology. Such examinations would also illuminate the fact that GNP calculations include production of unsold goods as well as spending on defense and other government projects that may not directly benefit households. These conditions were particularly manifest in the wasteful construction projects and unsold inventories of Communist countries.

``The HPSCI report concluded that the CIA practice of expressing its estimate of the Soviet-US GNP ratio... opened the door wide for misinterpretation, if not misrepresentation.''

An objective examination would also provide an opportunity to confront the ``counter-intuitiveness'' argument with certain realities such as: (a) the population of the Soviet Union exceeded the combined populations of West Germany and Japan by an amount greater than the combined populations of France, the Netherlands, and Belgium; (b) the Soviet GNP included production for what was probably the world's largest military establishment; (c) material extraction in the Soviet Union was the highest of any single nation; and (d) the principal problem with the Soviet economy was not its size but its distortions-not simply how large the GNP was but its composition and how it was distributed.

The presentational flaws identified by the HPSCI group may well be more weighty than the methodological shortcomings in the CIA estimates, a conclusion also implicit in the Schroeder and Becker studies:

  • The HPSCI report concluded that the CIA practice of expressing its estimate of the Soviet-US GNP ratio as a single-valued geometric average of separate ruble and dollar estimates opened the door wide for misinterpretation, if not misrepresentation. This clearly occurred over the years, although both ruble and dollar comparisons were shown in CIA's major GNP comparison papers and in CIA's annual statistical handbook.
  • More significantly, the Agency almost certainly failed to account fully for the differences in the quality of US and Soviet goods in its comparisons. How much this failure biased the results remains to be established.
  • CIA analysts correctly point out that their presentations noted the potential distortions in their calculations, but these ``caveats'' all too often were lost on many readers. For example, CIA measures of growth (GNP in constant rubles) at best were an approximation of changes in the USSR's production potential, not gains in welfare.

In the best of circumstances, numbers lend themselves to what have become known as ``sound bites'' (one could make a parallel case for ``sight bites''). They are easily taken out of context, misunderstood, or deliberately misrepresented. The more technical and complicated the derivation of the number, the more this is so, because much of the audience does not understand the intricacies included in its computation. The misperceptions arising from CIA's GNP work make a prima facie case that we did not always meet the required presentational rigor.

The problems that can be mitigated by more careful presentation are illustrated in the CIA document listed as reference 32, The Soviet Economy in a Global Perspective (March 1989). That document states that, at one time, the Soviet economy reached nearly 60 percent of the US GNP. The 60 percent illustrates one of the criticisms specifically cited in the HPSCI review-presenting a single-figure geometric mean of the ruble and dollar comparisons at a time when the spread between the ruble and dollar calculations was more than 25 percentage points.

Even so, the 60 percent appears in a lengthy paper devoted to describing the disastrous state of the Soviet economy, depicting it as more like a less developed economy than anything in the West and concluding that this dismal performance posed major political problems for the Soviet leadership. The Key Judgments of this paper are presented in their entirety in the appendix. The paper itself has been declassified and is thus available for further examination. Readers can judge for themselves the validity of criticisms that cite this ``60 percent'' figure as a basis for charging CIA blindness to the state of the Soviet economy, while ignoring the rest of the paper. The ``selectivity'' argument cuts both ways.

``What the enormous gap between CIA's analytic record and the perception of that record demonstrates . . . is that the channel of communication between CIA and the policy community has, at best, been poor . . . ."

A question that must be asked, however, is how would the message in that CIA paper have differed if the number had been presented as ``about 50 percent'' or ``two-fifths''? Or better yet, what would the public perception have been if the paper had given both the ruble and dollar calculations while stating that the actual ratio was somewhere between them? Would this have had an impact on the judgments the paper offered on the state of the economy over the preceding decade and on the resulting political instability in the USSR? On the implications of that political instability for the longer term prospects of the regime? On the implications for US security concerns? Would such differences in the calculation of the dollar value of Soviet GNP-as opposed to rates of growth of ruble GNP-have shaped the judgments in the long list of CIA products cited above?



10 HPSCI Review Committee report, p. 8.

11 Abraham Becker, ``Intelligence Fiasco or Reasoned Accounting: CIA Estimates of Soviet GNP,'' Post Soviet Affairs, 10 (October-December 1994), pp. 291-329

12 Gertrude Schroeder, ``Reflections on Economic Sovietology,'' Post Soviet Affairs, 11 (July-September 1995), pp. 197-234.

13 Anders Aslund, quoted in Henry S. Rowen and Charles Wolf, Jr., The Impoverished Superpower: Perestroika and the Soviet Military Burden, (San Francisco: Institute of Contemporary Studies, 1990), p. 15.

14 Herbert Meyer, quoted by David Kennedy, Sunshine and Shadow: The CIA and the Soviet Economy, Case Study C16-91-1096.0 for the Intelligence and Policy Project, John F. Kennedy School of Government (Cambridge, MA: Harvard University, 1991), p.18.



Historical Document
Posted: Mar 19, 2007 02:26 PM
Last Updated: Jul 07, 2008 01:10 PM