Scientific Estimating

22 SEPT 93


Problem of ranging technical possibilities in due policy perspective.


Wayne G. Jackson

Those of us in the estimating business have a troublesome time with the problem of incorporating scientific or technical contributions into a finished estimate. To make the point, a hypothetical case relating to missiles and nuclear warheads is discussed below, but the example might as well be any complicated piece of military hardware or other technical subject.


Technical Possibility

An estimate on the advanced weapons program of Upper Volta is started. In the normal routine a contribution is asked from the Guided Missiles and Astronautics Intelligence Committee. In due course, the estimators receive a contribution which concludes that, on the basis of an examination of the evidence, "Upper Volta could have an IRBM system ready for production in 1967-68 and carry out deployment in 1968-69." The economists submit a contribution saying that, given a high enough priority, the economy of Upper Volta could support such a program. The political analysts find that Upper Volta thinks it has an urgent requirement for such a weapons system. So the estimate comes out saying that "Upper Volta could start deploying an IRBM system in 1968-69."

The Joint Atomic Energy Intelligence Committee also submits a contribution, one on nuclear developments. Upper Volta has conducted a few atmospheric tests of nuclear devices, something is known of its general level of technical competence and production facilities; and so JAEIC states that warheads compatible with the IRBM's could be produced by the time GMAIC says the missiles could be ready for deployment. So the estimate adds to its sentence on deployment of the missiles the words "with compatible fission warheads." In the course of this exercise, what started out to be very special statements of raw capabilities get transformed into USIB-approved estimates that have an aura of probability. While the word "could," in the estimating business, is understood to be purely a statement of possibility, the mere fact that the possibility is stated with no further qualification gives it something more of substance. The reader is apt to think, "If there is not a good chance that the possibility will be realized, why mention it?"

Top of Page


Hypotheses on Thin Ice

It is possible that the estimate could be handled with so much emphasis on its being a mere statement of potentialities that the reader would not be confused into taking it as having any significant element of probability. But it is doubtful if drafting could convey the tenuousness of the many technical components of the estimate. For instance, the estimate that Upper Volta could have compatible nuclear warheads involves in itself at least two estimates, each based on a number of subsidiary estimates. What is the maximum weight of the warhead -- including guidance, firing mechanism, etc. -- which the Upper Volta missile can carry? What is the likely actual weight of each of these components? What are the warhead's dimensions? It is highly unlikely that anyone can make more than crude guesses on these questions, even if we had seen a missile in the Army Day parade in Ougadougou.

Similarly, we probably know little about the probable size, weight, and shape of the nuclear component of the postulated warhead, how much fissionable material would be in it, its yield, or even its general design. Yet some hypotheses on all these questions underlay the estimate that a warhead compatible with the missile could be available. The estimators ask the technicians for opinions, and they oblige. Indeed, the estimators often ask for even more speculative data, as for the CEP and reliability of missiles. Comparable estimative problems arise in all technical subjects, e.g., capabilities for CW and BW, specifications for most kinds of complicated hardware such as aircraft, naval vessels, etc.

The intellectual philosophy of a scientist leads him to consider his scientific statements, however couched in language, as hypotheses -- the most satisfactory synthesis that he can make of the available data at hand. If and as evidence changes, he will adjust the hypothesis accordingly, or even abandon it, without any feeling that he is changing previously established truth. Estimative intelligence judgments are of a different kind, even though they are based in large part on analysis of the known facts. The intelligence estimator feels instinctively that he should state what he believes true, qualifying the estimate to indicate his qualms about its validity. When it turns out to have been wrong, even though it was the most reasonable one he could make on the basis of available evidence (as on the missiles in Cuba), he feels that he failed. The biological researcher is not much upset when his hypothesis doesn't work out in laboratory tests, but the doctor is when the treatment he prescribes for his patient doesn't work and the patient dies. This analysis or analogy cannot be pressed too far, but it is part of the difference between scientific and intelligence estimating.

Top of Page



The scientist, in making an intelligence estimate, must have in mind the purpose for which he is making it -- as do all estimators. The temptation to estimate the "worst case" is just as strong with him as with anyone else. If U.S. security plans are to be made on the basis of his estimate, it seems better that they be based on the worst that is reasonably possible, not on hopes which may turn out to be false. This is not necessarily the phenomenon of "Pearl Harbor insurance," wherein one estimates the worst, secure in the knowledge that if his dire predictions do not turn out, no one will blame him for an unexpectedly favorable course of events. It is rather a judgment that when all hypotheses are shaky, the reader had best be prepared for the worst. In respect of other nations' weapons, this worst is often arrived at by taking the best skills, experience, and technology known to the estimator, discounting them by a relatively small factor, and coming out with an estimate of raw capability.

The non-technical estimator is at a great disadvantage in dealing with such technical contributions. He can be nowhere nearly as familiar with the evidence as the technician or as well equipped to deal with it. If he questions the hypothesis, he can often be silenced when his ignorance is pointed out. (This pointing out of his lack of competence to deal with technical subjects is most often done by people who serve on technical bodies but are at best amateur scientists. The vigor with which hypotheses are defended as truth often seems inversely proportional to the technical competence of the defender.) The non-technical inquirer can unearth, without too much prodding, the vast areas of uncertainty in our evidence on advanced weapons systems. But he is hard put to it to offer a more defensible judgment.

Taking into account what we know (which is little enough) about Upper Volta's experiments, technical and economic resources, and what we believe to be its national objectives, attributing to it a fair amount of the best technology we know (usually U.S. technology), and considering that it is better to over-warn the U.S. policy maker than to engender any degree of complacency by a judgment which cannot be documented, we thus come up with the estimate that "it is possible that Upper Volta could deploy IRBM's with nuclear warheads in three years."

Yet the estimator, technical or non-technical, feels in his bones that this worst case is highly unlikely. Does he estimate as above and add "but it might just as well be three or four years later, or even longer?" This hardly looks as if he's earning his living. He is also affected by a conscious or unconscious desire to avoid the bias that if it took the United States ten years to develop an IRBM it will take those foreigners longer.

Top of Page


Ways Out

Does the calling in of a consulting panel help? In most cases it is doubtful. The two- or three-day panel has not kept up with the evidence, could not possibly have done so. Just the classification of much of the evidence precludes this. The panel is briefed by the technicians, who under the best of circumstances feed into the mechanism the same data which formed their own views. The panel has many of the same compulsions as the original technical group and is apt to produce some variation of the "worst case." The consultant does not have to act or budget on the basis of the judgments he makes, and while the government estimator doesn't either, he does feel a longer-term responsibility for his advice to the budgeter.

A formal intelligence estimate should whenever possible give a judgment as to the most likely contingency. The scientist often says that there is no basis for determining the most likely. The estimator is therefore in a dilemma for which there may be no solution. Perhaps such estimates can only be so clothed with caveats and qualifications as to make them seem ethereal, and certainly annoying to the reader who craves certainty. (Incidentally, the use of footnotes to call attention to uncertainties is of limited value. Especially when numerical tables are given, the footnote, usually in microscopic type, is easily overlooked or forgotten.)

Perhaps it should be the rule that the non-technical estimator ingest the scientific contribution, append it as an annex to his estimate, and present his layman's best judgment with all the deprecating language he can think of as to the difficulty of making confident estimates. This is what sometimes happens. But in most cases, those participating in the coordination meetings on an estimate include the technicians, professional and amateur alike, and the pressure they exert on the chairman of the coordinating group to accept the scientific contribution's language is great. The chairman can, and often does, retreat to a strict interpretation of "could," "possible," "might" and not try to fight the experts. In this process the reader is likely to be given an impression of probability and firmness which is not warranted.


Historical Document
Posted: May 08, 2007 08:00 AM
Last Updated: Aug 05, 2011 08:26 AM