A Study in Indications Methodology
A first experiment in quantifying the relevance of indicator patterns to different types of hostile action.
A STUDY IN INDICATIONS METHODOLOGY
Diane M. Ramsey and Mark S. Boerner
The intelligence mission of the National Indications Center is to provide strategic warning of a possible attack upon the United States from the Sino-Soviet Bloc. Strategic warning differs from tactical warning both in timeliness and in derivation. Tactical warning relies exclusively upon mechanical detection devices and could not be given until the attack had been set in motion, thus providing no more than a few hours and probably much less-for U. S. forces to react. Strategic warning, given before the strike is launched, is derived from estimating enemy intentions as well as actions, and while it is to some extent dependent upon signals from mechanical detection devices, it also involves the analysis of a great deal of other data less easily quantified and correlated.
Because the indications officer attempts to determine intent from observed actions, he must rely heavily upon inference. The basic inferential structure used by the NIC is contained in its revised indicator list, which defines 123 types of action that a Sino-Soviet Bloc country might take if it intended to wage war. The specific occurrence of one of these indicators is called an indication, as the indicator "Active reconnaissance by aircraft, submarines or surface vessels" was realized as an indication in last winter's overflight of the U.S. carrier Constellation by a Soviet TU-95. The list covers virtually all phases of Bloc activity (with emphasis on the Soviet Union), and all the assets of the intelligence community can through it be brought to bear upon the task of determining whether-and then when and how-the Bloc plans to launch an attack.
Historically, the NIC has been principally concerned with giving warning of premeditated surprise attack on the United States by the Soviet Union. Although other possible courses of hostile action have been considered from time to time, there appears to have been no consensus as to what the range of alternatives might be. The present study, which became an experiment in quantitative differentiation of indicator patterns, originated as an attempt simply to define the component aspects of these alternative hostile actions with the same precision that had been applied to defining NIC's 123 warning indicators.
Seven general hypotheses of actions hostile to the United States were postulated as a first, rough approximation of all possible alternatives:
H1-Premeditated surprise attack
H3-Escalation (limited war to general war)
H6-Diplomatic crisis with no military intent
H7-Military suppression of internal conflicts
Each of the 123 indicators might or might not have a bearing on the acceptance or rejection of any one of these seven hypotheses as the true explanation for a series of observed actions. In order to represent this concept in a systematic fashion, a two-way chart was prepared with the seven hypotheses of hostile action across the top and a selected sample of indicators listed down the left-hand side. (It was decided to work with a sample rather than all 123 indicators because it was not obvious a priori whether this sort of approach would yield useful results.) The sample consisted of twenty-eight indicators considered to be highly important and representative of the three sectors of Soviet activity covered by the indicator list. Six were chosen from the 28 in Sector A bearing on intercontinental strike force capability, thirteen from the 62 of the general military Sector B, and nine from the 33 concerned with civilian activities in Sector C. These are listed in Table 1.
INDICATORS USED IN THE WEIGHTING EXPERIMENT
|Alc||Deployment of MRBMs, IRBMs and associated equipment to satellite nations.|
|Ali||Rapid increase in number of orbiting earth satellite vehicles.|
|A2e||Unusually large and realistic maneuvers of LRA units.|
|A2i*||Major deployment of tankers and long-range bombers to forward bases.|
|A3d||Intensive maintenance activity at submarine bases.|
|A3g*||Expanded submarine barrier operations.|
|B1f||Cancellation of leaves or marked restriction|
|B2f||Release or delivery to combat units of specially controlled weapons and equipment.|
|B3c*||Widespread appearance of new cryptographic or transmission systems.|
|B3f||Extensive interference with key Western telecommunications.|
|B4a||Abnormally large maneuvers at inter-army level or higher.|
|B4c*||Tightening of military security, such as new travel restrictions, etc.|
|B4g||Abnormally high levels of activity in airborne forces units.|
|B5d||Withdrawals of significant naval surface units from Black and Baltic Seas.|
Intensive naval active defense measures.
|B6b*||Major standdown in TAF for maintenance.|
|B6e||General alerting of Soviet air defense forces.|
|B7b||Increased intelligence collection efforts against key targets.|
|B7d||Active reconnaissance by aircraft, subs or surface vessels.|
|Cla||Progressive reduction in size of Bloc missions in Western countries|
|Cle*||Consultation by regional Satellite leaders with Moscow and Peiping.|
|C2a||Increased belligerency in official Soviet pronouncements and propaganda.|
|C2f*||Sudden shifts, especially in crises, to softer propaganda themes.|
|C3d||Imposition of abnormally heavy censorship measures.|
|C4b||Widespread construction or expansion of shelters.|
|C4g||Evacuation of government, military and technical personnel.|
|C5b||Conversion of industrial production from civilian to military items.|
|C6c*||Cancellation of scheduled visits by Soviet scientists outside the Bloc or their recall.|
The next step was to obtain a measure of the relevance of the sample indicators, and this was accomplished by weighting each of the 28 selected indicators for each of the seven hypotheses. The weight was designed to vary directly with the acceptability of the hypothesis, given the indicator as evidence; the larger the weight the more likely the hypothesis. Since these kinds of judgments had never been made before (and it was not by any means clear that they would be useful or even meaningful), it was decided that several analysts intimately experienced in evaluating warning data should make independent judgments about the effect of each indicator on the credibility of each hypothesis. If a group of experts agreed among themselves, then it would be reasonable to accept their consensual judgment as a basis for further experimentation.
Five persons familiar with the warning problem, then, were asked to make independent judgments of the relevance of each of the 28 indicators to each of the seven hypotheses, a total of 196 judgments per person. Each indicator was evaluated on a five-point scale, as follows:
+2 = Strong positive indication of credibility of hypothesis
+1 = Some positive indication of credibility of hypothesis
0 = No influence on credibility of hypothesis
-1 = Some negative indication against credibility of hypothesis
-2 = Strong negative indication against credibility of hypothesis
The agreement among the five judges was very close on many of the 28 indicators. Sometimes one judge might be consistently conservative in fixing indicator weights with respect to a particular hypothesis, avoiding the strong weights both positive and negative, while a second would weight in the same direction but use the extreme +2 and -2 more often. With allowance for this individual conservative or radical bias, the level of agreement gave some general basis for confidence in the rating procedure as an experimental tool.
On a number of indicators, however, there was little or no agreement among the five judges. In an attempt to reconcile or adjudicate these differences the judges were asked to set down their reasons for assigning the weights they had. From these explanations it could be seen that although some differences of opinion were deep-seated and not reconcilable, others were caused by divergent interpretations of the terms used in defining the several hypotheses. Once the ambiguity was resolved, new weights were assigned which eliminated much of the earlier discrepancy.
This exercise of attempting to locate and explain the areas of disagreement proved to be extremely useful. It immediately pointed up the need to go back and redefine the repertory of possible alternative courses of hostile action with much more detail and precision. The discussions showed it necessary to define each hypothesis with respect to at least seven component features. An effort was made to list under each component all admissible possibilities that came to mind. The elaborated taxonomy for defining hypotheses of hostile action appears below:
S- Element of surprise
1-Premeditated surprise attack
3-Deliberate unconcealed attack
V- Area attacked
1-Continental United States
2-U.S. possessions and bases
3-U.S. formal allies
4-Western-oriented countries not formal U.S. allies
2-Strategic air forces
3-Strategic naval forces
5-Tactical air forces
6-Tactical naval forces
W- Weapons used
4-Chemical, bacteriological, or radiological
2-Concentrations of military forces
4-Key production and support facilities
D-Diplomatic and economic measures
This formulation cannot of course be considered all-inclusive or necessarily complete. It could be expanded to cover additional components as well as to list additional alternatives within components. The "Other" listing allows for the inclusion of an unusual circumstance that does not fit into the more conventional alternatives, for example the Berlin wall in the "Weapons used" component. The general formula used to express an hypothesis in terms of the seven components is as follows:
S on V by A employing F and using W against T (after or accompanied by D).
In order to test the suitability of this formula for describing hostile action, several past periods of international crisis were subjected to a corresponding analytic breakdown. The following statements show how two of these, the Pearl Harbor attack and the outbreak of the Korean War, would be defined by this technique.
Pearl Harbor: Sl on V2 by A6 employing F2, 3 and using W3 against Tl, 2, 4 (after or accompanied by D8).
Or in expanded form:
A premeditated surprise attack on U.S. Possessions and bases by Japan employing strategic air and naval forces and using high explosives against military installations, concentrations of military forces, and key production and support facilities (following negotiations in Washington with Japanese envoys on the Far Eastern situation).
Korea: S4 On V4 by A4 employing F5, 6, 7, 9 and using W3 against T1,2 (after or accompanied by D2).
A premeditated surprise attack on South Korea by North Korea employing ground forces, tactical air and naval forces and clandestine agents and using high explosives against military installations and concentrations of military forces (following several propaganda warnings of an attack).
After the formula proved reasonably successful in describing past hostile events, it was applied to a plausible hypothesis of potential Soviet action against the United States. (Evidence to substantiate this particular hypothesis is extremely difficult to detect, and for this reason it is of major concern to those charged with strategic warning.)
World War III: S2 on V1, 2 by A1 employing F1, 2, 3 and using W1, 2 against T1, 2.
A pre-emptive attack on the continental United States and its possessions and bases by the Soviet Union employing strategic strike forces (missiles, air, and navy) and using thermonuclear and nuclear weapons against military installations and concentrations of military forces.
Excluding the open-ended "Other" category in the elaborated taxonomy, several hundred thousand hypotheses could be generated through permutation and combination of the alternatives under the several components. Although probably fewer than fifty of these are plausible, the number of conceivable related warning patterns is staggering, for a variety of combinations of the 123 indicators is relevant to each hypothesis and individual weighting of the indications from -2 to +2 raises their number to its power of five. It is this high order of complexity of events in the real world which, until the development of electronic digital computers, precluded mathematical simulation of any but trivial war problems.
Hypothesis by Indicator Profile
One possible approach to evaluating indications would be to construct the patterns of indicators and weights most likely to occur under particular plausible hypotheses. In the event, say, that the USSR was going to launch a surprise attack on the United States, the patterns would presumably differ from those to be expected if it were preparing to start a limited war. Probably yet another set of combinations would be likely to appear if the USSR were simply taking steps to defend itself. These various groups of general warning patterns could be compiled into a warning outline, which could then be used by indications analysts as a standard against which to measure accumulating indications information. Indications officers do employ subjectively such comparison techniques already, but the warning pattern used as the standard is not expressly stated and may change frequently, even for the same analyst.
Following this line of reasoning, the next step in the study was to investigate whether it is possible to differentiate among the various hypotheses by their patterns of indicator weights, and if so which indicators are of primary significance in this discrimination. If the weights assigned to certain indicators should be highly similar for two or more hypotheses it would be difficult to use these indicators as criteria for accepting one of the hypotheses over the others, and other indicators would have to be identified to give the necessary differentiation among them. But if the patterns of indicator weights assigned for different hypotheses are quite distinct, they provide a means for deciding whether to accept or reject a particular hypothesis on the basis of indications data received.
In order to illustrate this concept, let us consider a simple example. Suppose we had only three indicators (I1, I2 and I3) and only three hypotheses of hostile action (H1, H2 and H3). Let us say the three hypotheses have the following sets of average indicator weights:
If we drew a pattern or profile of the indicator weights for each hypothesis, the three would look like this:
From these profiles we can see at a glance that H2 and H3 are similar but H1 is quite different from the others. We might further observe that I3 provides no basis at all for distinguishing H2 from H3. Graphic display techniques frequently offer considerable assistance in summarizing unorganized data and may reveal relationships which are not obvious when there are large amounts of data; perhaps it would be useful to represent the profile information in geometric form. To portray geometrically more than two of the indicators, however, would force us into an n-dimensional space which is beyond our ability to represent easily on a flat surface. If we therefore take only indicators 1 and 2 from the example and graph their weights with respect to the three hypotheses, the result is this:
Even though this geometric representation has lost the information contributed by Indicator 3, it still makes clear that H1 is remote from H2 and H3, which are comparatively close together.
It was basically to this sort of statistical analysis that the study subjected the indicator weights assigned by the five judges. Because more complex computations were planned, however, the number of indicators had to be reduced once more to keep the experiment within reasonable time limits. The 28 indicators previously chosen as a representative sample of the original 123 were carefully studied and eight of them selected for this analysis, as follows:
I1-(A2i) Major deployment of tankers and long-range bombers to forward bases.
I2-(A3g) Expanded submarine barrier operations.
I3-(B3c) Widespread appearance of new cryptographic or transmission systems.
I4-(B4c) Tightening of military security such as new travel restrictions, etc.
I5- (B6b) Major standdown in the Tactical Air Force for maintenance.
I6- (CIe)Consultation by regional Satellite leaders with Moscow and Peiping.
I7- (C2f)Sudden shifts, especially in crises, to softer propaganda themes.
I8- (C6c)Cancellation of scheduled visits by Soviet scientists outside the Bloc or their recall.
The seven original hypotheses were all retained. An average was taken of the weights assigned by the five judges to each of the eight indicators with respect to each hypotheses. Seven profiles of the average weights were then drawn, one for each hypothesis. They are shown in Figures 3 through 9.
Of these profiles it can be seen that H4 and H5 (Figures 6 and 7) are the most highly similar of all, reflecting the logical similarity between the situations of limited war and guerrilla war. H6 and H7 (Figures 8 and 9) also have some, less striking similarity and thus corroborate our intuitive feeling that diplomatic crisis with no military intent should have many aspects in common with military suppression of internal conflicts. The other three hypotheses (premeditated surprise attack, pre-emptive attack, and escalation) appear to have unique profiles. These results appear to support the application of indicator pattern analysis to discriminate among hypotheses.
In order to show geometrically all of the information contained in these seven profiles it would be necessary to use an eight-dimensional space, which is impossible to represent clearly in two or even three dimensions. Fortunately, a powerful statistical technique 1 I permits one to transform the eight axes of this space to a new set of eight in which each is a linear combination of all the original ones. This means that each of the original indicator weights is represented in each of the eight new dimensions.
It is now possible to select the two most significant of these dimensions as the axes for portraying graphically in one plane most of the information contained in the seven profiles.
Profiles of Average Indicator Weights Assigned by Five Judges
Profiles of Average Indicator Weights Assigned by Five Judges
This statistical transformation having been performed with an IBM 709 computer, the resulting eight new dimensions were called variables Z1 through Z8 to distinguish them from the original indicators I1 through I1. Z2 and Z6 the two largest, were seen to account together for 76% of the information inherent in the set of seven profiles, Z6 for 52% and Z2 for 24%.2 The next largest, Z1, accounted for another 16%. All five other Z6 variables together contributed less than 8%. With Z2 and Z6 accounting for a little more than three fourths of the information in the profiles, it becomes meaningful to graph the relationships among the seven hypotheses using them as axes. This two-dimensional plot is shown in Figure 10.
Here the positive and negative signs do not signify probability and improbability respectively, but the position of an hypothesis along each axis is determined by the pull of the positively and negatively weighted indicators that are combined in the axis. On Z2 the positively weighted indicators exert a pull to the top of the space and the negatively weighted a pull downward. On Z6 the positively weighted indicators exert a pull to the right of the space and the negatively weighted a pull to the left.
On the vertical axis Z2 there are large positive coefficients for indicators 1, 6, and 8, and the largest negative coefficients are for indicators 4 and 5. On the horizontal Z6 the only large positive coefficient is for indicator 6, but there are fairly large negative coefficients for indicators 2, 3 and 7. These indicators with important positive and negative coefficients are the following.
Z2: (I1) Major deployment of tankers and long-range bombers to forward bases.
(I0) Consultation by regional Satellite leaders with Moscow and Peiping.
(I8) Cancellation of scheduled visits by Soviet scientists outside the Bloc or their recall.
Z6: (I6) Consultation by regional Satellite leaders with Moscow and Peiping.
Z2: (I5) Major standdown in the Tactical Air Force for maintenance.
(I4) Tightening of military security such as new travel restrictions, etc.
Z6: (I2) Expanded submarine barrier operations.
(I7) Sudden shifts, especially in crises, to softer propaganda themes.
(I3) Widespread appearance of new cryptographic or transmission systems.
Examination of Figure 10 reveals a distinct cluster of the three hypotheses describing a limited war situation (limited war, guerrilla warfare, and military suppression of internal conflicts). The distance between this cluster and a fourth hypothesis, diplomatic crisis with no military intent, is less than the distance between any pair of hypotheses outside the cluster. The diplomatic crisis hypothesis might therefore be linked with the limited war cluster in a "confined crisis" category.
There is only one indicator which can have pulled this cluster toward the positive end of the Z6 axis--Communist Bloc consultation. It seems consistent with Bloc activities in a limited war situation.
The split along the Z6 axis between the total war situations on the left and limited war situations on the right can be more fully explained, however, by noting the three indicators which have a strong negative pull along the horizontal axis. These are expanded submarine barrier operations, widespread appearance of new cryptographic or transmission systems, and sudden shifts, especially at a time of crisis, to softer propaganda themes. The expansion of submarine operations is a relatively unambiguous action which would increase Soviet ability to wage general war. The introduction of new cryptographic systems could be a protective preparation for surprise attack but by itself is susceptible of a number of other interpretations. The sudden shift in propaganda could be an attempt to lull the United States into a posture of reduced alert. The element of deception contained in this last indicator may be a sufficient explanation for the difference in position between the hypotheses of pre-emption and premeditated surprise attack on the left and that of escalation about midway between them and the limited war cluster on the right.
Along the vertical axis, escalation and pre-emption are at opposite ends and premeditated attack lies approximately midway between them. The indicators operating negatively which apparently favored pre-emption and rejected escalation are the tightening of military security and a major standdown in the Tactical Air Force. An increase in military security, one of the more difficult indicators to identify, would in fact be more likely to accompany pre-emption than escalation. A standdown in the Tactical Air Force does not seem to argue strongly for pre-emption, but it seems to explain the negative position on the Z2 axis of the limited war cluster, because tactical rather than strategic air forces would probably be used in a limited action.
The indicators which exert a positive pull upward along the Z2 axis and thus separate escalation from pre-emption are a major deployment of tankers and long-range bombers to forward areas, Communist Bloc consultation, and cancellation of scheduled visits by Soviet scientists outside the Bloc. Each of these actions could logically be associated with pre-emption except that their likelihood of detection is great; this is why they favor the escalation hypothesis. The convening of a Bloc strategy conference, a familiar Soviet pressure technique, would explain the positive location of the diplomatic crisis hypothesis along the Z2 axis.
The position of premeditated surprise attack on the vertical axis is the most difficult to explain; it was expected that this hypothesis would cluster with pre-emption. The two are indeed the closest pair along the Z2 axis, but evidently the positively weighted and negatively weighted indicators on the Z2 axis exerted an equalizing pull upon the hypothesis of a premeditated surprise. Its central position on the Z2 axis may be a reflection of the ambivalence of the judges concerning the role of deception in this situation. The cleaner polarization along the Z6 axis rests on an unambiguous distinction between limited war and general war.
The encouraging thing about this analysis is that a totally objective and dispassionate statistical arraying of the relationships among the seven hypotheses of hostile action corroborates our intuitive explanation of the roles played by the various indicators in distinguishing among them. It is both surprising and reassuring to discover that the independent assignment of indicator weights by the five judges, despite a lack of full consensus, disclosed so much information.
The sources of ambiguity in the experiment may be summarized as follows:
The use of indicators rather than indications data, i.e., specific occurrences of indicators, means that the time factor was not precisely stated, quantities of troops, equipment, etc., were expressed in general terms such as major, heavy, etc., and the geographic areas involved were not specified.
The state of international relations was not described; the general strategic setting was left to the judges' imagination. The present world situation might best have been explicitly assumed.
Each indicator was considered by itself, whereas few if any indicators are in practice evaluated in a vacuum. They are considered not only against the general background noted above but some of them in pairs or clusters with others. These groupings should be defined and treated together.
The list of indicators was incomplete. It is economical to select a sample for experimental purposes, and the sampling used here may have been valid; but the reader should recognize that only a portion of the problem was under study.
No attempt was made in this first trial to account for the probability of occurrence and the likelihood of detection of each indicator. These characteristics have been described for the entire indicator list and they form an essential part of the complete indicator definition. Perhaps the indicators should be grouped according to the ease with which the Bloc can manipulate them. Another possible classification scheme would be a chronological listing broken down by probability of occurrence.
Terms likely to occur in describing indicators and hypotheses were not standardized by any authority such as the Dictionary of United States Military Terms of Joint Usage. It became clear during the process of adjudicating weights that many of the original differences resulted from variations among the judges' definitions of key concepts.
Now that the validity of this approach seems to have been established, the statistical analysis should be done with the entire set of 123 indicators and the elaborated statement of hypotheses. If the eight indicators used before were truly representative of the total set of 123 and if the seven hypotheses used were a reasonable generalization of the elaborated statements, the full analysis should give a configuration similar to that of Figure 10. But it should have more precision and reflect more subtle nuances in the roles played by the chief indicators. Such a complete portrayal of the associations among alternative hypotheses in relation to the full complement of indicators could, one may hope, serve as a basis for the development of more sophisticated and advanced decision models.
1 The use of canonical variates. For a full explanation of the technique see C. R. Rao's Advanced Statistical Methods in Biometric Research (NY: John Wiley & Sons, 1952), chapter 9, especially pp. 364-370.