An Intelligence Role for the Footnote
APPROVED FOR RELEASE
CIA HISTORICAL REVIEW PROGRAM
22 SEPT 93
Modest proposal for a revolution in intelligence doctrine.
AN INTELLIGENCE ROLE FOR THE FOOTNOTE
A. John Alexander
After some dozen years' immersion in intelligence, I still find myself reacting uncomfortably to its rather cavalier disregard for the footnote. In that strange way each profession has of altering accepted words to its own meanings, "footnote" in the jargon of the intelligence community designates primarily the notation of a major disagreement on the part of a member with an otherwise agreed estimate. Here, however, I am referring to the footnote in its academic, scholarly, or scientific sense, as a device for identifying and in some cases even evaluating the source material used for a particular textual statement. Such a footnote is deeply scorned by practitioners of intelligence and makes only a rare appearance in most intelligence products.
During my years of intelligence apprenticeship I of course noted the omission, but I assumed that the master craftsmen knew best and there were very good reasons for it. I assumed that the suppression of footnotes was part of one's overall conversion from scholarship to intelligence: the paramount need of intelligence was a timely answer to a current problem. Intelligence could not afford the luxury of extended research, the comforting security of having explored all possible sources, the devotion of a lifetime of effort to the isolation and exact determination of one particular item of knowledge-culminating in a painstaking and exhaustive documentation of the entire research process.
And now, I suppose, after these several years I am something of a master craftsman myself. I have my brood of apprentices-and I teach them the same doctrine and they practice it. But throughout the whole process I continue to be troubled. I wonder if the abandonment, for the most part, by the intelligence community of the somewhat elaborate and carefully developed apparatus of scholarship has been altogether to the good. I wonder if we have not in fact been paying for it by an undesired but nevertheless real degradation of the intelligence effort.
As one trained in the rigorous academic disciplines, I find abandonment of the reassuring apparatus of scholarship disturbing in itself. But it is more than this general loss that disturbs me. There are certain specific practices that also provoke a sense of uneasiness. For example, and I find this quite ironic, the higher the level of the intelligence product, the less complete is its visible documentation. In other words, the more serious its import and the closer it is to the influential official who will act upon it, the slighter is its overt back-up.
At the lowest level, of course, is the raw intelligence report. This report is generally extraordinarily well evaluated and supported. No scholar could really, within the normal limits of national security, ask much more. The source, particularly in CIA-originated reports, is carefully and intelligently described as to his professional knowledge and competence, his outlook, his opportunity to gather the information, and his previous reliability. Not only the date of acquisition of this information but place as well is given. In some reports the rapporteur also provides a field evaluation of the substantive information elicited from the source. The user of this kind of report can easily and effectively apply the canons of evidence in evaluating and testing the information.
But as we move up the ladder of intelligence reports the documentation gets sparser. The NIS, to use a well-known example, is in effect a scholarly monograph, digesting a great multitude r € raw reports. Its total documentation usually consists of a single, very brief paragraph commenting on the general adequacy of the source material. No individual item within the NIS section can be tracked down to a particular source or specific group of sources. As one moves in the NIS from the individual chapter sections to the overall brief, the documentation becomes even more general and less meaningful.
At the more exalted level of the NIE, documentation even in the generalized form of comments on sources has usually disappeared altogether. One is forced to rely on the shadings given to "possibly," "probably," and "likely" and on other verbal devices for clues as to the quantity and quality of the basic source data. These examples from the NIS and NIE are paralleled in a great many other publications of similar refinement. One may admire the exquisite nuances and marvel at what a burden of knowledge and implicit validation the compressed language of a finished "appreciation" can be forced to carry, but one cannot help being concerned about the conclusions. Upon what foundations do those clever statements rest?
If the final products were at least based upon documented intermediate inputs, the uneasiness might be somewhat less. But in my own experience the "contributions" or inputs, with the exception of certain economic papers, are normally devoid of any specific identification of the kinds and types of reports or other evidence upon which they are based. And in my experience those inputs are often based on other inputs prepared at a lower echelon until at last we reach the analyst with access to the raw data. At the upper level of joint or national discussion and negotiation and compromise, which eventuates in tl.e exquisite nuance, the carefully hedged phrase, or sometimes a dissenting footnote, the remove from the original evidence can be, and often is, considerable.
The situation is not, of course, quite as dire as I have portrayed it. The intermediaries, in the process of review and consolidation of inputs, do query the preparers of these concerning items of unusual importance or of a critical nature, and in some cases they join the basic analyst in an examination of the raw data itself in order to get a firmer grasp of a particular issue. Furthermore, the final product, before being accepted and promulgated, is often returned to the analyst who prepared the initial input, and he has an opportunity to note any deviations from what he believes the situation to be. These processes do provide a measure of control and cross-check, some assurance that the available material has been thoroughly exploited and properly interpreted. But such processes seem partial and makeshift at best. They do not always occur. And they do not, of course, provide external participants in the final product with any real insight into the quality and quantity of material utilized by their fellow participants.
Another situation that troubles me--and this is a related problem--is the vast array of editors and reviewers under various guises and the several levels of examination to which an intelligence product is subjected before it is finally approved for publication. What troubles me is not the review, but the basis upon which it is accomplished. I recognize that many of these reviewers are highly talented, experienced individuals. Many are extremely devoted and conscientious and do their best to do a thoroughgoing job. But what basis do they have for their exalted "substantive" review?
In my experience, these reviewers have not generally--the notable exception would be members of the Board of National Estimates--been systematically exposed to the current take of raw data. Their knowledge of current intelligence events is based on hurried reading of generalized intelligence reports or on sporadic attendance at selected briefings. They are not aware in any particular instance-nor should they be-in any real detail of the material actually available on a particular subject. How do they know that this study in their hands for review has indeed explored the appropriate material? What variety of data has been utilized? Has the most recent material been examined? How can they do a spot-check on a particular item? Was a certain report seen, read, evaluated, and then discarded as erroneous, or was omission of the data in it inadvertent?
Lacking the apparatus of documentation, the reviewer generally has available only two methods by which to analyze the draft before him. One is to discover an internal inconsistency which calls into question the paper's overall accuracy or logic. The other is to find a statement that seems to contradict something he may have seen recently in his generalized reading and, on a hunch, to question its validity. The great bulk of any study, despite the reviewer's best intentions, is beyond his capability to question, analyze, evaluate, or critically review. What a haphazard and random method this is for highlevel substantive critique!
As a result much high-level review, in my experience, has consisted of the discovery of occasional typographical errors, small inconsistencies in numbers cited in different paragraphs or on different pages, minor inconsistencies in nomenclature, say between a figure or chart and a textual reference, unpreferred usage in spelling or hyphenating certain words, and other venial errors which a diligent proofreader should have caught. Any commentary on substantive validity, depth of research, or adequacy of analysis has been rare and exceptional. The minor changes are dutifully made, assurances given that more care will be exhibited next time, and the study is accepted and published as the agency's or the community's considered view.
I know that this is the system we live with, and I know that it often works surprisingly well. I know also that at times there are many vigorous discussions involving substance, and that in this oral exchange there is often a rigorous testing of propositions by an examination of the pertinent evidence. But much reviewing is done without this stimulating personal dialogue, without considering the evidence, and it is of this that I seriously wonder, is it worth the time and effort? Are we in fact getting our money's worth? Or are we not deluding ourselves? Is the review structure we have erected to assure ourselves that we are getting a high quality product not for the most part really a mere facade? Does the Emperor have any clothes?
If reviewing is sometimes a pious, well-intentioned fraud (one that I myself have had to commit), analysis at the basic journeyman level also at times leaves much to be desired. Not all analyses, of course, are based directly on the raw data, with its usable annotations and evaluations. Much analysis incorporates so-called finished intelligence, some of which is poorly dated, and the exact sources of which are not at all identified. Even the good and conscientious analyst does not know, nor does he have any means of learning, upon how solid a foundation that finished intelligence is based. It has an official imprimatur; so, not having supporting raw data in his files or time to procure and re-examine it-and, more important, following the traditional procedure of analysts-he uses it in his own study. His product eventually becomes a new piece of finished intelligence, which he or his successor will use in yet another study. And so the fragile structure can continue to be built of fragile materials. The weaknesses continually compound.
Another danger is the overconfident, glib, and persuasive analyst who writes his studies "off the top of his head." He can prepare a report rapidly and defend it with great self assurance, relying on his memory and general knowledge of the subject matter. Sometimes this assurance is justified. But how do we know when? Then there is the intermediate intelligence officer who sometimes, for whatever reason, ignores his analytical staff and prepares a report on his own again off the top of his head. It gets into the chain, and how is the next reviewer, or even consumer, to know that it has no substantial basis of research?
The hazards of insufficient documentation are evident enough to need no further elaboration. The value of proper documentation, moreover, and the system for it are not unknown to intelligence officers of the community. Most--whether in uniform or out--have at some time in their formal training been exposed to documentation and its virtues, if only in the preparation of a term paper. Many continue to evaluate externally prepared reports and monographs in part by reference to their bibliographies and footnotes. The scholarly habits persist--except in the intelligence field itself.
Part of the reason for this condition is an item of cardinal intelligence doctrine: do not betray the source. Concern for protection of sources is of course legitimate, but it can be carried to extremes. As illustrated above, there appears to be a contradiction in the respective application of this doctrine to raw reports and to finished intelligence. Meticulous definition of the source in an individual raw report is accepted (and correctly) as necessary to the proper appreciation of the report's content. It would appear equally necessary in finished studies derived therefrom.
The argument can be made that finished intelligence has a wider circulation than the raw reports and that there is therefore a greater risk of jeopardizing sources by identifying them in the finished product. In some cases this concern may indeed be valid-and could certainly be met by producing undocumented versions for the bulk of the circulation. But for internal consumption by operating officials who want to know (or should want to know!) the actual amount, validity, and reliability of the basic information, a documented form should be available. And it should certainly be available during the process of shaping up the final report-to the intermediate analysts, reviewers, and negotiators.
I am not persuaded, however, that fear of source compromise is a wholly valid argument. Footnotes will reveal report numbers, subjects, place of origin, and rapporteurs, but would not necessarily identify sensitive sources. Many sources are open or obvious and could be cited without danger. If a source is particularly sensitive, even its nature need not be revealed, but a neutral documentary reference should make it possible for a properly cleared user to run it down. (In exceptional cases of extremely sensitive sources it might of course be necessary to prepare versions at that level of sensitivity.) With effort and imagination, I believe that the source-compromise problem can be successfully met. One practical suggestion is included in the procedure recommended below.
Another argument that can be and often is advanced is that documentation is time-consuming and time is a luxury that intelligence cannot afford. Admittedly it is time-consuming to prepare documentation; it would increase analytical, typing, and perhaps reproduction time. It could even be argued that it would increase editing, review, and final processing time. This is a plausible argument-but anyone familiar with the realities of much intelligence production will, I'm afraid, be unimpressed. Anyone who has been personally involved with the time lags in production of NIS sections, say, with the prolonged back-and-forth traffic of editing and "nit-picking" at most routine papers, will not believe that in much intelligence production time is quite so greatly of the essence. I strongly feel that the additional burden would be more than compensated by the improved substantive quality of the final product and that, as a matter of fact, much time would be saved. There would, for example, be no frustrating searches for the uncited sources of questioned statements.
It can also be argued that footnoting is a cumbersome, awkward, and excessively time-consuming method of documentation-and here I would agree. I would not, for intelligence purposes, advocate the adoption of the formal, extended-entry, bottom-of-the-page footnote system, requiring exasperatingly frequent repetition of document source and title and producing further complications in proper textual alignment and pagination. I would propose a very simple system based upon that used in scientific journals. In this system sources are listed in a single bibliography and numbered serially. Textual references to sources are made in parentheses following the relevant statement by use of two groups of numbers separated by a comma, the first identifying the source by the number it has in the bibliography and the second giving the page reference.
Extended discussions of particular source problems can appear as a series of appended numbered notes, referenced in the text by the appropriate note number in parentheses. This system is easy to employ and should present no difficulties to the analyst; it should cause only minor inconvenience to the consumer. And if a particular report needs to be sanitized quickly of specific source references the bibliography and appended notes can simply be detached.
Why documentation has languished so long and amiably in desuetude in the intelligence community I do not know. Inertia and the relief from old academic requirements may be part of the answer. But however it came about, the present non-documentation system is well established and flourishing. The habit is almost an addiction. Efforts to upset it fly in the face of human laziness, tradition, even vested interest. In a sense, it is job protection for the mediocre analyst: it does not expose his work to careful examination. Years of living with undocumented intelligence has blunted our perception of its dangers and inadequacies. The voice of protestor -- or is it conscience? -- that is sometimes heard is exceedingly small. Yet I think it is challenging.
Import an Old Revolution
It seems to me that we need a major revolution in intelligence doctrine. What we need is the intelligence equivalent of the Academic Revolution that occurred in our schools of higher learning some hundred years ago when modern research methods were first introduced, primarily from Germany. This Academic Revolution, as all students of intellectual history know, brought to graduate academic disciplines (both scientific and humanistic) the tools, concepts, and apparatus of modern scholarship. Along with concepts of free inquiry, thorough exploitation of original sources, and objectivity it brought the requirement for precise documentation. A common methodology and certain common standards were developed; and the field of scholarship, originally the domain of the self-trained amateur, gradually became professionalized.
Intelligence is undergoing this kind of evolution. Its operations are becoming professionalized; a professional esprit and a common methodology are gradually developing. This journal has been an important step in that direction, following the classic pattern: it provides a necessary forum for the discussion of professional problems and helps create a common background of classic cases, basic concepts, general principles, and key problems in intelligence. It is in this forum that I should like to see argued out the advantages and disadvantages of a proper documentation of intelligence conclusions and findings. I have stated -- perhaps overstated? -- the case in its favor as a real necessity. Is there a valid defense for the status quo?
In addition to a serious, probing, and hopefully rewarding discussion of the problem, I would also recommend experimental application of the proposed doctrine to some specific areas of intelligence production. As a beginning, I would suggest it be tried on selected NIE's and NIS's, with careful evaluation of the results after reasonable trial periods. Do they seem worth the additional encumbrances? What is the response of consumer officials to the improved documentation? Has there indeed been a qualitative improvement in the product? Or is it clear that formal, detailed documentation has no real part to play in intelligence, that it is and has been properly excluded from intelligence methodology?
In addition to this formal trial on standard products, it seems to me that policy officials requesting ad hoc intelligence studies or reports could very well consider including among their proposed terms of reference a requirement for thorough documentation. Since such a requirement may not occur to them (assuming they are unlikely to have read this particular plea), the intelligence officials discussing the proposed terms of reference might suggest it be included. Let us make the offer and see if it is opted.
The end result of this discussion and selective application should be the development of an agreed working methodology for intelligence documentation. The methodology must be realistic. I should not like to see (and shudder at the possibilities!) the establishment of inflexible requirements for its application. The apparatus of documentation should be applied only where it helps, not where it hinders. Certainly daily field operational intelligence is an area where it might prove to be an impediment and costly luxury. But through intelligent trial and error a practical doctrine should evolve.
A system that has proved its worth in every other professional field surely deserves careful examination and consideration by members of this one. It does not seem too soon to consider applying here the concepts of a revolution now some hundred years old.