Intelligence Research--Some Suggested Approaches
APPROVED FOR RELEASE
CIA HISTORICAL REVIEW PROGRAM
22 SEPT 93
INTELLIGENCE RESEARCH -- SOME SUGGESTED APPROACHES
Research may be divided into two general activities, to collect information or extend knowledge, and to answer particular questions. Intelligence research properly consists of the latter kind. Because the problems of the intelligence community are many, research activity must be focused not only on intelligence problems but also must be directed at targets of highest priority, in order to make the most efficient use of the community's limited manpower and money.
Intelligence questions may range from such narrowly defined topics as how many man-hours it takes to produce a Soviet tank to broad inquiries about the industrial capacity of a satellite nation. Intelligence research may be undertaken for immediate, or current, use; the depth of an obscure harbor, for example, must be ascertained before an invasion, or a study of the economy of a country is called for because it will serve as a guide for answering more specific questions that may arise on short notice.
Even in such broad projects, however, the object of the intelligence research is not encyclopedic information; it must be limited to information that answers questions of intelligence interest. It is essential, therefore, both in planning and conducting intelligence research, that its urgency and its purpose be constantly borne in mind. From these imperatives will stem the interest, incentive, accuracy, and imagination required for creative work in intelligence.
Although intelligence research has much in common with other purposeful research, there is no single, simple technique which will solve all research problems. It may be said with equal truth that no one method is appropriate to all kinds of intelligence research. Techniques and methods must be adapted to the problem, its scope, its urgency, and to the nature of the evidence. It should not be forgotten that the researcher, himself, is a variable quantity. No two analysts are likely to use the same methods in solving a given problem. Within limits, therefore, research methods must be gauged to the training, background, and interest - the personality - of the intelligence analyst.
Although the intelligence research process is varied, it may be suggested that it has much in common with other research in the social sciences. Any intelligence research project may be broken down into basic operations, according to their role in the project as a whole. These operations may be termed (1) Project planning, (2) Collection of data, (3) Analysis of data (a general term, here), and (4) Presentation: Writing the report. Although these operations are not performed entirely in sequence, usually they take place in the order named, allowing, of course, for considerable overlap and a human tendency to back and fill.
These are arbitrary divisions. There is nothing natural or inherent about them, nor for that matter, about dividing the whole process into four parts. Other names might do as well, and undoubtedly more than four skills are used in any given project. The four stages chosen here are merely suggested as convenient divisions for discussion.
Because an intelligence project seeks to find an answer to an intelligence question, it is of prime importance to secure an early understanding of the problem to be investigated.
If the problem is a question from the National Security Council, it probably will be clearly stated in the Terms of Reference put out by the Office of National Estimates. Another kind of problem will have to be formulated internally in terms of projects initiated at or below the divisional levels. In either instance the object is to overcome ignorance in a matter of intelligence importance. To plan the project, then, it is essential to know just what it is that must be learned about the subject in the time available. When the question is understood, it becomes possible to define the scope of the project in terms of a) what is relevant and irrelevant, b) what is known and what must be investigated, and c) the number of manhours to be allotted to the work.
In order to achieve full understanding of the question at this stage, it is also essential that the analyst think the problem through, going beyond a mere statement of the question. Although such an analysis may appear premature, it is imperative that the problem be mulled over thoroughly for disclosure of its implications and ramifications and that these be formulated in the shape of a preliminary outline of what is desired to be known about the subject.
At first thought, it may seem impossible to outline a project until the research has been completed and the threshold of the writing stage has been reached. When a project is conducted in this manner, however, it tends to veer away from purposeful, sharply focused research to fuzzy shotgun collection characterized by uneven coverage and inefficient allocation of time.
Nevertheless, outlining a project when it is still in its planning stage and before any research has been undertaken on the subject admittedly raises difficulties.
This initial outline presents not all that is known about the subject, but rather all that we wish to know. The analyst can infer, from the mission of his component, the kind of information that will be sought.
This is not to suggest that initial ignorance of a subject is an advantage, but merely that it is not as great a handicap as appears at first glance. An analyst with a technical knowledge of automotive construction, of aircraft factories, or tank factories in the US, in a sense already partially knows what to look for when undertaking a study of the Soviet ability to satisfy requirements for these items.
Naturally the analyst will want to brief himself on what is known about these industries in the USSR. Where, in a new research project, a scholar would turn to a short, general account for a quick survey of the kind that might be found in an encyclopedia, textbook, or technical monograph, so an intelligence analyst has recourse to similar summaries in the intelligence field, such as the appropriate chapter of the National Intelligence Survey, one of the National Intelligence Estimates, or completed intelligence reports on the subject. Occasion ally, however, the analyst finds no background reports available, so he must begin his research without the benefit of organized information.
From his background experience, his scanning of general reports on related subjects, and his initial analysis of the problem, the analyst should be able not only to prepare a preliminary outline of questions about the subject, but also to begin to think about the precise kind of data to look for as well as ways of processing the information in order to find answers to the problem.
Having made preliminary plans for a project, the analyst is justifiably eager to begin accumulating data. This impatience is understandable, but it is also an urge that should be restrained until a survey can be made of the amount and kind of information available. In a well-organized research project, the analyst first makes an inventory of accessible information and sets this up in the form of a bibliography or list through which he can then work systematically. The completeness of the inventory depends in turn on whether the project is a basic study or whether it is merely a quick answer to a simple question. If the study is to be exhaustive, then the search for material should be systematic and intensive.
Unlike the academic research worker, the intelligence analyst may draw on a world-wide collection organization. In a sense, he has at his command all the collection resources of the US government as well as of certain other nations. He can also draw upon information possessed by private organizations and independent specialists. Much of the discussion of this article has reference to the CIA facilities, which are available to analysts throughout the intelligence community.
Because the data collected by CIA are great in quantity and growing rapidly, machine methods have been devised to help get information from the mass of data in the CIA Library (a part of the Office of Central Reference - OCR). IBM machines are used to sort the punched cards upon which most documents are coded and books are catalogued.
The Dewey decimal system is embodied in a book entitled Intelligence Subject Code, a volume with which all intelligence analysts are vitally concerned and with which all should be familiar. This book is used in the reference services of CIA, the Air Force, the Signal Corps, and other components of the government. In order to request documents from the Library, the analyst first selects the appropriate numbers from the Intelligence Subject Code and requests a tape run on these numbers. The Machine Division of OCR will mechanically sort out the cards that fall within the requested number limits, place the selected cards in an intellofax machine, and send the resultant tape run to the analyst, via the Library. From the tape run the analyst selects those items that pertain specifically to his problem and orders the documents from the Library.
This process seems deceptively simple because it is a mechanistic description; what has been left out is the analyst's ingenuity in selecting the numbers from the code and his ability to identify a likely looking document on the basis of the title alone. In the use of the code, he can rely on expert help from a librarian; but in calling for the documents proper he must either rely on his own perception or order all documents of any possible use.
This, then, is the first step in preparing a bibliography of the information in CIA bearing on the project. This sort of information will often be mostly classified, as opposed to open literature. In some instances, however, it will be found that more valuable information is to be found in the open or unclassified sources than in the classified ones. After checking the CIA Library card run for such open sources, the analyst should then visit the various appropriate departmental libraries and the Library of Congress. A complete list of all the libraries in the area of the District of Columbia, with a description of what they contain, who can use them, where they are, and whether or not they are members of the interlibrary loan system - entitled Library and Reference Facilities in the Area of the District of Columbia, is available in the CIA Library.
Although the library facilities in the Washington area are extensive, on certain subjects the best specialized libraries are elsewhere in the country. The locations of such special collections can be found by checking the index of the useful list entitled Special Library Resources (New York, 4 vols. 194147) published by the Special Libraries Association.
This initial effort to locate materials may or may not turn up much pertinent information. In any event, the scanning of card catalogues is only part of the search for sources. On the whole, one will find in these card catalogues references only to books and pamphlets and not to articles in periodicals. The card catalogues generally will be strongest in information about domestic affairs, and the analyst will have only an incidental interest in the US data. It is necessary, therefore, to secure better coverage on pertinent foreign printed sources than exists in library card catalogues. For this the analyst should consult the Readers Guide to Periodical Literature.
Those delving into subjects in the areas of economics or politics should check the issues of the Public Affairs Information Service. For technical and engineering references, the place to look is in the Engineering Index, the Industrial Arts index, or other appropriate guides to books and articles. These guides appear currently and are bound annually. By a patient and ingenious search through such works the analyst can be reasonably certain of learning what unclassified information on his subject published in the US is available.
The New York Times Index is an excellent example of a newspaper research source, and other standard sources include the International Political Science Abstracts and the International Bibliography of Economics. These are just a few of the research aids available in a standard library. The CIA Library includes the Intelligence Publications Index (Secret, Noforn), Selected Reference Aids to Cyrillic Alphabet Materials CIA/CD (sic) 1952 (Secret), the Monthly List of Russian Accessions, and the East European Accessions List.
In addition to the CIA Library, OCR includes the Biographic Register (BR), which contains information on foreign scientists, industrialists, and social scientists; the Graphics Register (GR), which maintains files of photographs of intelligence significance and will provide assistance in the use of photographs for intelligence purposes; the Industrial Register (IR), which contains information on foreign industries, industrial resources, individual plants, companies, and related research and commercial activities, plus information on ports, power plants, pipe lines, inland waterways, communications, and storage and other facilities; and the Machine Division (MD), which supports the operations of the other divisions.
As a result of this inventory of available materials, the analyst- although he will have collected very little substantive information -will have a valuable checklist of where to go for what. This inventory should also reveal which parts of the project outline can be thoroughly answered and which parts represent gaps in immediately available (that is, in the Washington area) information. At this point he can begin to initiate new requirements and request any needed translations of foreign language material.
Now the analyst can start digging into the information itself, recording it, and placing it in his planned file, where it can be retrieved easily. A part of this recording and filing procedure should be a consideration of the significance of each piece of information and an assignment of a priority to each, so that as the parts are used, the most important information will be retrieved first. (The same consideration should be applied in considering which references to consult first.) Such priority assignment should be a standard practice in intelligence research and analysis, as most projects are scheduled against deadlines.
In building a file of information for a project, it is usually best to work from the general to the specific, to read first the previous reports on the subject as a whole and then to work on into more detailed aspects of the subject, in as logical a sequence as possible. Thus the analyst should seek first to master the history, technology, organization, output, and other pertinent characteristics of an activity, industry, or science from comprehensive, over-all, evaluated intelligence reports before attempting to cope with the mass of unevaluated intelligence reports that will be encountered.
Unfortunately, the intelligence process - which requires specialization and compartmentation and which balances security and efficiency -is not conducive to accuracy. At each step in the process of transmitting information, in translating it from one language to another, and in editing it, errors are likely to creep in and distort the original meaning. For this reason, accuracy is best served when the original document can be incorporated directly into the working file. This also saves the time of the analyst. Of course, this policy must be balanced with the need for filing information by units. Suppose, for example, that the analyst received from FDD a special hundred-page report on the nomenclature of his subject, breaking it down systematically and providing detail on each aspect of it. For the analyst to reduce all of this detailed information to cards for inclusion in a card file would be not only a dubious allocation of scarce time, but also a violation of the principle of accuracy. Even if his files, as set up, provide for incorporation of data, topic by topic, it would be a good idea to preface such a section with a category into which material such as this translation - which cuts across more specific entries - could be placed.
This example, however, is exceptional. More often the analyst may note in a source, a single paragraph dealing with a unified subject. When the item is very short and stated clearly, and when it could not be misconstrued even out of context, it may be appropriate to make a handwritten or typewritten note for the file. Even here it is preferable to quote rather than to paraphrase.
If the item is slightly longer and still deals with one subject, however, accuracy and efficiency both can be achieved by obtaining a copy of the document for clipping. When clipped, the information can then be marked for reference and incorporated in the file without anxiety over whether or not accuracy has been sacrificed in the process.
Clearly there are times when a long document must be paraphrased and condensed into brief notes. The analyst must then make the conversion from the extended statement to a brief, usable account without distortion of meaning or loss of essential information.
It should not be forgotten that the comprehension and reliability of the analyst are not constant qualities. Skilled as he is in a general area, his competence at the beginning of a new project is less than it will be in the later stages of that same project. Early in a project, therefore, he should be careful both to extract from the material all of the pertinent data and to protect himself from unconscious errors in recording data.
A note file is adequate when a balance is achieved between collecting too little and collecting too much. Lest this be construed to mean, "if it is adequate, it is adequate," note that "too little" or "too much" are relative terms and acquire meaning only when applied to the project or measured against the amount of information necessary to answer the intelligence question.
The analyst who makes too few notes will be prone to conclude erroneously, later in the project, that gaps of information exist which preclude a satisfactory completion of the report. And the one who makes too many notes will have difficulty meeting his deadline, will have to condense his material further by making notes on his notes, or may find himself so overwhelmed by information that he loses sight of his mission and writes a disorganized report.
It is equally embarrassing to make notes that later prove to be incomplete because some essential point of fact, such as quantity, place, date, evaluation, or classification, has been omitted. Each note should be scanned for this type of completeness. The time to do this is when making the note, not when the document has passed on for further routing or when an extensive file has been returned to its cabinet.
It should be emphasized that notes should be documented when they are made. Although it may seem onerous, the documenting of notes is an essential step in the intelligence research process. An undocumented note may contain an almost priceless gem of intelligence information. Unless, however, the note can be related to its source, it is useless to the analyst, and of no value in a report. If a note is completely documented, the analyst can assign the proper importance to it based on its source, date, evaluation, and other facts. Such a note thus contains more meaning than one that is only partially referenced. It permits comparison of information with that contained on other notes and facilitates decisions as to whether to use the information, where to use it, and what weight to assign to it. Finally, it makes possible the use and citation of the information in a report without having to go back to the actual document.
With these principles and methods well in hand, the analyst proceeds with the gathering and filing of his data. He will find that from time to time it is desirable, in the midst of a research project, to pause long enough to survey the status of things. This operation may take the form of skimming through the files to determine the areas in which coverage is complete and those in which information has been coming in slowly. (It will be facilitated if notes are filed when made, or shortly thereafter.)
The prompt filing of material and the periodic review of the files makes one aware, also, of the organization into which the material is falling. This review encourages continuous critical assessment of the organization (and hence of the outline) and makes possible a more logical and finished report structure than would result if notes were filed serially and not organized until the writing stage.
The review should consider both the detailed level, as just indicated, and the project as a whole. Incoming information should be tested against the criterion - given an appropriate methodology - of its utility in answering the intelligence question upon which the project is focused.
Too often the organization of a paper is deferred until after material has been collected and when the pressure to begin writing has mounted to a considerable degree. Countless papers have been organized as they were written. To do this reduces not only the quality of the writing, but also the clarity of the answer to the problem. To avoid this, projects should be thoroughly planned and tentatively outlined during the planning stage and, as it comes in, data should be incorporated into a working file. In a well executed project data is not collected at random; it is sought because the analyst has in his mind at least the glimmering of an idea that this data can be worked over in such a way that it will contribute to the solution of the problem.
It is the manipulation and marshaling of the data that constitutes the methodology of the paper. No one method can be recommended as being suitable for all situations; a method must be adapted to the question and to the available information. In terms of these factors, the method appropriate to any particular project may range from a simple, direct approach to a complicated statistical manipulation. A few possible approaches are suggested.
In terms of efficiency of operation, the natural place to look for the answer to an intelligence question is in the open literature of a country - official census reports, the results of other national surveys, the reports of trade and technical associations, the reports of nationalized institutes and industries, and the various professional journals published. In recent years the Soviet Bloc has resumed the publication of detailed statistical handbooks and yearbooks. In many areas, these publications greatly ease the collection problem of the analyst. They do not, however, include military information, such as tank, aircraft, or artillery production. Analysis has shown that official statistics from the Soviet Bloc are not to be dismissed lightly as Communist propaganda. On the other hand, they should not be accepted uncritically without being checked for internal consistency, consistency with other official data previously released, and agreement with related sources, open and covert.
Care must also be exercised in arriving at a precise understanding of definitions upon which official statistics are based. What, for example, do the data on East German figures on crude steel production include? Close examination and comparison with plant production have revealed that the published figure of crude steel output fails to include a considerable quantity of steel for casting produced from scrap in large machine building plants. Likewise, students of Soviet Bloc national income have pointed out that economic aggregates, such as national income and gross national product, are defined differently in Communist countries than in the Free World and hence must not be compared with official data from the Free World without appropriate adjustments. Thus uncritical and uninformed use of official data is subject to hazards.
Suppose, for example, that the analyst wants to know the answer to a question which is a classified matter in the target country - the production of a military end item, for example. By surpassingly clever use of bibliographic techniques, an analyst may be able to determine precisely what he wants to know. If the USSR is the target country and the information is not overtly published, getting the answer may require a fairly high level of covert penetration. To corroborate the facts, the covert effort would have to be duplicated by a second and independent substantiating report - another operational project.
Where national statistics, either overt or covert, are difficult to come by, attention must often be shifted to the other end of the scale, and directed at the collection of production figures for an institute, product, or region. Finding this type of information requires a painstaking search through technical publications, press, and radio accounts, as well as careful evaluation of covert reports. Even then it is unlikely that the entire picture will be exposed; information may be available on some aspects and not on others. For this reason it is usually necessary to combine over-all approaches with calculations based on less direct indicators of activity.
When total effort in a field is known and the target consists of a segment of that total, then the unknown segment can be computed simply by subtracting production of known items from the total. Again, this method is often applicable in conjunction with other techniques.
When progress in a field has been established on a firm base for a series of years, it is then in order to project, or extrapolate, that trend to ascertain future developments. The reliability of this projection will depend in part on the accuracy of the knowledge of the field and upon whatever variables may come into play in the future.
If the analyst knows generally the technology of the field within which the project falls, he may be able to make valuable estimates of requirements, schedules, and capabilities. Thus with a sample of Soviet penicillin on hand for laboratory assay, with a knowledge of the growth curves of the strain of penicillin used by the USSR and of the composition of the culture medium, a well-grounded analyst may be able to give a good estimate of Soviet penicillin production.
In economic intelligence numerous correlations exist between inputs and outputs. To a considerable extent coefficients for these correlations are still being established. Where they are known they can be of great help in solving intelligence problems. They may consist of such diverse relationships as kilowatt hours of energy consumed per ton of output, floor space in square feet per employee, or number of motors of subassemblies of a given type per unit of product.
At times a knowledge of one nation's technology helps the analyst to understand the operation and requirements of another's. It is important, of course, that corrections be made for variations in efficiency, when that is possible. At other times such variations may be negligible and will not affect the reliability of the over-all estimate. Because the USSR and the US employ different methods of coal mining, it would not be appropriate to estimate Soviet needs for coal cutters on the basis of US data. Once Soviet production of coal cutters has been estimated, however, the input data for US coal cutters can supply a basis from which calculations can be made, with various adjustments for inputs into equivalent Soviet coal cutters. When reasoning by analogy, as in this output calculation, the starting base can be national estimates, industrywide figures, plant data, or information on particular models. Such data can be secured from War Production Board files, Munitions Board data, industry consultants, and various intelligence sources.
Whichever method is used, there is great opportunity for an imaginative approach to the problem. The methods discussed are examples of the many research approaches. In any project many combinations of method are in order. Indeed, it is highly desirable to check any given method by another and independent procedure.
An intelligence report should be thought of as the answer to an intelligence problem. In this sense it should raise the question, present the answer, indicate the gaps as a guide to collection, explain the method used in achieving the answer, and document the discussion so that any reader may conduct an independent check of the results.
Although some sections (such as organizational or installation summaries) are best written as the research on that portion of the total problem is completed so that the information will be fresh in the mind of the writer, it may be argued that it is preferable on the whole to defer the bulk of the writing until most of the material has been collected and digested. The analyst gains in comprehension of the subject as he collects and organizes his material. He is therefore more likely to do a better job if he defers writing until he finds that the incoming material is beginning to be repetitious and unrewarding. By writing during the latter part of the period he also has the advantage of having organized and reorganized his files and of having gone over them several times.
If the files are well organized they will correspond fairly closely to the revised project outline. When the files are well organized it should also be possible for the analyst to place at his finger tips most of the information he needs to write any given section of his report. Although he will also need to have recourse to certain general information, it will not be necessary for him to attempt to assimilate during the writing stage numerous undigested reports covering a number of different subjects. If he has used the unit note system and has organized his material well, his files will be-in logical sequence and he can arrange his individual notes on any particular part of the subject in the order in which that information is to appear in the text of his paper.
When reports include quantitative information, such as requirements, production, and input figures, it is good policy to work up all tables and graphs before writing the text, rather than during the writing. If the tables are prepared in advance and if each table is carefully documented at that time, the writer will have the advantage of greater perspective; he will be able to simplify his text by references to the tables, and in addition he can greatly reduce the amount of long and repetitive documentation, again by referring to the tables. From every point of view it is wise to prepare the tables and graphs in advance of writing the text.
The writer should make clear to the reader the nature of the problem to which the report is addressed, and he should indicate what is included and what is excluded. The analyst must define terms the first time that each is used, then use the terms consistently. He should not introduce technical synonyms without indicating with what they are synonymous. Table headings should be consistent with one another and with the text.
It is important to anticipate, insofar as is possible, the character of the group to which the paper is addressed. Readers may be various kinds of specialists. At the same time that a report must prove instructive to fellow specialists it must also be understandable by persons whose interests are more general.
Because it is the practice of most members of the intelligence community to place citations in an appendix at the end of a report, footnotes are reserved for parenthetic and explanatory remarks that could interrupt the flow of thought of the body of the text.
The purpose of documentation is utility rather than an exhibition of scholarship. A good working rule to follow is that citations must permit the reader to make an independent check of factual statements. Therefore, each separate fact that is not a matter of common knowledge presented should ordinarily be buttressed by a documentary citation. Books should be cited by author, title, date, and page (publisher optional, a matter for the production staff of the analyst's component). Articles in periodicals should show in addition the name of the journal. Intelligence documents should be identified by the symbols assigned by the issuing office, which will therefore bear meaning throughout the intelligence community, rather than merely by the accession number of the component library. In general, documentation should disclose to the reader the same types of information as suggested in the section on data gathering and should include date of source, date of information, classification, and evaluation; these should be presented in the form required by the production (editorial or publications) staff.
In addition to the standard list of references in each report, it is sometimes useful to provide the reader with a critical bibliography. In this section each of the major sources used in answering the intelligence question should be evaluated in a few words.
The method used in the body of the text to solve the intelligence problem should be made explicit enough so that the reader may follow the logic as he proceeds through the paper. If this is done in the text,- then the appendix section on methodology may consist of a brief resume of the methods employed.
It is vitally important to attach to the report the analyst's assessment of the gaps in intelligence. These gaps will serve as a basis for levying additional requirements. The gaps also will indicate to field recipients of the report where collection effort should be concentrated.
Whereas it is important that the research analyst develop a high degree of interest in his project, in order both to motivate himself and to be able to prepare a good report, it is equally important that he maintain an attitude of detachment. Although the information contained in the report may contribute to the making of policy, it is neither the analyst's responsibility nor his mission to make policy.
It is equally important to retain a sense of objectivity toward the problem. The analyst should explore the various hypotheses he develops in the course of his work. He should follow the evidence where it leads him. If necessary he must make choices, evaluations, and judgments about which data are factual and which are false. He must not become enamored of any theory or position to the extent of losing his sense of judgment. He should regard himself as a scientist who has worked with a problem, considered the evidence, drawn a conclusion, and presented both the evidence and the conclusion.
Then he should circulate his report for revision and criticism by his fellow analysts. He is not expected to be infallible, and his answer can be no more than the best possible at that time.