Document Type: 
Document Number (FOIA) /ESDN (CREST): 
Release Decision: 
Original Classification: 
Document Page Count: 
Document Creation Date: 
November 16, 2016
Document Release Date: 
February 3, 2000
Sequence Number: 
Case Number: 
Publication Date: 
March 1, 1965
Content Type: 
PDF icon CIA-RDP78-03952A000100050001-7.pdf20.08 MB
Approved ForRelease 2000/05130 : CIA-1DP78-03952A0q, 0 SYSTEM ORGANIZATION, FUNCTIONS, AND PROCEDURES DIRECTORATE OF SCIENCE AND TECHNOLOGY OFFICE OF COMPUTER SERVICES df 1 Approved For Releas6-20601051,30`-: U1A-RDID78-03952A000100050001-7 GROUP Excluded Fr.rn ClO? 4 2114*,1 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 WARNING This material contains information affecting the National Defense of the United States within the meaning of the espionage laws, Title 18, USC, Secs. 793 and 794, the trans- mission or revelation of which in any manner to an unauthorized person is prohibited by law. Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 :go- 78-03952A000100050001-7 CONFIDENTIAL; 2/4.04,C- ././- Phase II Final Report Volume V SYSTEM oRWTgATTPL? FM4T115-0-nD PROCEDURES CHIVE/R-3-65 1 March 1965 DOE REV DATE al Mr Pi Y g 0R1G COMP TYPE OR1G MAAS PAGES REV CLASS - JUST NEXT REV eV/ Minh RR 104 CONFIDENTIAL Approved For Release 2000/05/30 : IA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cl CONR 8-03952A0Z:010-0Q TABLE OF CONTENTS Page 5.1. Introduction 1 5.1.1. General 1 5.1.2. System Overview 2 5.2. System Organization 9 5.2.1. Background 9 5.2.2. Proposed Organizational Concept 10 5.2.3. Position Descriptions 21 5.3. Data Base 31 5.3.1. The Selection Problem 31 5.3.2. Basic Selection Criteria 31 5.3.3. Sources to be Exploited 33 5.3.4. Level of Coverage 34 5.4. CHIVE Indexing Technique 41 5.4.1. Introduction 41 5.4.2. Concepts 41 5.4.3. System Description 53 5.5. System Files 5.5.1. Introduction Approved For Release 2000/05/30 : C -03952A000100050001-? - 67 67 Approved For Release 2000/05/30 : C 8-03952A000100050001-7 CONFIDENT1 Page 5.5.2. Document Index Files 73 5.5.3. Document Image Files 78 5,5.4. Vocabulary Control Files 81 5.5.5. Unsynthesized Information Files (UIF) 106 5.5.6. Summary Information Files (SIF) 115 5.5.7. Special Projects Files 121 5.5.8. Referral Service Files 127 5.5.9. Management Data File 133 5.6. System Flows and Transactions 141 5.6.1. Document Input 141 5.6.2. Document Retrieval 151 5.6.3. Information File Building, Maintenance and Retrieval 159 5.6.4. Task Tables for System Transactions 168 5.7. File Conversion 189 5.7.1. Introduction 189 5.7.2. Document Index Files 190 5.7.3. Document Image Files 203 5.8. Computer Interface 211 5.8.1. General 211 Approved For Release 2000/05/3 52k8001 -00050001-7 A?roved N411E11E140 8-03952A000100050001-7 Page 5.8.2. Command Language 212 5.8.3. File Definitions and the EDP File Analyst 216 5.8.4. Summary 218 5.A The Organizational Problem 221 5.A.1. Organizational Objectives 221 5.A.2. Alternative First-Level Organizational Concepts 228 5.A.3. Organizational Alternatives Within A Geographic Division 258 5.B Preliminary Evaluation of the CHIVE Indexing Experiment 273 5.B.1. Summary Description of Experiment 273 5.3.2. Preliminary Findings 279 Feasible Alternatives in Index Design 287 CONTIDEIVT AL Approved For Release 2000/05/30 : -03952A000100050001-7 Approved For Release 2000/05/30 : Cl NFIDENT1AL 2A000100050001-7 5.C. CHIVE Indexing Guide Page 297 5.C.1. Introduction 297 5.C.2. Content Indexing System 298 5.C.3. Header Data Transcription Guide 324 Tab A Code Schedules 351 Tab B Project CHIVE Tags 365 Tab C CHIVE Index Terms 387 Tab D CHIVE Header Form 388 Tab E Authorized Abbreviations/CHIVE 389 5.D Inherited Files 393 5.D.1. Introduction 393 5.D.2. Index Files 397 5.D.3. Document Image Files 463 Approved For Release 2000/05/30 : CS40511 V)11bW4A . 60100050001-7 Approved For Release 2000/05/30 : ClitigF81039L2ff, ,A,PrirrinVA I FIGURES Page 5-1 CHIVE System Flow Chart 3 5-2 UIF File Building Alternatives 112 5-3 SIF File Building Alternatives 120 5-4 Document Input Processing 142 5-5 Document Retrieval Processing 152 5-6 Information File Maintenance 164 5.D-1 List of China-Related Inherited Files 494 5.D-2 Vocabulary Control, Summary and Unsynthesized China-Related Inherited Files 495 5.D-3 Format A - SR Subject/Commodity File Card 496 5.D-4 Format B - SR (China) Area Detail File Card 498 25X6 5.D-5 Format C - SR Organization 25X6 File Card; SR Personality File Card; SR Foreigner File Card 500 5.D-6 Format D - SR Soviet Organization File Card; SR Soviet Personality File Card; SR Soviet Foreigner File Card 502 5.D-7 Format E - All Other Organization File Card 504 5.D-8 Format F - All Other Personality File Card; All Other Foreigner File Card 506 Approved For Release 2000/05/30 : 1111-03952A000100050001-7 Approved aNase eFIDal 0/05/30 : CIA 103952A000100050001-7 5.D-9 Format G - PI Subject/Commodity File Page Card; PI Area File Card 508 5.D-10 Subject/Commodity and Area Files 510 5.D-11 Organization Files and Derivative Files 511 5.D-12 Job 3 File Statistics 512 5.D-13 Reports Title Index 513 5.D-14 Job 3 Card Format 514 5.D-15 Job 3 (KWIC) Elements of Information 515 5.D-16 FIB Town/City Information Card Format 517 5.D-17 FIB Installation Information Card Format 518 5.D-18 FIB Location Cross Reference Card Format 519 5.D-19 FIB ICF Coordinate Card Format 520 5.D-20 FIB ICF City Cross Reference Card Format 521 5.D-21 FIB ICF Name Card Format 522 5.D-22 FIB Model-Type Brochure Index Card Format 523 5.D-23 Punched Card Characteristics of the IRS DOcument Index File (New) 524 50D-24 Punched Card Characteristics of the IRS Document Index File (Old) 525 5.D-25 Punched Card Characteristics of the Film Index File 526 Approved For RelpaRVF05ITFAUggr-03952A000100050001-7 laU141-1"1 Approved For Release 2000/05/30 LIP6E-1110.7 Page 5-1 CHIVE inputs 38 25X1A 5-2 Index Report 171 5-3 Over-Counter Document Search 175 5-4 Generation and Input Processing of Formatted Information/Index Records Prepared Under Contract 179 5-5 Information Analyst Activity Relative to an All-Source, All-File Search for a Named Personality 183 Approved For Release 2000/05/30 : CIA- CON ODZI') IAL 03952A000100 500v - Approved For Release 2000/05/30 : Clh-WER:5118-03952A000100050001-7 Chapter 5.1. INTRODUCTION 5.1.1. GENERAL This volume of the report is primarly concerned with the non-EDP aspects of the CHIVE system, that is, the organization of personnel required to operate the system and types of personnel needed, the nature and extent of the data base to be exploited, the indexing philosophy and technique, the files which will be identified to the user, system flows and data handling procedures, and the man- machine interactions projected for the computer-centered system. Of course, not all design problems have been resolved. Moreover, even if they had, it would not be possible to describe within the confines of one volume all of the transactions which must be performed in a system as large and complex as this. However, illustra- tions of representative tasks are included and some concepts of system data flows are presented to dEmonstrate the impact of hardware and programs upon personnel actions. The recommendations interspersed in this volume result from Phase II of the design study, including a preliminary evaluation of the CHIVE Indexing Experiment INTRODUCTION General 5.1.1. - 1 - Approved For Release 2000/05/30: CIA-MBE143952A000100050001-7 Approved For Release 2000/05/MMECRDP78-03952A000100050001-7 conducted between November 1964 and January 1965, and are supported by material in some of the appendices to this volume as well as in earlier CHIVE documentation. The other appendices present further details on the indexing language and technique and the files to be inherited from the existing central reference reposi- tories. All are recommended reading for recipients of this report who desire more detail on specific aspects of the system, as well as further background on the alternative configurations considered and the steps taken to arrive at the recommended system. A supple- mentary appendix to this volume will be issued later describing the CHIVE Indexing Experiment in greater detail, and reporting the final conclusions derived therefrom. 5.1.2. SYSTEM OVERVIEW A simplified graphic view of the CHIVE system can be obtained by referring to Figure 5-1. In this diagram the flow paths within the system are separated for descriptive purposes into three major functional cate- gories--document input processing (flow path 2), document retrieval processing (flow path 1), and information file building and maintenance (flow path 3). The following INTRODUCTION System Overview 5.1.2. - 2 - Approved For Release 2000/0?e@gAIA-RDP78-03952A000100050001-7 25X1B Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-VER'V03952A000100050001-7 paragraphs will summarize briefly the major elements of the system, leaving the more detailed explanations to subsequent chapters in the volume. In general, the philosophy of the CHIVE system is to combine the required intellectual talents of trained intelligence information analysts with the processing and storage capabilities of the computer. The source documents to be input to the system, the necessary human functions to be performed relative to these documents (i.e., reading, selecting, indexing, querying and reporting) and the outputs to be derived from the system are quite similar to those which characterize one or more elements of the existing central reference operation. Only if the proposed system is compared to an individual register s,lbsystem within the current OCR complex does the contr,st appear, and then only with respect to certain features of the existing subsystem. In terms of file organization, the system follows the approach used in SR/OCR and DD/OCR in maintaining a separation between an index and the document holdings to which it refers. This necessarily has implications in terms of input time which may compare unfavorably with some of the current systems which are oriented toward multiple-filed documents (e.g., BR/OCR), but it also INTRODUCTION System Overview 5.1.2. - 4 - Approved For Release 2000/05/30 : CIA-RgFrTgg3952A000100050001-7 Approved For Release 2000/05/30 : Cli&W8-03952A000100050001-7 offers certain advantages in such areas as procedural standardization, index integration, number and variety of access points to the files, space requirements, etc. The information is received primarily 'n the form of documents; however, index records to maps, photographs, and films will also bu included in the system, as will certain machine-language data prepared on contract (but under CHIVE control) by external organizations (e.g., the Library of Congress). Following preparation of the index record (a function normally performed by humans except where only a limited retrieval capability seems required), the index will be converted to machine storage with the aid of an optical character reader and placed in a random access device, ultimately the IBM/System 360 Data Cell Drive. The information storage capacity of one Data Cell Drive will allow us to accommodate the content of an estimated 600,000 index records (the actual storage capacity is 400 million characters of information), and there is no practical limit on the number of modules that could be provided. The same device would be used te hold what might be called the directory to the index records themselves, i.e., a list of the terms which appear in INTRODUCTION System Overview 5.1.2. - 5 - Approved For Release 2000/05/30 : CIA-FEEK*63952A000100050001-7 Approved For Release 2000/05WINA-RDP78-03952A000100050001-7 the index records and, for each term, the record and phrase number(s) containing said term. This would obviate the need to examine every index record in the file to see if it contains the term (or terms) sought. Index entries can be retrieved from the index store at the rate of about two per second depending on the number of terms involved in the search formula. CHIVE's recommendation is that most textual docu- ments should be converted to microfilm and stored either in the form of 35 mm. aperture cards (containing up to 8 image,_ per aperture) or packed microfiche (sheet microfilm records containing up to 60 letter-size pages on each microfiche). Documents in excess of a certain page limit and those of poor image quality should be kept in hard copy. Maps, films, and photos will continue to be stored in the conventional manner in the physical reposi- tories in which they are now located. Whether the 35 mm. aperture card or microfiche storage system is chosen, the document images should be filed in motorized card files, but should be retrieved and refiled manually. Assuming 10 million documents were to be stored on site, the estimated floor space required for a packed microfiche system would be an area approxima- tely 30' x 60'; for the 35 mm. system, 40' x 70'. Output INTRODUCTION System Overview 5.1.2. - 6 - Approved For Release 2000/05/SECREtr-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: CaKBAT8-03952A000100050001-7 from either the hard copy document or microimage files would consist of paper copies. The integrity of the document collection will be maintained such that none of the master microimages, or original documents if filed only in hard copy, will leave the file except for photoduplication or hard copy printing. INTRODUCTION System Overview 5.1.2. - 7 - Approved For Release 2000/05/30 : CIA3iRU03952A000100050001-7 Approved For Release 2000/05/30: CIAKA305-03952A000100050001-7 Chapter 5.2. SYSTEM ORGANIZATION 5.2.1. BACKGROUND The organizational configuration recommended by CHIVE is the product of much thought and discussion Imo extending back into the Phase I study and reflects a variety of views expressed by persons both internal to um; the CHIVE design team as well as to OCR. One of the two most vexing and, at the same time, one of the most important of the CHIVE design problems, it is not anticipated that the organizational plan which has evolved will be attractive to all. Nevertheless, it Iwo appears to offer the best hope of achieving the isme desired system objectives, consistent with the human factor requirements imposed by the environment within Iwo which the system must operate. The search for a revisioli of the existing central mow reference organizational structure was largely influenced by the findings of the DD/I survey and the set of system 10.40 'vomit requirements derived therefrom. These findings and inferred goals have been described in the CHIVE Phase I Report, in CHIVE/R-1-63, and (in more abbreviated fashion) in Volume IV, Chapter 2, of this report. The organization study SYSTEM ORGANIZATION Background - 9 - 5.2.1. Approved For Release 2000/05/30 : CIAMIRIF03952A000100050001-7 Approved For Release 2000/05/36EUETRDP78-03952A000100050001-7 itself may be said to have consisted of three phases: a. An analysis of the personnel or management requirements imposed on the system by the overall system objectives. b. A study of various alternative organizational configurations which might be adopted, ranging from a completely decentralized activity to various kinds of centralized operations, including alternative configurations at different hierarchic levels. c. An evaluation of one organizational concept by the process of subjecting the concept to a practical experiment which simulated to some extent the problems to be encountered in a live environment. Phases a. and b. are described in some depth in Appendix 5.A. to this volume and are briefly reviewed below. Phase c., which resulted in some revision of the organizational concept, is discussed in Appendix 5.B. and only its conclusions are reflected here. In considering the managerial problem of how best to organize the input and retrieval functions to be performed, as well as the personnel to carry out these functions, a number of organizational requirements were set forth. SYSTEM ORGANIZATION Background 5.2.1. - 10 - Approved For Release 2000/05/gtcRt-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAW8-03952A000100050001-7 These requirements, or objectives, may be summarized for the purposes of this review as follows: a. Specialization with minimum processing duplication b. Minimum customer contact points c. All-source service from any point d. Close communication between input and query handlers e. Close communication between system operators and users f. Document control as the first priority g. Operator job satisfaction h. Personnel flexibility The next step was to pass various organizational configurations against these objectives to determine which would appear to offer the best hope of accommodating the defined goals. Because of the size of the contemplated activity in terms of the number of personnel needed to operate the system, this required that alternative con- figurations be considered not only at the first organiza- tional level, but at least at one additional level below that. For the first cut the following four different organizational concepts were considered: a. Retention of the existing OCR configuration SYSTEM ORGANIZATION Background 5.2.1. - 11 - Approved For Release 2000/05/30 : CIAMtEX03952A000100050001-7 Approved For Release 2000/05/MCCRFORDP78-03952A000100050001-7 b. Development of a single, all-source document retrieval system, with a separate biographic information facility c. Dispersal of some or all of the information storage and retrieval activity among the research and production components d. Continuation of the central system, but on an all-source, geographically-organized basis Where the additional subdivision of personnel would be required because of the size of a particular component these additional means of grouping the analysts assigned thereto were studied: a. Organization by document source (Collateral, Comint, etc.) b. Organization by function (input, retrieval, information file maintenance, etc.) c. Organization by class of data to be stored and retrieved (biographic, installation, subject/commodity, etc.) d. Organization by topic (political, scientific, economic, military, etc.) The study very quickly made clear that none of the alternatives considered resolved all problems that could be anticipated. However, the combination of the geographic SYSTEM ORGANIZATION Background 5.2.1. - 12 - Approved For Release 2000/05a0GrOk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClagF8-03952A0001000500017 approach at the first level, and topical specialization (where required) at the second, seemed to come closest to meeting the organizational objectives outlined above. It remained to be seen, however, whether an information analyst could perform all-topic indexing of all-source documents satisfactorily, and what effect it mi-it have on his morale and attitude if he had to operatL in this kind of environment. The CHIVE Indexing Experiment afforded the opportunity to test the configuration proposed and, as detailed in Appendix 5.B., identified a number of problem areas which suggested that some additional organizational and procedural alternatives might well be considered. Of principal interest from the organizational point of view was the recommendation that the geographic concept be retained but that the coding process per, se be separated from the function of selecting documents and identifying the subjects or objects to be indexed. Acceptance of this approach meant some compromise of the single-point indexing concept but offered the advantage of increased job satisfaction on the part of the more highly qualified analyst, helped reduce the selection problem, and suggested the possibility of acquiring more personnel for less money to perform the more routine input functions. Since it still permitted achieving all the other organizational SYSTEM. ORGANIZATION Background - 13 - 5 2 . 1. Approved For Release 2000/05/30 : CIA-Frgant3952A000100050001-7 Approved For Release 2000/05/3V:CdiATRDP78-03952A000100050001-7 objectives, it was selected as the alternative best satisfying the system requirements and is the approach recommended here. 5.2.2. PROPOSED ORGANIZATIONAL CONCEPT The responsibility for implementing a spec_fic organizational configuration must be left to those who will direct the operation since there are a variety of factors to be considered which are beyond the purview of the system designer. To assist those, however, who will be charged with this activity, it might be useful to summarize the principal CHIVE organizational recom- mendations in the context of the major functions to be performed within the system, and to give some feel for the interrelationships between these functions since these could have implications for management in terms of communication interface, assignment of physical space, and so forth. This first _,00k will be an abbleviated one since much of the same ground is covered (if from a slightly different point of view) in more detail in other sections of this volume. A set of position descriptions outlining the duties and responsi- bilities of the various types of personnel within the system concludes the chapter. SYSTEM ORGANIZATION Proposed Organizational Concept 5.2.2. - 14 - Approved For Release 2000/05ntR04-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: gMtp1978-03952A000100050001-7 Input Control and Customer Service The CHIVE system would be built largely around Information Analysts organized (at the first level) into some four or five geographic components. It is our view that it is difficult to identify any better way of organizing the input and retrieval activity than by grouping the primary individuals involved by geographic area. As stated in earlier documentation, this approach loses the advantage of source specialization in processing and poses the problem of geographic overlap in document analysis and query coordination. At the same time, it contributes to standardization of vocabularies and procedures important in an all-source environment, anC is in focus with customer inquiries which normally relate to a particular geographic region of the world. Thus, on balance, while it does not overcome all operational problems that can be envisaged, of all the alternatives considered it seems to come nearest to meetin7 the system objectives. Without specific restrictive criteria (which, thus far, seem impossible to obtain) with respect to the content of the documents to he processed, the experienced Information Analyst, operating in close communication with his customers, appears to offer the best hone of resolving the data selection problem. The Information Analyst would, SYSTEM ORGANIZATION Proposed Organizational Concept - 15 - Approved For Release 2000/05/30 : CIPM1M-03952A000100050001-7 Approved For Release 2000?IftitTCIA-RDP78-03952A0001 050001-7 therefore, be responsible for determining not only what documents entered the system files but what data within these documents was captured for retrieval purposes. The Information Analyst operating out of a geogra- phic component would also be solely responsible for the selection and processing of data input to information files required by customers, and would handle all queries levied on the system. By virtue of the fact that he was personally involved in the input process, he would not only be familiar with the current reporting but would know what material had been stored for retrospective searching and how to get at it. Whether the Information Analyst should also specialize by topic within area or by some class of intelligence data (e.g., biographic, installation, etc.) remains a moot point. CHIVE continues to favor the former in the belief that it would lessen the number of times a document would have to be handled, but additional testing of both concepts is desirable. Index Preparation The function of physically preparing the index records to documents, including both the header (bibliographic) as well as the content data descriptions, would be assigned to special personneZ known as Header Indexers and Content -1 Approved For Release 200gge SYSTEM ORGANIZATION Proposed Organizational Concept 5 . 2 . 2 . 2 . : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: GIWREVIP78-03952A000100050001-7 Indexers, operating in close communication with the analytical components. Content Indexers serving one geographic desk, e.g., the Far East, should probably be located together as a unit attached to said component. The Content Indexers, like the Information Analysts, would be subdivided by geographic area and each would normally process the output of his counterpart analyst or analysts. Content Indexers would each have a set of the dictio- naries and other vocabulary control tools pertinent to hi: area of responsibility. In addition, a master set of other area dictionaries would be located within each content indexing group for reference purposes. Content Indexers would translate the items of data tagged by Information Analysts into the codes and other descriptors dictated by the vocabulary of the system. To increase their sense of participation in the more intellectual aspects of the input process (and, thereby, reduce turnover), they might be given full responsibility for general subject indexing as distinct from named- object control. Header Indexers would perform a function similar to content indexing, but on the bibliographic elements of a document. One group of Header Indexers would operate in a SYSTEM ORGANIZATION Proposed Organizational Concept - 17 - Approved For Release 2000/05/30 : CU:SWEV-03952A000100050001-7 Approved For Release 2000/?Et3REICIA-RDP78-03952A000100050001-7 centralized mode, serving all geographic components by header indexing, immediately upon receipt, those documents for which CHIVE has a repository responsibility. Other Header Indexers would be assigned to each geographic organization to capture the necessary bibliographic data pertaining to non-repository-type documents which had been reviewed by Information Analysts and selected for retention by the system. Dissemination The dissemination function, apart from any necessary re-routing of documents within a CHIVE geographic component, is external to the system per se. However, it might be advantageous to co-locate dissemination personnel with the centralized header indexing group between document receipt and file repository-type documents. This operation equipment Data Transcription function required is to be refers if, as to shorten the time availability for to the rather formalized typing planned, used to convert optical recognition index and other records into machine-recognizable form. Header Indexers can type their inputs in a form suitable for processing by a page reader. However, a central pool of typists will also be SYSTEM ORGANIZATION Proposed Organizational Concept - 18 - Approved For Release 200W4,r. CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : qttlikle78-03952A000100050001-7 needed, operating at the system level, to convert the majority of transcript sheets received from Content Indexers as well as search rr,quests from Information Analysts into the graphic quality required. This central pool can be supplemented by typists assigned to the various area desks who, in addition to typing finished reports, memoranda, etc., would also transcribe many of the file maintenance and query transactions for input to the page reader. Image Processing and Document File Maintenance Image processing is that activity conducted by the so-called "Document Delivery System," i.e., the micro- filming and associated operations required to convert incoming documents to microimage form, as well as the reproduction of items retrieved from the document store for delivery to customers. This is a relatively discrete function although, if an aperture card storage system is employed, it requires some support from the machine side of the house. Otherwise, its principal interface is with the document store itself to which materials are passed after microfilming and from which it receives, in turn, items to be reproduced. During the evolutionary development of the CHIVL system both the new and old system operators will require SYSTEM ORGANIZATION Proposed Organizational Concept - 19 - Approved For Release 2000/05/30 : Cl4gait7r8-03952A000100050001-7 Approved For Release 200ogg1ft'ETCIA-RDP78-03952A000100050001-7 access to many of the same document collections. If the logistical problems are not too severe, it would seem advisable to co-locate all master document files in one general physical area to lessen the communication Problem as well as render file maintenance operations more efficient. This might increase the distance which now obtains between an existing central document collection and a set of users, but over time the majority of users would probably benefit from the establishment of one "Document Center." Similarly, because of the close relationship between the document files themselves and the image processing function, it is recommended that the latter be connected both physically and organizationally to the former.* Machine Functions The principal machine-related activities and hardware include: a. EAM personnel and equipment needed to input data to files not yet absorbed into the new system and to retrieve data therefrom. Assuming no conver- sion to an EDP storage medium, the latter, in particular, will necessitate the retention of an EAM facility for as long as the inherited files have value. b. EDP hardware needed to operate the new system, including associated I/O devices (e.g., the page reader), and computer operator personnel. * Problems involved in co-locating files are discussed in Volume III. SYSTEM ORGANIZATION Proposed Organizational Concept - 20 -5.2 2,..6 Approved For Release 2001-: CIA-Ku1378-03952A000100050001-7 Approved For Release 2000/05/30 : Cl1aW3-03952A000100050001-7 c. System analysts/programmers (referred to in this report as EDP File Analysts) who will develop and refine the machine operations to be performed, define new files to the system, etc. Logically, all of these personnel and operations should be centralized in one organizational component whether located within the central reference complex or external to it. 5.2.3. POSITION DESCRIPTIONS The personnel involved in making up the CHIVE operator complex will include the following: Information Analyst, Content Indexer, Header Indexer, Dictionary Editor, Data Transcriber, Information Control Clerk, Document File Clerk, Reproduction Equipment Opeator, EAM Operator, Computer Operator, and EDP File Analyst. Information Analyst The Information Analyst will be the principal inter- mediary between the customer and the system. He will be responsible for selecting what goes into the files and will screen all output before it is delivered to a requester. Senior Information Analysts will serve in various supervisory capacities frm the sub-Section to the Branch or Division level, directing, coordinating, and reviewing the work performed by their subordinates. SYSTEM ORGANIZATION Position Descriptions - 21 - Approved For Release 2000/05/30 : CIAMMT-03952A000100050001-7 Approved For Release 2000/05/?ifcgURDP78-03952A000100050001-7 All Information Analysts will hold professional positions, and will specialize in a particular geographic area and (where required by-reason of work volume) by topic within area. Every Information Analyst will be trained in applying the indexing vocabulary to documents by actual involve- ment in the coding process. He will also be thoroughly familiar with all the CHIVE-built files available within the system as well as the query language used to interro- gate or modify said files. In addition, he will know what inherited files were acquired from the existing system and their general content, although not necessarily the vocabulary used in these files. The duties of an Information Analyst will include: a. Receiving and reviewing the content of documents, cables, graphics and other incoming data for information worthy of retention by the central reference system. b. Selectively marking the elements of information to be extracted from the documents for represen- tation in the system's index files and distribu- ting the marked documents to Content and/or Header Indexers. c. Exploiting the content of document index records for the purpose of building formatted information files pertaining to a specific subject or class of subjects. d. Preparing file maintenance transcript sheets as the means of adding da,-a to, or changing data within, said information files. SYSTEM ORGANIZATION Position Descriptions - 22 - Approved For Release 2000/05g@c041-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clktidifir3-03952A000100050001-7 e. Receiving requests from customers and preparing the necessary search prescription after consulting the relevant vocabulary control files and other Information Analysts most familiar with the vocabularies of certain inherited files. f. Requesting copies of documents as well as dossiers and other master records from the central docu- ment repository. g. Reviewing, analyzing, and synthesizing data recovered as a result of the search process and preparing responses in raw or finished form for delivery to the customer. h. Advising customers about files or persons external to CHIVE that might be worthwhile consulting, and personally contacting same if required. 1. Recording necessary management data relative to requests received, responses furnished, and other system processes. Content Indexer The Content Indexer will he a semi-professional possessing at least a high school education. His duties will include: a. Extracting the elements of information in a docu- ment identified for him by the Information Analyst. b. Consulting the relevant dictionaries and other vocabulary control files for the purpose of seJecting the appropriate controlled terms to express these items of data. c. Arranging the data into a form for machine entry using pro-forma content data transcript sheets. d. Consulting with the appropriate Information Analysts and Dictionary Editors with regard to the applica- tion of the index language and possible revisions to the system vocabularies. SYSTEM ORGANIZATION Position Descriptions - 23 - Approved For Release 2000/05/30 : CIA-RgEn?013952A000100050001-7 Approved For Release 2000/05/AE.CMIRDP78-03952A000100050001-7 e. Initiation of additions or changes to the vocabulary control files through preparation of file maintenance transcript sheets. f. Reviewing printout of changes and additions to the files including incorrect entries. Header Indexer The Header Indexer will occupy a clerical position and must be a qualified typist. The duties of the Header Indexer will include: a. Extracting the standard header (bibliographic) data appropriate to the category of document involved, and expressing this data (where re- quired) in the codes used by the system. b. Typing the data in the prescribed manner for machine entry using the correct header data transcript sheet. c. Consulting with the Dict 'nary Editor for Header Data with regardto':he use of the header data codes and format, al. recommending changes when required. Dictionary Editor The Dictionary Editor will be an Information Analyst with primary responsibility for control of one of the system's vocabulary files. Some Dictionary Editors will have system-wide control over the application of terms in their respective subject areas. Others (e.g., an Organiza- tion Dictionary Editor) may govern the use of terms only within a given country or other geographic area. Tho SYSTEM ORGANIZATION Position Descriptions - 24 - Approved For Release 2000/05SEERelk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CggBg8-03952A000100050001-7 duties of a Dictionary Editor will include: a. General review of the content and format of sample transcript sheets emanating from Indexers assigned to the area unit of which he is a part. b. Providing advice and counsel to Indexers on the use of the specific dictionary for which he is responsible. c. Reviewing all new entries to the dictionary for the purpose of determining whether each was a legitimate entry and whether format and content met established procedures. d. Personally initiating changes to a dictionary where required. e. Reviewing printouts of changes and additio,ls and insuring that all revisions to the dictionary are published and disseminated. f. Consulting with other Information Analysts and custr?rs regarding current requirements and possible improvements to the system's vocabulary control files. g. Advising Information Analysts preparing request statements on the terms to be used in the query prescription. Data Transcriber The Data Transcriber includes any person exclusively assigned to operate a key-driven device from copy provided via another system operation. The duties of a Data Transcriber will be as follows: a. Receive format instructions from Information Analyst, Content Indexer, or other individual for typing, tape perforation, or card punching. SYSTEM ORGANIZATION Position Descriptions - 25 - Approved For Release 2000/05/30: CIA-BDPRE103952A000100050001-7 Approved For Release 2000/05/36ECKTRDP78-03952A000100050001-7 b. Prepare typed copy, punched paper tape, or cards for optical character recognition or other form of computer entry. c. Check transcribed cop l for accuracy and correct if necessary. d. Operate typewriter, Flexowriter-like device, 026 Key Punch, and 056 Verifier. Information Control Clerk Information Control Clerks will be assigned to most operational components of the system. Their general duties will include: a. Receiving material such as hard copy documents, machine listings, document request forms, paper and magnetic tapes, card decKs, etc. b. Accounting for material received and maintaining necessary special-purpose logs of requests and other actions. c. Intra-office routing and delivery of materials to staff personnel and mailing of system products to customers. d. Assisting Information Analysts in the routine maintenance of manual files including the insertion of handwritten entries to machine listings and other hard copy records. Document File Clerk The duties of the Document File Clerk will include: a. Filing newly-processed documents or refiling old materials in+-..o one or more of the following types of document files: personality or installation dossiers, card files, open-shelf document files, and 16 mm. or 35 mm. aperture card collections. SYSTEM ORGANIZATION Position Descriptions - 26 - Approved For Release 2000/05EXICKMk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClOCEIM-03952A000100050001-7 b. Receiving requests for documents or other records to be retrieved and recovering same from the appropriate files on either a routine or priority basis. c. Maintaining dossier and other special-purpose logs pertaining to transactions affecting the document files. d. Recording action taken on document request forms, forwarding requests for unrecovered documents to other file repositories for searching, and transmittal of master records to image processing for photographic reproduction. Reproduction Equipment Operator The duties of the Reproduction Equipment Operator will include: a. Receiving inroming documents and determining which are photographable and which must be stored in hard copy. b. Operating the appropriate microfilming equipment required to reduce the documents to a micro- storage medium and reviewing the quality of the photographic record. c. Receiving documents retrieved from the master files and reproducing same on a variety of image- processing equipment. d. Servicing and supplying reproducing equipment. e. Supplying copies of documents to Information Control Clerks for delivery to internal or external requesters. EAM Operator EAM Operators will be required to process certain card files inherited from the existing system as well as - 27 - Approved For Release 2000/05/30 : CIA-SIE1RT03952A000100050001-7 SYSTEM ORGANIZATION Position Descriptions Approved For Release 2000/05/1C8aRDP78-03952A000100050001-7 select new files. In general, the duties of an EAM Operator will include: a. Operating electrical accounting machines including interpreter, reproducer, tabulator, sorter, and printer units. b. Performing routine machine operations in accordance with conditions outlined by EDP File Analysts, Information Analysts, and Indexers. c. Wiring panels in accordance with directions. Computer Operator In general, the duties of the Computer Operator will include: a. Maintaining a schedule and operating log of the components of the computer complex. - b. Loading and unloading Tape Units. c. Loading and operating stored programs. d. Tracing and correcting program errors. e. Correcting failures in card, paper tape, or optical character reading equipment. f. Wiring and/or selecting control panels for use in card reading machines. EDP File Analyst The duties of the EDP File Analyst will include: a. Determining from Information Analysts requirements for new system files and devel(N-ing the record structures, file formats, and output products needed to establish and maintain such files. SYSTEM ORGANIZATION Position Descriptions - 28 - Approved For Release 2000/05MOCRHA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: CgkBP8-03952A000100050001-7 b. Preparing general and special-purpose programs either for the purpose of converting extant machine-language files or to provide new data processing capabilities. c. Testing newly designed programs utilizing the computer and necessary input/output units. d. Conducting studies of system data flow for the development and refinement of programs. e. Determining utilization requirements for input/ output devices including displays, and designing programs to permit exploitation of input/output capabilities. f. Designing quantitive techniques and statistical devices for special program applications. g. Preparing procedures descriptions including coding formats and flow charts for operator task guidance. SYSTEM ORGANIZATION Position Descriptions - 29 - Approved For Release 2000/05/30 : CIA-&PaCI:RiFt3103952A000100050001-7 Approved For Release 2000/05/30 : CIRW8-03952A000100050001-7 Chapter 5.3. DATA BASE 5.3.1, THE SELECTION PROBLEM The selection problem has been with OCR since its inception. No coordinated study of selection as an entire OCR problem has ever been made. Individual registers have established selection criteria, some more formalized than others. An attempt to summarize these criteria for compatibility, or to establish common criteria to be used by all registers was not deemed necessary heretofore. Since each register has been more or less independent, ipso facto, its criteria have for the most part been unrelated to those of any other register. This condition has led to non-uniform levels of coverage and, in some cases, duplicative processing of the same subject matter. Regardless of CHIVE, if OCR adopts a geographical organization posture, uniform criteria for document series and depth of subject indexing become mandatory within geographic component. 5.3.2. BASIC SELECTION CRITERIA Selection criteria will depend on several factors: DATA BASE The Selection Problem 5.3.2. - 31 - Approved For Release 2000/05/30 : CIASRORES-03952A000100050001-7 Approved For Release 2000/05/3tHWTRDP78-03952A000100050001-7 (a) the documents used and information needed by the analytic offices; (b) the all-source concept and organizational configuration thereof. These two factors have to be balanced against the manpower and resultant capability available for the operation. There seems to be a consensus of opinion that several levels of indexing should be applied to the various categories of documents: - Entire series to be indexed in depth. - Entire series to be rejected for depth indexing, but to receive header or bibliographic control. - Entire series to be rejected completely. - Specific documents within a series to be indexed in depth. Selection of an indexing level for a particular document category is contingent upon customer reaction and acceptance, which determination requires discussion of interest in series not covered now and re-examination of series presently covered. Customer participation in determining selection criteria can mean the success or failure of the system in terms of usage. Once the level of indexing is agreed upon, document priorities will need to be established for implementing the CHIVE system since all categories cannot be implemented within the initial System simultaneously. DATA BASE Basic Selection Criteria 5.3.2. - 32 - Approved For Release 2000/05/3SteTRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAWf?T-03952A000100050001-7 5.3.3. SOURCES TO BE EXPLOITED The following major document series are planned for CHIVE control: Raw Intelligence Reports (Collateral) State Airgrams Military Attache Reports CIA Reports--00,CS, etc. Military Command Reports Selected Other Governmental--AID, USIA, etc. 25X6 International Organizations--NATO, etc. 25X6 25X1A Cables (Collateral) CIA-TDCS Non-CIA Finished Intelligence U.S. Open Publications and Translations FDD JPRS DATA BASE Sources to be Exploited 5.3.3. - 33 - Approved For Release 2000/05/30 : CIAMU1T-03952A000100050001-7 25X1A Approved For Release 2000/05/35Ctra-RDP78-03952A000100050001-7 COMINT Messages Reports Photo Interpretation Reports (T/KH) Maps, Films, and Ground Photos Miscellaneous Select Contractual-IIIIII etc. State Biographic Cards Unclassified Selected Periodicals, e.g., for China: Peking Review, Survey of China Mainland Press, etc. Criteria for the depth of coverage will be developed by the CHIVE information analyst working in concert with the research offices. He will direct the indexer as to coverage and depth, i.e., which personalities, which organizations, and/or which subjects should be indexed. The CHIVE Indexing Experiment has shown the need for title coverage of most documents regardless of the level of indexing unless the document or series is completely rejected. This includes title preparation for those types to be selectively indexed which have no titles, e.g., non-CIA cables. 5.3.4. LEVEL OF COVERAGE Raw Intelligence Reports Since the information content of IR's supports a DATA BASE Level of Coverage - 34 - 5. 3. 4 . 1 . Approved For Release 2000/05/gtckht-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CAR8-03952A000100050001-7 variety of intelligence interests, all IR's will be considered for some level of indexing. Duplicative information which frequently occurs between sources will be eliminated wherever possible, based on the information analyst's recall capability supplemented by data contained in dictionaries and identifier lists. Cables CIA cables (TDCS's) have always been handled as Information Reports and should be continued as such. As for non-CIA cables, the very fragmentary and highly perishable nature of these cables and the frequent duplication by follow-up reporting would indicate that only a small percentage of these cables are worthy of storage for retrospective search purposes. Only those cables containing positive foreign intelligence infor- mation will be indexed for header control as well as content. All others will be rejected completely--the Cable Secretariat continuing to retain repository responsibility for same. Finished Intelligence The Intelligence Publications Index (IPI) and Special Register's Job 3 are published by OCR to provide current awareness, and, to a lesser extent, retrospective subject - 35 - DATA BASE Level of Coverage Approved For Release 2000/05/30 : CIA4KIKET-03952A000100050001-7 Approved For Release 2000/05/aCM-RDP78-03952A000100050001-7 and area searching for finished intelligence. In addition, some finished intelligence is incorporated into the files of BR and FIB. Since the Agency has a repository responsibility for finished intelligence, bibliographic control over such material will be established in the CHIVE system for document retrieval purposes. Furthermore, some named-object indexing of finished intelligence documents will be performed similar to the control currently maintained by BR and FIB. During the evolution of the CHIVE system, the biblio- graphic and named-object control achieved by the 25X1A 25X1A and subsequent branches will in part duplicate the contents of the IPI and Job 3. This dupli- cation seems unavoidable since the issuance of these publications should continue in order to serve the current awareness needs of analysts, and it does not seem feasible during the implementation period to split the preparation of the publications between CHIVE and the existing activities. When implementation has been completed, however, it will be desirable to investigate the feasibility of producing a permuted title index to finished intelligence from the machine-stored data base as a replacement for both the IPI and Job 3, DATA BASE Level of Coverage - 36 - Approved For Release 2000/05/WW-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CgRE*1-6-03952A000100050001-7 STATSPEC Eapila Translations FDD and JPRS translations can be considered as one type. Heretofore in CIA no subject indexing scheme has incorporated both of these open literature sources. The manpower needed to cope with this large volume (see Table 5-1) is of significant concern. However, broad customer interest dictates in-depth subject and named-object control. DATA BASE Level of Coverage - 37 - Approved For Release 2000/05/30 : CIASHEMT03952A000100050001-7 25X1A I I I pprov d For Release 2000/05/30: CIA-RDP78-03952A000100050001-7 Table 5-1 CHIVE INPUTS Series Approximate Annual Volume Repository Responsibility Bibliographic Content (C) miity B) and/or Control Remainder ,moiliel?IN????????? Ram Intelligence (including Tres's) Cables Finished Intelligence Translations FDD JPRS COMUNT Photo Interpretation Reports Maps Films and Ground Photos 25X6 253,500 192,000 7,803 78,000 items 44,300 items 109,050 7,900 6,000 87,000 903,035 7,200 36 625 items 1,560 items 3,400 items 104 items 12,925 X X IND ?00. ??? 9111 B and. C Not processed Approved For Release 2000/05/30 : CIA-RDP28-0$3952A000100050001-7 B only B and C B and C B and. C B and C B and C B and C Not processed B and C B and C B and C B and C B and C B B and C B and C B cnly B B only B only B only B only B and C Not processed Not processed Not processed Not processed Not processed Approved For Release 2000/05/30: ClAfiefE18-03952A000100050001-7, COMINT All hard-copy SI material with the possible exception of military order-of-battle data will be considered for indexing in depth. Teletypes are excluded in their entirety pending the design of an automatic processing capability which will take advantage of the fact that the data is available in machine-language. An information analyst knowledgeable in both collateral and SI may be able to spot duplicative information if such exists. One large series of SI material which, in the present OCR/SR system, is given cursory control, will be studied to determine whether it should receive any title or subject control whatsoever. A few items in this series were processed during the experiment, but the titles were so general as to be practically worthless for retrieval. Photo Interpretation Reports The unquestioned value of this category requires that all published reports receive in-depth content and header indexing. Maps, Films, and Photos These categories of receipts will be excluded from CHIVE processing control because of the specialized knowledge needed for their analysis and input, the DATA BASE Level of Coverage - 39 Approved For Release 2000/05/30 : CIA-EMET03952A000100050001-7 25X1A Approved For Release 2000/05/35ECREIRDP78-03952A000100050001-7 difficulty of separating the indexing function from the acquisition activity, etc. It has been agreed, however, to have these materials indexed by GR and the Map Library according to the CHIVE indexing scheme and the index records will be incorporated into the CHIVE data base. Other A number of miscellaneous classes of documents will also be processed by CHIVE. Most (e.g., press reviews and surveys) will receive named-object indexing primarily. The large volume of State biographic cards needs rigid selection and weeding not only to determine names of interest but also to eliminate repetitive information. Like the there is little high- grade ore contained therein in relation to volume. DATA BASE Level of Coverage - 40 - Approved For Release 2000/05@ket-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClaSRI-03952A000100050001-7 Chapter 5.4. CHIVE INDEXING TECHNIQUE 5.4.1. INTRODUCTION The most critical design element of the proposed system is the indexing system to be applied to input documents; the performance of the system is no better than the data which it is supplied. The transformation of textual material to the system language is an expen- sive process - one which has been given more attention than any other in the Phase II effort. 5.4.2. CONCEPTS Document/Information Retrieval The system will provide combined information retrieval and document retrieval capability. Documents themselves will be at the heart of the system, with their index records providing access to them through content control. The index records will also be the base from which information files will be built. That is, in the process of indexing documents, facts about CHIVE INDEXING TECHNIQUE Concepts - 41 - Approved For Release 2000/05/30 : CIASIECRU-03952A000100050001-7 Approved For Release 2000/05/56Ctra-RDP78-03952A000100050001-7 named things of intelligence interest will be extracted and stored. The approach will be to extract information about specific named objects, keep this information in the context of the document for document retrieval, and manipulate this information out of context for informa- tion retrieval. It is not proposed to create non- redundant summary records from index records at input time either through human or machine collation. Summary records will be formed and maintained on select high- interest personalities, installations, and other finite subjects, but the creation of these records will be an analytic activity requiring the synthesis of index records and documentary information. In addition to the index records, the indexer working aids will themselves be a source of answers to questions. For example, the Organization Identifier List will contain names of organizations, their locations, type of activity, etc. Manual Indexing An investigation of the state-of-the-art of automatic indexing reveals that it is still largely experimental and CHIVE INDEXING TECHNIQUE Concepts - 42 - Approved For Release 2000/0512CW-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClkaaV8-03952A000100050001-7 is not sufficiently precise to meet most of the Agency's retrieval requirements. Automatic indexing techniques usually involve word frequency counts, assigning weights to high-frequency words, and storing these words as index terms. Other techniques include syntactic analysis, sometimes in conjunction with the above statistical process. It is obvious that these techniques could not be applied to an intelligence storage and retrieval system requiring a high relevance/recall rate, since much intelligence information is inferential and inter- pretive and requires analysis for high-quality indexing. Human indexing, therefore, with its recognized faults is still superior to automatic techniques and is the only feasible system for CHIVE. However, some documents will require only title indexing and in these cases automatic title-indexing techniques can be applied. The most notable title-indexing system is the Key-Word-In-Context (KWIC) method. In this system, the key words in titles are permuted so that each word appears in its alphabetic file position along with the other significant surrounding words from the title. The permuted titles can be machine stored for searching on demand, or printed listings can be generated for manual perusal. CHIVE INDEXING TECHNIQUE Concepts - 43 - Approved For Release 2000/05/30 : CIA3KFRU03952A000100050001-7 Approved For Release 2000/05/39EaRAIRDP78-03952A000100050001-7 Depth--Subjects vs. Named-Oblects It need hardly be argued that intelligence interests are catholic in nature, and that if an information storage and retrieval system arbitrarily decides to limit its coverage to personalities, installations, or conceptual- type subjects, it automatically limits its ability to satisfy its total customer population. Intelligence analysts have found that "named-objects"-- e.g. installations, personalities, organizations--most often provide the clues to resolving research problems. OCR request experience is an accurate reflection of this interest. We recommend, therefore, that these subjects receive the greatest emphasis; and, in view of OCR experience relating to the kinds of things users are interested in concerning named-objects, we recommend that an increased number of attributes of named-objects be brought under control. The latter are the elements of information which identify a named object, e.g., a person's address, organizational affiliation, etc. In-depth indexing of named-object attributes does not necessarily have to mean an equivalent increase in the volume of data indexed or in CHIVE INDEXING TECHNIQUE Concepts - 44 - Approved For Release 2000/05gfefe-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIARSI3g-03952A000100050001-7 indexing time since common attributes, such as addresses, types of organizations, and products of an installation, will be stored in indexer identifier lists (see Section below), and it will not be necessary to re-index this data when it is reported repetitively in documents. We recommend that subject indexing, that is, the kind of indexing performed by the Intellofax system and the Subject/Commodity Section of the Special Register be continued at least to the present level, but on a broader data base to include important document series (e.g., foreign translations) which are excepted today. Index Language; Linkage The CHIVE indexing language consists of controlled entries taken from identifier lists and code schedules, as well as words and phrases extracted directly from documents. Identifier Lists and Code Schedules In the case of certain kinds of named-objects, identi- fier lists are required to ensure that the same organization, place, etc., is always entered in the same manner so that information is not missed during retrieval because of CHIVE INDEXING TECHNIQUE Concepts - 45 - Approved For Release 2000/05/30 : CIPSBORET-03952A000100050001-7 Approved For Release 2000/05SKIMA-RDP78-03952A000100050001-7 incorrect or synonomous entries. In the subject indexing area, a subject authority list or code scheme is required to control the depth of indexing, synonyms, and homographs. In some cases, the authorized entry form will be identical to the way the entry will frequently appear in documents. In other instances, the entry will be converted to a code to either express the hierarchic structure built into the identifier list--e.g., the hierarchic arrangement of organizations in a Communist country--or to compress a long entry into more abbreviated form to conserve storage space. Extracted Words and Phrases Words or phrases extracted from documents are used (a) to index certain kinds of named-objects which will not receive identifier list control, (b) to give greater specificity to subject indexing, and (c) to provide information retrieval via the index record. In the first instance, it is felt that identifier list control of all named-objects is impractical and impossible. Where the volume of reporting is reasonably restricted, or where one can predict fairly well which CHIVE INDEXING TECHNIQUE Concepts - 46 - Approved For Release 2000/05/itaArRDP78-03952A000100050001-7 Approved For Release 2000/05/30: CgOPT8-03952A000100050001-7 named-objects will be the subject of customer queries, it makes sense to control input through identifier lists. Such is the case, for example, for place names and priority organizations and installations. Personalities, however, are neither few in number nor can one readily anticipate which names will be requested. Similarly, in the case of lower-level installations, it would not pay to exercise a high degree of input control when it is probable that the referenced information will be retrieved infrequently, if at all. For both these categories, therefore, we recommend that the burden of overcoming the synonym problem be transferred to the output end of the system. Key words taken from documents are added to subject index categories to provide greater retrieval specificity without complicating the subject schedule. The subject indexing vocabulary provides a medium-depth, generic searching capability. Key words added to the subject schedule provide a specific search capability, e.g., equipment nomenclatures, types of research, new concepts, etc. The third application for entering key words from documents is to provide a level of information retrieval. CHIVE INDEXING TECHNIQUE Concepts - 47 - Approved For Release 2000/05/30 : CIAMMU03952A000100050001-7 Approved For Release 2000/05/3UMTRDP78-03952A000100050001-7 In this case, the entry is uncontrolled, but the class of entry is searchable. For example, one of the personality attributes in the CHIVE system is "Reason for Travel." information would be provided by the index record which would aid in selecting documents or in some cases obviate the need to refer to documents. Linkage Index entries which are related (e.g., an organization and its address) will be linked together in the index record so that the relationship can be interrogated at the index record level, thus negating the need to refer to documents to determine ties among elements of information. This is necessary because intelligence documents typically include many people, organizations, areas, subjects, and their interrelationships. If there were no way to deter- mine the contextual relationship between these subjects, the system would be overburdened with false retrieval matches (false drops) requiring reference to many irrelevant documents. CHIVE INDEXING TECHNIQUE Concepts - 48 - Approved For Release 2000/05/?ElaW-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CKWT78-03952A000100050001-7 Linkage can be accomplished through the use of formatted input, as is typical in punch card systems (i.e., all entries in one defined record are by definition linked), or by appending a linkage symbol to each index entry, as is typical in systems utilizing un- formatted input. Formatted input records are not practical for CHIVE because of the long record lengths and large number of variable elements of information included. Experimentation with appending the linkage symbol to each entry has worked very successfully and will be adopted. Requirements for Identifier Lists and Thesauri The use of identifier lists is recommended for the following reasons: (a) There is little consistency in the way named- objects are reported, e.g., the Institute of Physics of Moscow University may be referred to as the Moscow Institute of Physics, or the Moscow Physics Institute, or the Physics Institute of Moscow University, or the Nuclear Physics Institute, etc. Even place names are translated and transliterated in a variety of ways. Therefore, if named-objects were entered as reported, it would be a very difficult retrieval problem to determine the right synonyms to use in order to find the variant entries. An identifier list includes variants but allows only one correct entry format. CHIVE INDEXING TECHNIQUE Concepts - 49 - Approved For Release 2000/05/30 : CIASMftT03952A000100050001-7 Approved For Release 2000/05/3CUCIRMDP78-03952A000100050001-7 (b) An identifier list (e.g., for organizations) contains not only the name of the organiza- tion, but also a number of identifying attri- butes of the organization, including address, commodities produced, etc. This capsule summary aids the indexer in identifying and discriminating among organizations and improves the quality of the indexing. (c) As was pointed out earlier, an identifier list helps decrease redundant indexing because the common attributes of a named-object do not have to be repetitively indexed when they are listed in the identifier list. (d) Identifier lists are of value for answering queries of a non-complex nature such as the correct spelling of an organization or place, the precise location of a facility, etc. Identifier lists will be required for installations and organizations, place names, significant national and international meetings and conferences, and personalities on whom physical or logical dossiers are maintained. The initial identifier lists will be constructed from the machine language data which exists in OCR, and will be issued to indexers in machine-listing form organized geographically in the various sort orders as required. Key words will be appended to hierarchic classification terms to reflect the terminology of documents and to provide greater search specificity. The initial concept is that these words will be entered as written in documents and will not be subject to thesaurus control. The key words CHIVE INDEXING TECHNIQUE Concepts - 50 - Approved For Release 2000/05/311CM-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: ClAfiefq8-03952A000100050001-7 may be printed out, however, in answers to queries on the hierarchic subject codes to which they are appended, and should aid in determining which documents are relevant. For example, if a requester is searching for a particular aluminum alloy and three of the index records retrieved refer to alloys in which he is not interested, the requester can screen out these references from further consideration. If in the future it is determined that dictionary control over key word entries will raise the quality of the indexing and retrieval, key word thesauri can be created by obtaining listouts of the key words which have been applied to the individual hierarchic codes. These key word lists would be turned over to dictionary editors who would resolve synonym and homograph problems and weed out undesirable terms. It is felt that this method of building a thesaurus, i.e., building it from the actual terminology used in documents, is both superior to and cheaper than trying to adapt an established dictionary to the Agency's indexing problem. In addition, one can take advantage of the uncontrolled key word indexing prior to the building of the thesauri. CHIVE INDEXING TECHNIQUE Concepts - 51 - Approved For Release 2000/05/30 : CIA-SIMET03952A000100050001-7 Approved For Release 2000/05/3g6KRIDP78-03952A000100050001-7 Header Data Indexing The foregoing discussion dealt with CHIVE concepts related to indexing the subject content of documents. Another important aspect of document indexing relates to the so-called header (or bibliographic) elements of the document such as title, author, control number, etc. Header data indexing is required for the following reasons: (a) To obtain bibliographic control of documents over which the Agency has a repository responsibility. (b) As searching parameters in conjunction with subject or named-object searches. (c) To provide minimum index control over docu- ments which are not indexed in depth. In the first instance above, header data control would perform a service comparable to that performed by the source card file maintained by the CIA Library. The machine-stored header data record will be used to verify the receipt of documents in the Agency and to re- cover specific documents whose control numbers are unknown. In the second instance, header data control will be used most often to limit searches (e.g., searches can be restric- ted to certain document series or dates), or a subject request can specify that information is required only when authored by a particular scientist. In the third CHIVE INDEXING TECHNIQUES Concepts - 52 - Approved For Release 2000/05SeCREA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIWPT8-03952A000100050001-7 instance, header data will provide minimum, but important, search keys at very little input cost. Permuted title indexes can be published for certain series (e.g., finished intelligence) in lieu of in-depth indexing. Similarly, searches can be made for all reports issued by a particular post during a specific time period when an important event occurred. In this latter case, all documents can be retrieved whether they were subject indexed or not. Whereas the selection of documents for content indexing will be subject to well-defined criteria and therefore limited, it is anticipated that most documents can be brought under header data control. This possibility is rendered more likely by the fact that header data indexing (with the exception of title expansion) can be performed by clerical personnel, as borne out by the recent CHIVE Indexing Experiment. 5.4.3. SYSTEM DESCRIPTION What follows is a summary description of the indexing technique. A detailed description is given in Appendix 5.C. CHIVE INDEXING TECHNIQUES System Description 5.4.3. - 53 - Approved For Release 2000/05/30 : CIA-SEIMF03952A000100050001-7 25X1A Approved For Release 2000/05/3gMTRDP78-03952A000100050001-7 Elements of Information and Indexing Tools As stated above, the CHIVE indexing concept includes "named-objects" and "subjects." Named-objects refer to people, places, organizations/facilities, and conferences/ meetings. Subjects include commodities, concepts, research activities, military activities, and all other topics and events which do not fall under the above-defined named- objects. Personalities Personality names will be entered more or less as they appear on documents. Only those misspellings will be corrected which it is possible to recognize without reference to identifier lists or other support files. The use of name search tools such as the Name Tables and printouts of unique personal name/surname combinations entered into the system will be investigated as substitutes for controlling names during input pro- cessing. When a specific name is searched, and all of the records relating to that personality have been identified, this identification will be retained so that subsequent searches for the same personality will have to address only those records which have been entered since the previous search. CHIVE INDEXING TECHNIQUES System Description - 54 - Approved For Release 2000/051KROA-RDP78-03952A000100050001-7 25X60 Approved For Release 2000/05/30: CI5kEkBg8-03952A000100050001-7 A detailed list of the attributes of personalities which will be indexed is included in Appendix 5.C. Most of these attributes will be entered in a prescribed manner and thus will be available for direct searching in term files. For example, all locations will be entered from approved gazetteers, dates will be formatted, organi- zation affiliations will be entered from organization identifier lists, etc. This will provide the capability to make information retrieval type queries from the index Organizations/Installations Two levels of control will be applied to organizations and facilities. Priority organizations will be included in identifier lists. These lists will also include significant attributes of the organization, e.g., addresses, synonymous names, function code, products, etc. The lists will be built from the machine language data which exists in SR, BR, and FIB. The organization identifier lists will be issued on a country basis in several arrangements, i.e., by name of organization, by function, and by place name location. CHIVE INDEXING TECHNIQUES System Description - 55 - Approved For Release 2000/05/30 : ClAglECIRE8103952A000100050001-7 Approved For Release 2000/05/36Eetk1RDP78-03952A000100050001-7 For organizations on the list, the indexer will enter an identifying number in lieu of the organization's name, thus ensuring that all indexed information relating to a specific organization can be retrieved exclusive of other organizations with the same or similar names. Attributes of organizations included in the identifier lists will not be re-indexed when the same information is repetitively reported. Low-level installations and organizations will not be identifier list controlled. They will not be indexed by name but rather by location and a function code. It may be desirable later to produce listings of these facilities for mapping and aerial photographic customers. Once these listings are established, it is unlikely that any further indexing of these facilities would be required unless the status of the facility changed. Area/Locations For indexing large geographic areas, e.g., blocs, countries, and provinces within countries, the ISC area code has proven a satisfactory tool and it is recommended that it, or a similar country code, be adopted. For place CHIVE INDEXING TECHNIQUES System Description - 56 - Approved For Release 2000/05/?tcak-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClAFFW15-03952A000100050001-7 names within countries, there are a number of gazetteers available, e.g., OCR generated gazetteers, the NIS gazetteer, etc. The NIS gazetteer has recognized faults, but it is generally conceded to be the most authoritative tool available and it is recommended that it be used as the authority for entering place names. The basic gazetteer will be updated with new place name entries encountered in documents and will be issued on a country-by-country basis. Place names will be entered in clear text as they are spelled in the gazetteer, appended to the appropriate country code. Geographic coordinates will be entered in index records only when they are not associated with a place name. Coordinate searches will be accomplished by a machine search of the gazetteer to locate the appropriate place names having the desired coordinates, followed by a search of the place name term file plus a search for those coordinates that were disassociated with a place name. Meetings/Conferences Significant national and international meetings and conferences will be controlled in identifier lists. CHIVE INDEXING TECHNIQUES System Description - 57 - Approved For Release 2000/05/30 : CIAMHOF03952A000100050001-7 Approved For Release 2000/05/3MCERATRDP78-03952A000100050001-7 Earlier comments on the use of identifier lists for organization control apply to this category also. Less significant conferences will not be indexed by name, but will be subject indexed with appropriate ISC subject codes. Subjects The Intelligence Subject Code has been used throughout the Intelligence Community for a number of years for subject indexing, and it is generally recognized as the best general indexing tool for intelligence documents. For these reasons, CHIVE has recommended that it be used as the basic subject indexing tool in a revised OCR system. However, during the CHIVE Indexing Experiment, several weaknesses were noted which should be corrected prior to its adoption in a going system. ISC Structure The 1960 revision of the ISC did much to simplify its structure. However, experience in using this edition points to several areas where further simplification is desirable. CHIVE INDEXING TECHNIQUES System Description - 58 - Approved For Release 2000/05/?kett-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Ca-gg8-03952A0001000500017 (a) Expanded Use of Modifiers: The ISC subject modifiers are a faceting device which can be combined with certain subjects to specify actions or states which affect those subjects. For example, the modifier "049 Production" can be combined with any commodity to indicate production of the commodity. The 1960 revision greatly expanded the use of these modifiers over previous editions, but further expansion is desirable in two ways: (1) The 1960 revision limited the use of the modifiers, i.e., each modifier could only be used with specific chapters or sections of the ISC. As a result, in sections where a modifier cannot be applied, it has been necessary to set up a subject code in lieu of the modifier. For example, modifier "069 Government Policies, Laws, Legislation, etc." can only be applied to the commodity chapter of the ISC. As a result, a subject code for government policy has had to be set up in various non-commodity sections of the ISC. If the modifiers were freed and the redundant subject codes deleted, it would increase the efficient application of the ISC. During the CHIVE Indexing Experiment, the modifiers were freely applied, and no particular difficulties ensued. (2) In a subject classification system, the same subject is often repeated in several different sections because each section gives a different meaning or emphasis to the subject. For example, in most classifi- cation systems, guided missile subjects would be found under engineering, production activities, and military activities. This repetition is logical, but it complicates the structure of the system and makes it hard to apply. A generalist indexer often CHIVE INDEXING TECHNIQUES System Description - 59 - Approved For Release 2000/05/30 : CIAWIRAT03952A000100050001-7 Approved For Release 2000/05/AEUEIRDP78-03952A000100050001-7 finds it difficult to determine whether the information he is reading is oriented toward engineering or production. If he mistakenly puts engineering information under production, it may be lost in a later retrieval run. With the addition of some new subject modifiers, much of this repetition could be eliminated, i.e., the various subject facets could be shown through the use of modifiers to distinguish production from military activities, etc. This would also considerably reduce the size of the ISC. (b) Expanded Use of Clear Text: Some of the detailed subject breakdowns inthe ISC could be eliminated with a more liberal use of clear text. During the experiment's indexing consistency test, it was found that there was a low-level of consistency in applying the ISC. This can be attributed to the depth of subject detail in the ISC, i.e., one indexer will use "621.349 Uranium" and another indexer will index the same subject matter using "621.351 Natural Uranium." If some of this subject detail were further reduced so that there was only one subject code for uranium, the consistent application of the ISC would rise measurably. Moreover, indexing specificity (e.g., natural vs. enriched uranium) could still be achieved by using controlled clear text as an extension of the subject code. The advantage of this approach is that with more consistent application of the ISC there is less likelihood of losing information. This may often put a burden on the searcher in that with fewer subject categories, more material will initially be retrieved, but this is preferable to losing information and the free use of clear text can help alleviate the problem. Thus, if the clear text is uncontrolled, it can be used as a screening device to get rid of unwanted references, or if it is controlled, it can be used as a searching device to restrict the volume retrieved. CHIVE INDEXING TECHNIQUES System Description - 60 - Approved For Release 2000/05attalf-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CI1AP-8-03952A000100050001-7 Subject Schedules for Occupations and Installations A subject schedule or code is required for occupations 25X60 and installation types in order to respond to queries on such subjects as all 25X60 During the recent Indexing Experiment, specified subject codes in the ISC were designated for this purpose. Since the ISC was not constructed with this aim in mind, the designated codes proved quite inadequate. Problems were caused by the previously alluded to duplication of subjects (e.g., an atomic installation could be indexed in several different places), and by the multiplicity of subjects in the ISC (i.e., a rather simple code schedule was required, and the ISC was too detailed for the required need). In addition, the ISC did not have appropriate subjects for some occupation and installation categories. *This need for a generalized subject schedule for occupa- tions and installation types is to be distinguished from the requirement to retrieve by specific activity. The latter capability will be provided either through the ISC code itself or, where necessary, through ISC plus key word. CHIVE INDEXING TECHNIQUES System Description - 61 - Approved For Release 2000/05/30 : CIA-SE1V03952A000100050001-7 Approved For Release 2000/05/3gCERRIRDP78-03952A000100050001-7 In view of the above problems, it is recommended that the ISC not be modified to perform this function, but that separate subject schedules be developed based on the experience available in the Foreign Installations Branch and Biographic Register. Area Rules The present ISC area rules proved inadequate on several counts during the recent experiment. (a) The terminology is sometimes confusing--e.g., some rules read "nationality is primary area." Since the CHIVE indexing procedures provide for area tags for nationality and primary country, the terminology is subject to ambiguous interpretation. (b) Some of the rules are illogical--e.g., there are two subject codes in Chapter VII which can be used for foreign military training and the area rule for one of the codes is the opposite of the other. (c) The CHIVE indexing technique allows more flexibility in area relationships than is allowed in the ISC as used in the Intellofax system. Consequently, there are many subjects which do not have area rules which should have them appended for CHIVE purposes. All these rules should be re-examined and modified before the CHIVE system goes operational. CHIVE INDEXING TECHNIQUES System Description - 62 - Approved For Release 2000/05MCW1-RDP78-03952A000100050001-7 SECRET Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Subject Gaps Prior to the recent experiment, it was felt that a number of subjects occurred in Codeword materials which did not appear in Collateral documents and that, therefore, the ISC would not be adequate for indexing these materials. For this reason, sections of the Special Register code manual were utilized as a supplement to the ISC during the experiment. As it turned out, the SR supplement was not used a great deal because the ISC had subject cate- gories which were almost comparable. However, there are a limited number of special-purpose subjects which should be added to the ISC to make it fully suitable for all- source indexing. Tags Each entry in the CHIVE indexing system is preceded by a tag. A tag is a three-digit mnemonic symbol which identifies the entry which follows. Tags are used to: (a) Distinguish between homographs, e.g., Washington a person vs. Washington a city or street. (b) Organize machine files, i.e., separate people's names from organizations and subjects and thereby facilitate searching. CHIVE INDEXING TECHNIQUES System Description - 63 - Approved For Release 2000/05/30 : ClAk5liFt3103952A000100050001-7 Approved For Release 2000/05/AE:CaRTRDP78-03952A000100050001-7 The CHIVE tags were made mnemonic as a memory aid. The first character of a tag represents a major subject category, e.g., "P" = Personality, "0" = Organization, etc. The second and third characters further specify the element of information being indexed, e.g., "PVN" = Personality Name Variant, "POH" = Personality Organization Head, etc. (See Appendix 5.C. for a detailed list of the CHIVE elements of information and their associated tags.) Phrasing The requirements for linkage were discussed earlier. In the CHIVE system, linkage is accomplished through a system defined as phrasing. A phrase is simply a group of tags and terms which the indexer relates together with a unique number which is assigned to each tag and value in the group. On retrieval, queries can specify that the input linkage must be present for the query to 25X6 be satisfied--i.e., a query may specify all information on a person 25X6 Without phrasing (linkage), all documents which contained CHIVE INDEXING TECHNIQUES System Description - 64 - Approved For Release 2000/05/MM-RDP78-03952A000100050001-7 25X60 Approved For Release 2000/05/30: CIWPf8-03952A000100050001-7 these two terms would be retrieved and in some cases the relationship would be accidental. On retrieval, the phrase linkage can be reconstituted by testing for those terms which have the same document accession number and phrase number. The rule for phrasing is very simple. All terms which are logically related can be combined in a phrase. Thus, if a person is affiliated with an organization in these three elements of information can be combined together in a phrase. However, if additional information were given that this individual also traveled 25X6 to an additional phrase would have to be constructed otherwise it might be interpreted that the organization, if it appeared in the same phrase, was located in both 25X6 Phrases can be very simple or complex. The simplest phrase contains only a place name or an area and one subject. A complex phrase may contain a number of index terms which constitute a rather detailed biographic sketch of an individual. Further details on phrasing with examples are contained in Appendix 5.C. CHIVE INDEXER TECHNIQUES System Description - 65 - Approved For Release 2000/05/30 : CIASIONT03952A000100050001-7 Approved For Release 2000/05/3WAETRDP78-03952A000100050001-7 Header Data Indexing Header data indexing will be performed by clerical personnel who will type the information on formatted transcript sheets. During the recent experiment, as illustrated in the header section of Appendix 5.C., a single transcript sheet was used for all documents. This does not appear to be as efficient as developing unique transcript sheets for different series. The elements of information comprising header data will be taken from the document or the information from the document will be translated into code to achieve conciseness and uniformity of entry. A formatted transcript sheet can be used since the header data elements are fixed in number for each document series, and the length of entries is either fixed or a maximum field length can be determined. The use of a formatted sheet obviates the need for tags and the only linkage required is to the document control number. The latter will be appended automatically to each header data term. A detailed list of the elements of information comprising header data is included in Appendix S.C. CHIVE INDEXER TECHNIQUES System Description - 66 - Approved For Release 2000/05aGCRGIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIV-gg8-03952A000100050001-7 Chapter 5.5. SYSTEM FILES 5.5.1. INTRODUCTION This chapter classifies and describes the logical files and sub-files which will be available in the CHIVE system. These are the files which are identified to the user--i.e., the CHIVE information analyst and, perhaps ultimately, the research analyst. They are the files he must be familiar with, if he is to take full advantage of the resources of the system and exploit it intelligently. The total number of individual system files, including old as well as new, might easily exceed a hundred. However, it is possible to classify all the various files into no more than nine types, each with very distinctive functions and properties. categories are as follows: Document Index Files: Files These nine containing all the raw document index records in the system, including not only the complete index records themselves but the access mechanism to these records. The documents referenced by these records may include any form of information carrier --e.g., maps, photos, films, or other, and need not neces- sarily be readily accessible to the system. - 67 - Approved For Release 2000/05/30: CIA?kW-03952A000100050001-7 SYSTEM FILES Introduction 5.5.1. Approved For Release 2000/05/AKWRDP78-03952A000100050001-7 Vocabulary Control Files: Files required to insure consistent entry of index terms (tag and value) into the Document Index Files and other system files. The principal function of these files is to reduce the synonym problem at search time. They include "identifier files" for named objects (which, like scope notes in a code schedule, help to distinguish one specific subject from another), code books, dictionaries, thesauri, and other authority lists. Unsynthesized Information Files: Files consisting of select phrases or terms extracted from documem_ index records or directly from the raw documents themselves. Such files would be built to facilitate retrieval where a substantial number of requests for the pertinent data can be anticipated on a continuing basis. Unlike Summary Information Files (see below), records in these files would often contain duplicative and/or contradictory information. Periodically, however, inforation in such files might be reviewed and added to the appropriate Summary Information Files. Summary Information Files: Files built either from records (or portions of records) in the Document Index Files, from records in Unsynthesized Information Files, or from the raw documents themselves during or after input processing. The distinguishing feature of these files is the fact that they will ordinarily contain evaluated, non- redundant data about named objects or events associated SYSTEM FILES oduction Approved For Release 2000/0WricRe114-RDP78-03952A00010000.#1t. Approved For Release 2000/05/30 : ClaSP1-03952A000100050001-7 with named objects. Named-object identifier files could be placed in this file category, the only apparent difference being the limited amount of historical data ordinarily found in such files. Special Project Files: The unique features of these files are as fellows: (a) the inputs to the files originate outside CHIVE; (b) CHIVE actually acquires the files and not simply "profiles" thereof; (c) additions or modifica- tions to the files can be anticipated; (d) the files do not use the elements of information and/or vocabulary controlled in CHIVE. Special Project Files may otherwise have the properties of any of the file classes named above. These files will be processed by CHIVE but maintained by CIA or other agency analysts. The degree of CHIVE involvement in such files remains to be determined since the responsibility for such files is currently assigned to the Applications Division of OCS. Referral Service Files: These files differ from Special Project Files in that they are not substantive data files but rather descriptions or profiles of files located outside the CHIVE system. Referral Service Files will consist both of profiles of analysts' special fields of competence as well as files maintained by analysts and/ or information repositories external to CHIVE. CHIVE will not maintain, or retrieve data from, the substantive files themselves. It will simply inform customers of those files potentially relevant to a given query. SYSTEM FILES Introduction - 69 - 5.5.1. Approved For Release 2000/05/30 : CIA1M03952A000100050001-7 Approved For Release 2000/05/%69K-RDP78-03952A000100050001-7 Document Image Files: Files of documents stored by the CHIVE system. From a functional point-of-view they include "aspect" systems (where the index is stored separately from the documents) as well as self-indexed document files. Both existing OCR document collections as well as CHIVE-originated document repositories are encompassed by this category. The storage media for such files will include hard copy, various types of microimages, and even digital storage in some instances. Similarly, the categories of documents involved will differ widely in size, shape, classification, and point of origin. Management Data Files: Files of data collected on the activity of the CHIVE system to (a) enable operational management to evaluate the cost/performance ratio of the system and (b) to guide system designers in improving hardware and software support. From the point-of-view of what data is collected, most of the Management Data Files will have to do with either system processing times or processing volumes. System Processing Files: Files used to support the system in processing data. Most such files will be organized in table form enabling values to be obtained from arguments. Examples would include a file of legal tags and other error correction files, decode dictionaries which would convert codes into clear text for display to a reader, SYSTEM FILES - 70 - Introduction 5 . 5 .1. Approved For Release 2000/05$11CROA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAW8-03952A000100050001-7 intermediate files which exist only temporarily during the processing, of a transaction, working storage files, etc. Since these files are largely internal to the CHIVE EDP System and the information analyst need not interact with them in any direct way--only know what functions the system is capable of performing--they will not be covered further in this volume but rather in Volume VII of the report. For each of the file categories listed above a second-level categorization may be required, i.e., one which classifies CHIVE files fron the point-of-view of the origin of the files. These classes are three in number: Chive-Built Files: Files built by and for the CHIVE system either from new inputs or through the conversion of existing OCR files to the format and vocabulary of CHIVE. These files will be continually updated as part of the regular processing cycle. Inherited Files: Files originally established by the various OCR systems which it was not found possible to integrate with new CHIVE files. Such files will include records in hard copy as well as machine language. In some instances these files may be transferred to another storage medium (e.g., magnetic tape) if querying and output can thereby be improved. Similarly, some existing machine- readable files may be restructured and interrogated in SYSTEM. FILES - 71 - Introduction Approved For Release 2000/05/30 : CIASMEBT03952A000100050001-7 5.5.1. Approved For Release 2000/05/31EaRITIRDP78-03952A000100050001-7 the vocabulary of a single CHIVE language. Neither of these changes, however, implies true conversion to the CHIVE system. Another significant difference between these files and Chive-Built Files is that while both will be used by the CHIVE information analyst, no additions will be made to the Inherited Files once the CHIVE system is fully operational. Supplemental Files: Files not built or maintained by CHIVE, nor inherited from OCR, but which contain data functionally useful to CHIVE as a secondary source of information. All Special Project Files (see above) fit this category, as do reference aids of various kinds (e.g., Who's Who compilations, gazetteers, commercially published indexes, etc.) obtained from external sources and left essentially in the form in which they were received. In the broadest sense the CHIVE system must necessarily include not only the new files it creates but the files it inherits from the existing system. The discussion of these separate but related subjects, however, has been divided in the pages to follow to lessen the possibility of losing the reader in the file forest. In the main body of this chapter, we will focus primarily on the CHIVE-Built Files, providing a summary description of their functions, data content, maintenance criteria, and SYSTEM FILES Introduction 5.5.1. - 72 - Approved For Release 2000/05g1E0W-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIASIkgg-03952A000100050001-7 other characteristics. Appendix 5.D to this volume describes the principal Inherited Files which must be accommodated by the system, with primary attention given to those which fall in the categories of Document Index and Document Image files, as defined above. It should be emphasized that the basic objective in this chapter is to communicate a more or less static image of the files in order to simplify understanding of the structural framework (or file philosophy) of the system. In Chapter 5.6. we will examine the more dynamic aspects of file activity within the system, i.e., the transactions which will affect the files, interactions which might take place between files, etc. 5.5.2. DOCUMENT INDEX FILES Master Index File (MIF) The Master Index File of the CHIVE system will contain the index entries for all the documents available in the CHIVE system as well as certain classes of documents located in repositories not under CHIVE management. Examples of the latter include maps, the storage responsibility for which will be retained by the Map Library Division of ORR, and select open-source books and periodicals which may be accessible only at the Library of Congress or at some other holding agency. Conceivably, certain documents indexed by SYSTEM FILES Domment Index - 73 - Approved For Release 2000/05/30 : CIAIRInt-03952A000100050001-7 Approved For Release 2000/05/kTRDP78-03952A000100050001-7 CHIVE may not even be available at all--for example, select Soviet periodicals never received in this country, but which were described in secondary sources that were accessioned. In all cases, however, whether the original source document is readily available or not, the preparation of index records for the CHIVE Master Index File will be under CHIVE format and vocabulary control no matter where the records are physically prepared. All index records will be stored in such a manner that a search, based on certain criteria, will produce all the records in the system or, at the customer's option, phrases and/or terms within records which may apply to the search criteria. The index records will contain sufficient information to enable the requester to determine if the document referred to in the index entry should be requested for detailed study. In the case of named- object associated informPLion, the entries will have sufficient information-bearing content to permit summary data files to be built and responses given to certain queries directly from the index records themselves without referral to the source documents. Records entering the Master Index File will originate from the following sources: - CHIVE information analysts processing incoming documents in the CHIVE geographic divisions. - Graphic analysts indexing photos and films in the Graphics Register (GR). SYSTEM FILES - 74 - Document Index Files Approved For Release 2000/05/acWr-RDP78-03952ACIC5i00050001-7 uproif mod Tommie 4111W Approved For Release 2000/05/30 : eragD78-03952A000100050001-7 - Map catalogers processing maps in the Map Library Division (ML), ORR. - Miscellaneous additional organizations (either under Contract to CHIVE or agreeing to follow CHIVE input procedures) exploiting primarily foreign language documents. Examples of such organizations might be the Library of Congress, FDD, etc. - Documents received by CHIVE in machine language (e.g., Comint teletype) on which a limited form of automatic indexing is to be performed. - Machine-converted document index files from existing central repositories. With regard to input selection criteria, assuming continuation of present practices, CHIVE will have the responsibility to serve as the Agency's repository for community-published positive intelligence materials (with the exception of cables and maps), and to provide reference service on "active" documents. In addition, it will pre- sumably assume OCR's obligation to serve as the office of record for archival storage of certain CIA document series In order to fulfill these responsibilities, CHIVE will be obliged to index at least the header (or biblio- graphic) data for every "intelligence" document received. By "intelligence" documents we mean all categories of textual materials generally considered to be in the mainstream of intelligence reporting. These include Comint (messages, reports, and teletype), T/KH reports, USIB-produced IR's USIB-produced finished intelligence, the FBIS, photo enclosures to IRs, and USIB-produced trans- SYSTEM FILES - 75 - Document Index Files Approved For Release 2000/05/30 : CISMISO78-03952A0001600%080117 Approved For Release 2000/05/3WaIRDP78-03952A000100050001-7 lations of foreign documents. By agreement with the Library it will also store map index records generated by ML. The preparation of index records on other categories of materials e.g., cables, non-USIB-produced reports studies,ni films, and original open-source literature, depend on the substantive content therein. The content of document index records can include any term type permitted by the vocabulary of the CHIVE ndexing system. (For a list of all permissible term see Appendix 5.C.) No single record will, of course, oortain all poss bte term types sin-e some terms will be uue to certain kinds of documents. Outputs from the Master Index File will consist of scheduled and ad hoc products. The principal items be provided within each category are briefly described Scheduled Products KWIC listin of titles or expanded titles of all documents which ha- not been content indexed, as well as the PHIS laily Reports. The permuted port!on of the list4nrY will be the title and exnahded title, wh.le the reference portion will clude basic header data for the document including document control number. Separate, as well as combined, listings will probably be run for the different categories of documents involved, e.g., SI Teletype, FBIS, Finished Intelligence, and Raw Intelligence Reports, - 76 - Approved For Release 2000/055513ERGR-RDP78-03952A000100050001-7 SYSTEM FILES Docnr?ni- Index Files 5., 2.1. Approved For Release 2000/05/30 : CIAFFW8-03952A000100050001-7 - Map catalog cards in 3" x 5" form containing in clear text on each card the entire index record for a map. This record would include accession number, area code, subject, scale, classifica- tion, map title, date of publication and name of publisher. The cards would be outputted in the sequence of the Map Library Card Catalog to facilitate interfiling at the Map Library. - Output similar to the map cards, but reflecting index records on films stored in the Master Index. In this instance, the records will probably be of tab card size to conform with the size of the existing file. The sequence will also conform with the existing Intellofax reference card file on the film collection. - Accessions lists comprised of clear-text index records on maps, ground photos, and perhaps select additional document receipts (e.g., tables of contents of foreign scientific periodicals) processed by CHIVE. Ad Hoc (Query) Products - Listings in natural language of document index records or subsets thereof (i.e., phrases within records) containing the search terms specified in the query. Subject or concept-oriented queries will normally require output of complete document index records including header as well as content data. Named-object-oriented queries will ordinarily result in the output of select phrases only which match the search criteria, together with a limited amount of header data (e.g., document classification, document type, and appropriate document control numbers). - Listings of control numbers only for documents whose index records match the search parameters. SYSTEM FILES Document Index Files - 77 - Approved For Release 2000/05/30 : CIA-Millif03952A000100050001-7 Approved For Release 2000/05/3gC:RAIRDP78-03952A000100050001-7 - Listings containing simply a computed figure of the number of index records matching the search parameters. This kind of intermediate output will enable the customer to broaden or narrow the search prescription depending on the volume of the anticipated output. - Listings of index records containing terms which match standing customer queries or analyst interest profiles. Whenever hits occur because new information is received on a subject or person of interest to a particular research analyst, the customer would be notified through the transmittal of the listing containing the pertinent record. 5.5.3. DOCUMENT IMAGE FILES The Document Image File (DIF) is the central repos- itory for active textual intelligence documents. Maps and graphics are to be retained within the respective organizations currently responsible for their retention, although all of these will be used in conjunction with the computer-based Master Index File described in the previous section. The Document Image File shall consist of those textual intelligence documents for which the Agency has repository responsibility as well as other documents which are judged to contain information of potential value within the intelligence community. As a central repository it is to be all-source, containing SYSTEM FILES Document Image Files 5.5.3. - 78 - Approved For Release 2000/05/?tcRt-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAW8-03952A000100050001-7 USIB finished and unfinished intelligence reports, FBIS, Open-Source literature (including JPRS and FDD translations), COMINT (messages, reports, and teletypes) and selected cables. The system will be inclusive of inherited document image files (see Appendix 5.D.) as well as newly-accessioned, CHIVE-processed documents. To effect this, files currently maintained in various locations (SR, LY/Circ, etc.) would be moved to a single physical area within the headquarters building along with the CHIVE document system, thus offering the user a single point of entry for his reference needs. The discussion of a proposed approach to implementing this central repository is contained in section 5.7.3. The primary purpose of the Document Image File is to serve as a central reference point from which identified documents may be retrieved and copied for distribution. The identification of the documents (by unique identification number) may be accomplished via a computer search of the Master Index File, or it may be known by some other means by the requester. The file must be responsive to either type of demand. Documents SYSTEM FILES Document Image Files 5.5.3. - 79 - Approved For Release 2000/05/30 : CIA-911E1113T03952A000100050001-7 Approved For Release 2000/05/30SFE#4DP78-03952A000100050001-7 are not to be circulated outside of the file area; and requests are to be serviced by producing a durable, hard-copy replica of the document master for distribu- tion to the requesting user. The design goals for the volume and turn-around times in responding to these file demands are outlined in sections 6.2.1. and 6.5.6. Aside from its primary purpose of providing a repository for retrospective reference, a number of secondary purposes must be served by the system. First, provision must be made for a backup file capa- bility. This duplicate file must be produced as a by-product of the input procedure, and must be suitable both as an alternate reference point in the event of loss or destruction of items in the main file, as well as a means of reconstructing the main file in the event of catastrophic destruction. Provision for selective protection of vital records is also within the scope of the document image subsystem although no special design consideration has been devoted to this requirement in this study. An additional implicit requirement of the document system is the need to provide archival quality records for those items requiring prolonged retention. SYSTEM FILES Document Image Files 5.5.3. - 80 - Approved For Release 2000/05.1513:CREIA-RDP78-03952A000100050001-7 MOI pip 25X1B Approved For Release 2000/05/30: Ca00}8-03952A000100050001-7 5.5.4, VOCABULARY CONTROL FILES Personality Identifier Files Master Dossier File (MDF) The functions of the Master Dossier File are: - To identify hard copy folder files maintained by CHIVE on select personalities.* - To reduce search time on requests for select personalities by virtue of the fact that the information analyst determined in advance of the request which incoming name references pertained to these personalities. An analogy could be drawn here to the difference between searching a tightly controlled classification *The reason for maintaining hard copy folder files, in addition to storing documents in the Master Image File in microimage form, would be the anticipated high request activity on these select documents which would increase reproduction costs significantly. An alternative to maintaining hard-copy personality files would be to maintain lists of documents referring to select individuals, and, when one of these individuals' file is requested, to reproduce all the documents referred to in the list. This approach of building a dossier on a "demand" basis would make sense if experience proves that redundancy in name searches is minimal. Pending further study, however, of the redundancy factor during Phase III, we have assumed a requirement for some hard-copy personality files and made provision for same in the system design described here. SYSTEM FILES Vocabulary Control Files - 81 - Approved For Release 2000/05/30 : CIA-Mege103952A000100050001-7 Approved For Release 2000/05/305.EgkElbP78-03952A000100050001-7 system and an uncontrolled keyword index. A listing of the Dossier File, wherein is contained a unique file number and set of attributes for each personality cited, is directly analogous to a listing of a classified schedule which contains both a unique code and often scope notes defining each term in the listing. - To provide by means of a printout of the identifying information on each dossier personality a summary-type information record which can be used to answer requests, serve as a reference aid for research analysts, facilitate screening of dossier files without the necessity for examining the files themselves, etc. The initial CHIVE Master Dossier File will be derived from the BR dossier system with possibly some deletions to the latter file. The subsequent creation of new dossier records will occur largely as a result of name searches, taking advantage of the fact that document index records and their related documents have been analyzed in the course of answering the customer's inquiry. Dossier identifier records may refer either to physical or logical dossiers. In those cases where it seems desirable to establish a hard-copy file on a personality, a digital record containing select elements of identifying information (see below) will be prepared and added to the Master Dossier File. In addition, all SYSTEM FILES Vocabulary Control Files - 82 - Approved For Release 2000/05a0CgMk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : 4-g5'8-03952A000100050001-7 documents containing information on the individual will be reproduced and stored in a folder on the personality involved. Logical dossiers will also be represented by identifier records in the digitalized MDF, but the docu- ments relevant thereto will be accessible only in the Master Image File. Maintenance of both the hard copy dossiers as well as the digitalized Master Dossier File--although left to the discretion of the analyst--would also ordinarily be performed as a corollary function to name searches and not at the time incoming documents are indexed. This means that a given hard-copy folder file will not necessarily contain all the available documents on an individual which may be held in the Master Image File except immediately subsequent to a request having been answered on said personality. Similarly, the digital record on a dossier personality will only be current as of the time of the last request. The contents of a digital record in the Master Dossier File will be: SYSTEM FILES Vocabulary Control Files 83 - Approved For Release 2000/05/30: CIANSHEMX03952A000100050001-7 Approved For Release 2000/0513g6KRDP78-03952A000100050001-7 - Personality Name - Variant Name - Telegraphic Code - Dossier Number - Birth Date - Citizenship - Date of Death - General Occupation - Organization Affiliation - Position Title - Organization Affiliation Date (year only) - Date Record was Last Updated - Document Reference Numbers Whenever new dossier records are added to this file or changes made to existing records as a result of name searches, the following actions will take place: (a) Documents not previously filed in physical dossiers will be reproduced for same. (b) Dossier identifier records will be created or updated. (c) The list of document control numbers attached to each identifier record will be compiled or updated. The effect of (c) will be to establish the identity of the individual mentioned, thus capturing the results of SYSTEM FILES Vocabulary Control Files - 84 - Approved For Release 2000/05at4tft-RDP78-03952A000100050001-7 AIM Approved For Release 2000/05/30 : CIA#EW8-03952A000100050001-7 the analysis. Persons searching the same name at a later date will have to employ standard search strategy techniques only to recover those records from the Master Index File which might have entered the system subsequent to the previous search. Earlier references will be available either via the hard copy dossier itself or, in the case of logical dossiers, through an "absolute" search on the document numbers known to be relevant to the individual concerned. The latter can be a semi-automatic process in which the information analyst need only specify the dossier number involved-- i.e., the computer will find the document numbers perti- nent to the dossier, and either print them out or use these numbers to locate and output the corresponding index records. In the initial CHIVE system, scheduled outputs from Master Dossier File will consist of: - Master listings in natural language of the personality identifier records arranged by name within citizenship. - Cumulative supplemental listings in the same arrangement as the master listings. SYSTEM FILES Vocabulary Control Files - 85 - Approved For Release 2000/05/30 : CIA-MERIEV3952A000100050001-7 25X1A Approved For Release 2000/05/3%E:CdihIRDP78-03952A000100050001-7 Demand (ad hoc) products of the file will include natural language printouts of the records on a variety of media (e.g., cards, listings, etc.) in any sort order desired by customers. Name Group Tables The function of name group tables is essentially that of any dictionary of synonym and "see also" references. Such tables properly belong in a list of "vocabulary control files" since, like any term dictionary, they serve to relate the several ways in which a term (in this case personality name) can be spelled to a standard code. In the CHIVE system, it is proposed to experiment with the two kinds of name group tables developed by and for the Surname Table and Given Name Table. Each of these tables contains a list of all the surnames or given names, as the case may be, which have occurred within the system. Listed with each name is a reference to the name group to which it has been assigned. The functions of these name tables are: (a) to determine if a specifically spelled surname or given SYSTEM FILES Vocabulary Control Files - 86 - Approved For Release 2000/051tirElfk-RDP78-03952A000100050001-7 25X1A Approved For Release 2000/05/30 : CgtMN;i8-03952A000100050001-7 name is contained within the system, and (b) to associate a group number to the name. A new name entry in a document index record, before it can be filed, must match a name in the name table. If it finds no match, the machine will print out a notice to the information analyst to this effect. The latter will then consult his tables or the...II/CHIVE expert concerned (pro- cedure to be determined), assign a name, and re-enter the Ideally, the name record into group table group number to the the machine file. concept reduces the intellectual problem for the name searcher by providing for a guided search of potentially relevant, alternative name spellings. This capability will not, however, preclude the searcher from bypassing the name grouping feature if he wishes the machine to yield only those records which exactly match the spelling(s) in his request. Scheduled products of the Name Group Tables will include listings of both the surname and given name tables arranged in both name and group number order. Query products may include the variant names searched within a given name group as well as "see also" references to names in related groups. SYSTEM FILES Vocabulary Control Files - 87 - Approved For Release 2000/05/30 : CIA-Ma103952A000100050001-7 Approved For Release 2000/05/31 6KRDP78-03952A000100050001-7 Organization/Facility Identifier Files The Master Organization/Facility Identifier File (MOFIF), like other vocabulary control files, is required to insure consistent indexing of items of information derived from documents--in this case organi- zations or facilities (installations). In the sense used here, organizations and facilities are defined in the broadest possible terms. They include political, economic, military, cultural and scientific bodies, as well as physical installations which are relatively fixed in terms of geographic location (e.g., a weather station). Like the Master Dossier File, the functions of the MOFIF will be: - To identify hard-copy folder files maintained by the system on select organizations where high request activity is anticipated. - To provide, via printouts from the file, identifying information about organizations which an information analyst can browse in order to determine (a) whether he has previously assigned a code or unique identifying number to an organi- zation and/or (b) whether there is a hard-copy dossier available on an organization. - To reduce search time on requests for "controlled" organizations. SYSTEM FILES Vocabulary Control Files - 88 - Approved For Release 2000/0WROA-R0P78-03952A000100050001-7 25X6 Approved For Release 2000/05/30: Clglqfq8-03952A000100050001-7 - To provide a summary-type information record which can be used to answer requests, serve as a published reference aid, etc. Not all organization and facility names will be placed under MOFIF control. Furthermore, not all organi- zation references encountered in documents will be indexed by the specific name and/or identifying number of the organization mentioned. Some organizations will be indexed only by type using the OTF tag. Still others (e.g., a laboratory or committee) will be indexed by their parent organization's code, but will not be assigned a unique identifying number of their own, con- sequently they too will not appear in the MOFIF. The initial CHIVE Master Organization/Facility Identifier File, which will be composed will be built during Phase III of the CHIVE project from existing organizational dictionaries developed by FIB, SR, and BR. Each organization record resulting from this process of analysis and synthesis will include, in addition to the CHIVE-assigned identifying number and name, the code number or numbers (if any) by which the organization was previously 25X6 SYSTEM FILES Vocabulary Control Files - 89 - Approved For Release 2000/05/30 : CIA-BEEIZET03952A000100050001-7 Approved For Release 2000/05/36ECKIRDP78-03952A000100050001-7 identified in the OCR register(s) from which it was derived. These "cross reference" numbers will help indicate to the searcher whether there is information stored on an organization in one or inherited from OCR (e.g., SR Detail and plant folder files, BR dossier, more of the files Index, FIB card etc.). The absence of such cross references in a record would mean either that there was no inherited information available on the organization, or that the organization was so loosely controlled in the earlier system that a subject search or some other method for accessing the files would be required to uncover the pertinent data. As in the case of personalities, it is planned that certain organizations would have hard-copy dossier files where a high request rate is anticipated (but see footnote in section regarding request redundancy study). If this plan is implemented, incoming documents containing information on dossier-controlled organizations would probably be added to the dossiers as a part of the initial processing activity rather than at the conclusion of search operations on said organizations (as was proposed in the case of personality dossier maintenance). The SYSTEM FILES Vocabulary Control Files - 90 - Approved For Release 2000/05/HdeVRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cl?tEk8P8-03952A000100050001-7 reason for this is that the MOFIF would ordinarily have to be consulted when indexing an organization in order to obtain the correct CHIVE identification number for the organization. Such being the case, little additional effort would be required to determine from the MOFIF listing whether a dossier was being maintained on the organization and, if so, to direct that a copy of the document be deposited in the dossier concerned. The contents of a digital record in the Master Organization/Facility Identifier File will be as follows: - Translated Name and/or Number - Functional (Assigned Name) - Foreign Language or Transliterated Name - Variant Name(s) - Previous Name(s) - Name Abbreviation - Telegraphic Code - CHIVE-Assigned 0/F Number - Dossier Indicator - Cross-Reference Numbers ? FIB ? SR SYSTEM FILES Vocabulary Control Files - 91 - Approved For Release 2000/05/30 : ClAfalia8f03952A000100050001-7 Approved For Release 2000/05/30SF6WDP78-03952A000100050001-7 ? COMOR ? BR ? BE ? NPIC ? TDI - Address ? Country ? Political Subdivision ? Place Name ? Coordinates ? Street Address ? Cable Address ? Post Box Number - Parent Organization - Type 0/F - Source Citations - Remarks Not all the elements of information listed above will appear in every organization/facility record in the MOFIF. Not only will the type of organization have an effect on the elements of information that will customarily SYSTEM FILES Vocabulary Control Files - 92 - Approved For Release 2000/05/Alcaitic-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClAkiAlg-03952A000100050001-7 appear in its identifier record (e.g., a political body will not have a COMOR or BE number, nor perhaps a specific address), specific elements of information will be unavailable on many organizations and facilities. Source citations, where desirable, for items of data carried in the MOFIF can be included in the identi- fier records by referencing the control number of the document which provided the information. One source reference for each element of information in an MOFIF record would probably be sufficient. If additional supporting evidence for a given fact was required, the index records in the Master Index File could be searched. The "Remarks" field of an MOFIF record is intended for use in recording historical facts about changes in organizational nomenclature, hierarchic relationship to other organizations, etc. This information will not be directly accessible, but may be displayed on printout to enable searchers to determine how to formulate a request which will insure recovery of all pertinent data about an organization despite organizational changes which might have taken place over the years. SYSTEM FILES Vocabulary Control Files - 93 - Approved For Release 2000/05/30 : ClAgRIRM-03952A000100050001-7 Approved For Release 2000/05/SECtiA-RDP78-03952A000100050001-7 In the initial operational system, current thinking is that human interface with this file, as well as all other vocabulary control files, will be through the medium of the printed listing. It is recognized that this is not a wholly satisfactory solution (although a familiar one), particularly in view of the probable increase in the number and size of vocabulary control files which the CHIVE indexer must routinely consult. For this reason, an in-depth study of the matter is planned during Phase III of the CHIVE project. Scheduled outputs of the Master Organization/ Facility Identifier File will consist of: (a) Master listings in natural language of the organization identifier records pertaining to a single country. The types of master listings required are: (1) A permuted title listing of all organization names (including official, assigned, variants, etc.). The reference portion of the listing will be ordered by 0/F number and contain the complete identifier record for each organization referenced. (2) A listing of MOFIF records, without organization name permutation, ordered on place name. (3) A listing identical to (2) but ordered on type of organization/facility. SYSTEM FILES Vocabulary Control Files - 94 - Approved For Release 2000/05/glitai-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAS-E14g03952A000100050001-7 (4) A listing identical to (2) but ordered on geographic coordinates. (b) Cumulative supplements to the master listings issued in the same arrangements as the master listings but bound together. Alternatively, supplementary listings may be issued in the form of pages to be inserted in master listings. Demand (ad hoc) products of the file will include natural language printouts of the records on a variety of media (e.g., cards, listings, etc.) in any sort order desired by customers. Meeting/Conference Identifier Files In the planned central retrieval system a requirement exists for retrospective searching of documents dealing with certain meetings and conferences. The conditions which dictate whether a given conference should be indexed by name or identifying number cannot be stated with complete precision at this time. Nevertheless, the fact that some conferences (possibly international scientific meetings attended by USSR nationals) must be controlled dictates that the capability be provided in the CHIVE system to retrieve the pertinent data, whatever the input criteria might ultimately turn out to be. SYSTEM FILES Vocabulary Control Files - 95 - Approved For Release 2000/05/30 : CIA*51RfET-03952A000100050001-7 Approved For Release 2000/05/58QU-RDP78-03952A000100050001-7 The function, therefore, of the Master Conference Identifier File (MCIF) is to relate the several ways in which the name of a conference or meeting may be spelled to a standard code and, in addition, to supply other identifying information which would facilitate distinguishing meetings having similar names. The initial data base for the MCIF may be derived 25X6 from the BR's International Conference and Travel File. In this instance, a requirement does not exist to merge separate OCR system vocabularies since BR is the only organization which maintains a conference authority file. Informal consultation between the CHIVE conference dictionary editor and BR's authority in this area during the evolution of the CHIVE system should enable standardization to be achieved between the two systems on the identification of international meetings which both systems index. The contents of the MCIF will be as follows: - Name of Meeting/Conference - Assigned Code Number - Location ? Country ? City SYSTEM FILES Vocabulary Control Files - 96 - Approved For Release 2000/05/gpca*-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: Clh-1EPA-03952A000100050001-7 - Date of Conference - Type of Meeting (Subject) - Sponsor Organization Scheduled outputs from the file in the initial CHIVE system will consist of the usual master listings arranged in this instance by name of conference, location, and sponsoring organization. The name listing will be a permuted title arrangement with the reference portion of the listing ordered by code number and containing the complete identifier record for each conference referenced. Supplementary listings will also be provided either in cumulative form or (as indicated earlier) as pages to be inserted into master listings. Demand products will, of course, be issued in any sort order desired. Location Identifier Files The function served by the Master Location Dictionary (MLD) is to specify the approved entry form for certain classes of locational-type information listed below. (Detailed address information, e.g., cable address or street name, will not be under vocabulary control and, consequently, will not appear in this file.) Additional SYSTEM FILES Vocabulary Control Files - 97 - Approved For Release 2000/05/30 : CIASINFREV03952A000100050001-7 Approved For Release 2000/05/aCOKRDP78-03952A000100050001-7 uses of the file will be to confirm file coverage by location, to show hierarchical and synonomous relation- ships between place names and political/administrative regions of the world, and to support requests by location defined in terms of country, political subdivision, or place name. Location identifier (authority) files will ultimately be maintained on all countries, but in the initial system primary concentration will be placed or the element of the Master Location Dictionary. Specific senior content indexers will serve as dictionary editors for certain geographic portions of the file, rejecting or approving all new entries generated as a byproduct of the document input process. The initial Master Location Dictionary will be constructed on the base of the NIS Gazetteer. The ISC 4-digit classification system will be used to identify country and political subdivision. Place name entries will be carried in full text. Map catalogers at the Map Library, according to the terms of a tentative agreement arranged between CHIVE and SYSTEM FILES Vocabulary Control Files - 98 - Approved For Release 2000/05akW-RDP78-03952A000100050001-7 25X6 Approved For Release 2000/05/30 : dxf-WaT78-03952A000100050001-7 the Map Library, will employ a modification of the ISC area code which appends the ML provincial codes to the ISC area code. This expanded code will permit ML to continue to index provinces and other political sub- divisions where this degree of index specificity may not be required for document retrieval. During the preparation of the map index transcript sheets at the Map Library, the map cataloger will either (a) enter the modified ISC area code in addition to the ML area code on the transcript form, or (b) enter the ML area code only. If the former procedure is followed, both codes will be converted to machine language but only the modified ISC area code will be stored in the CHIVE Master Index File. The map catalog cards returned to ML, however, will carry the ML area code since the ML card catalog employs this system and the use of another area code would upset the existing file arrangement. Alter- natively, a conversion table may be built which would permit the computer to convert the ML area code appearing in the index records (option [b] above) to the modified ISC area code, thus obviating the need for both codes to be entered on the transcript forms by the cataloger. SYSTEM FILES Vocabulary Control Files - 99 - Approved For Release 2000/05/30: CIA-BEIERS03952A000100050001-7 Approved For Release 2000/05/3gCNTRIDP78-03952A000100050001-7 The Master Location Dictionary records will contain the following elements of information: - ISC 4-Digit Numeric Notation - Major Area, Subordinate Geographic Region, Country, or other Political Subdivision (including cross references) - Remarks (scope notes and comments on historical changes) - Place Name (including cross references) - Remarks (place name scope notes and comments on historical changes) - Geographic Coordinates Scheduled master and supplemental listings of the Master Location Dictionary will include: (a) A listing arranged hierarchically by ISC area code and containing the major area and political subdivision names together with any "Remarks" pertaining thereto. (b) A listing identical to (a) but ordered alpha- betically on area and political subdivision name. (c) A listing arranged hierarchically by ISC area code with the minor sort alphabetical by place name. This listing will also include the "Remarks" field pertaining to place names, as well as geographic coordinates. (d) A listing identical to (c) but ordered on geographic coordinates. Ad hoc (demand) products of the file will include a geo-coordinate computation capability. This program, SYSTEM FILES Vocabulary Control Files - 100 - Approved For Release 2000/0VQRRA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClgkR0T8-03952A000100050001-7 which uses a mathematical technique based on the overlap of two convex polygons, will allow an information analyst to retrieve all references to place names falling within any regular (or irregular) shaped area whose vertices are known. Subject/Commodity Authority Files Intelligence Subject Code (ISC) A modified form of the ISC classified schedule will be used in CHIVE as the dictionary authority for entry of all terms of a descriptive, semi-abstract nature whether they modify named objects or stand alone. The file is designed to perform three main functions: (a) to display relationships among these descriptive terms, (b) to define these terms when required, and (c) to serve as a code book for input to the computer. The relation- ships displayed include synonyms and alternate spellings as well as class inclusion and class membership. The file also serves an important mechanical role by requiring that every ISC code in a new index or query be present in the file before the transaction of file maintenance or searching is processed, hence controlling input errors. The file will be maintained manually, that is, all SYSTEM FILES Vocabulary Control Files - 101 - Approved For Release 2000/05/30 : CIA-WFM03952A000100050001-7 Approved For Release 2000/05/313EaAIRDP78-03952A000100050001-7 relationships and original entries will be externally controlled with changes made only by a change sheet following approval by the ISC dictionary editor. To increase the specificity of the ISC, it will be augmented by the addition of key words in clear text. Initially, the information analyst will be permitted to append these key words to any ISC code without reference to a controlled list. After a suitable length of time, however, a key word dictionary separate from the ISC classified schedule may be developed to provide guidance for consistent entry. The ISC is generally recognized as a satisfactory mechanism for indexing intelligence documents to a medium level of specificity. It is detailed enough to organize a document collection into manageable categories, but not so detailed that it is difficult to learn or apply with reasonable uniformity. The ISC is particularly strong for indexing political and socio-economic concepts. Key word indexing will be used to supplement the ISC in those areas where it is weakest, and to obtain more specificity in commodity indexing. Heavy emphasis will SYSTEM FILES Vocabulary Control Files - 102 - Approved For Release 2000/05athEA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA613g-03952A000100050001-7 be placed on key word indexing of equipment nomenclatures and model types. Key words will also be used to index scientific processes and concepts, as well as military strategy and tactics. In other fields, e.g., politics, there will be less need for key word enhancement of ISC codes. Specific revisions required of the ISC to accommodate it to an all-source document base include the following: - Reduction of the depth of subject coverage in selected areas to simplify its application. - Development of separate schedules for the classifi- cation of such subjects as organization types and personality occupational categories. (This would require the deletion of organization types which are currently scattered throughout the ISC.) - Expansion of the list of coded modifiers. - Expansion of the ISC to provide for special subject requirements unique to certain sources, e.g., photos and SI documents. The contents of a digital record in the ISC file will consist of: - ISC 6-Digit Numeric Notation - Clear-Text Term Definition - Scope Notes SYSTEM FILES Vocabulary Control Files - 103 - Approved For Release 2000/05/30 : CIA4UNN03952A000100050001-7 Approved For Release 2000/05/gcni-RDP78-03952A000100050001-7 Outputs from the file will include: - Master listings of the complete ISC dictionary arranged hierarchically by ISC code with appro- priate indentations for each lower-level category. In addition to codes and term definitions, the listing will contain pertinent scope notes. - Master listings of the subject index to the ISC arranged alphabetically by index term, including "see references." Both of these types of listings must be classified as demand products since the frequency and number of changes to the ISC vocabulary will dictate the periodicity of master re-runs. Indeed, it is likely that this vocabulary control file more than any other will be updated by page inserts rather than by re-issuance of the complete master schedule. Header Data Dictionary This file encompasses a variety of separate and distinct system dictionaries which control the entry of data pertaining primarily to the header portion of document index records. It includes such specialized system tables or dictionaries as the following: - Document Category File - Report Producing Component File SYSTEM FILES Vocabulary Control Files - 104 - Approved For Release 2000/05gfcaf-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClgiglE-78-03952A000100050001-7 - Series/Periodical Name File - Classification File - Codeword Control Stamps File - Dissemination Controls File - Photo Type File None of these files are of such magnitude that their size alone or access requirements would justify their storage in a digital medium. Nevertheless, an important goal of the system is to present information to the external customer in a language with which he is familiar. This must be accomplished even though the information is carried within the system in a different form. For this reason, wherever a convention of codes has been established for a certain type of information, and this information must be displayed to a user on output, the file must be available in digital storage and a conversion routine provided to substitute clear text for codes on output. The data content of records in all the separate authority files making up the Header Data Dictionary is identical, i.e., code (whether numeric, alphameric, or alpha) and applicable term. No machine-generated products SYSTEM FILES Vocabulary Control Files - 105 - Approved For Release 2000/05/30 : ClAiiWWW03952A000100050001-7 Approved For Release 2000/05/3gWRIDP78-03952A000100050001-7 either on a scheduled or demand basis, are presently envisaged from the file as such, although, as indicated above, the file will be machine searched to serve other system functions. 5.5.5. UNSYNTHESIZED INFORMATION FILES (UIF) No attempt will be made in this section to specify the particular Unsynthesized Information Files which will be built by information analysts in the CHIVE system. It is assumed there will be a continuing requirement for certain of the analogous information files currently being maintained (e.g., BR's International Conference and Travel File), but this is a decision best left to the information analysts within the system, working in concert with their external customers. This section merely out- lines the characteristics of Unsynthesized Information Files (UIF), explains the rationale underlying their establishment, and describes some of the methods by which such files will be constructed and maintained. As indicated in the introduction to this chapter, Unsynthesized Information Files consist of select elements of information about a given subject whether it be a SYSTEM FILES Unsynthesized Information Files 5.5.5. - 106 - Approved For Release 2000/05SECREA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA510g-03952A000100050001-7 personality, an installation, or some class of activity or event. They are to be distinguished from Special Project Files (discussed in section 5.5.7.), some of which may be information rather than document reference type files, in that they reflect only the elements of information contained in CHIVE document index records. Similarly, they are distinguishable from Summary Informa- tion Files (see section 5.5.6.) which are evaluated, concise statements of fact about similar topics. Most Unsynthesized Information Files being maintained in OCR today are the products of specialized input activity which is separate and distinct from other input processing. The principal reason for this situation is that the regular processing system (or systems) cannot readily be modified to accommodate these specialized indexing requirements, primarily because of the limitations of the supporting EAM equipment. A good example is the BR travel index which, since its inception, has been functionally and physically separate from the dossier processing system. In the CHIVE concept of document processing, however, wherein all the data of significance in the document is captured in one pass, so to speak, by the person originally SYSTEM FILES Unsynthesized Information Files 5.5.5. - 107 - Approved For Release 2000/05/30 : CIABINRE/-03952A000100050001-7 Approved For Release 2000/0566CMT-RDP78-03952A000100050001-7 assigned to index the document, the resultant product will feed both the document reference system (i.e., the Master Index File) as well as such Unsynthesized Informa- tion Files as the information analyst has decided to build. This means, obviously, that the UIF contain data no different from that stored in the Master Index, only select subsets of the same records or phrases. The reader might well ask at this point why have Unsynthesized Information Files at all if their content is identical with elements of information stored in the Master Index File? Why not simply query the Master Index when a specific set of data is desired? This brings us to the criteria for establishment of a UIF: - The customer's information requirements must be capable of definition in terms of logical data units which have specified characteristics--i.e., that there is a logical separation of data elements into related files so that any one file contains data relative to a given subject or function. - A sufficient number of requests can be anticipated on a continuing basis for the particular set of data elements contained in an information file to justify establishment of the file. SYSTEM FILES Unsynthesized Information Files 5.5.5. - 108 - Approved For Release 2000/05/ilider-RDP78-03952A000100050001-7 uni Approved For Release 2000/05/30 : Cbagsg-03952A000100050001-7 Where neither of the above conditions obtain, the data would remain in the Master Index File, and requests for the retrieval of specific elements of information would be handled like any other ad hoc queries levied on the system. On the other hand, if these requirements are met, it is generally agreed that it is useful to group the data elements involved into files, organized on a functional basis, since they can then be handled as logical elements in the system for maintenance, retrieval, and system output. The basis of organization is affected not only by the type of information to be processed, but also by the relative activity of data within a file and the user's control of the information stored in the file. Thus, in establishing an information file system, the user will probably want to functionally group his data (e.g., personality travel, leader appearances, missile site order-of-battle, directories of government officials, etc.). For particular applications the user may desire to have his stored information combined on a different basis. =Mt SYSTEM FILES Unsynthesized Information Files 5.5.5. - 109 - Approved For Release 2000/05/30 : CINEME9-03952A000100050001-7 Approved For Release 2000/05WINI-RDP78-03952A000100050001-7 Ordinarily, he would exercise this option only on the Summary Information Files discussed in the next section, since the resultant output would be an evaluated, higher-quality product. However, the system proposed will provide for a multi-file output capability from any file stored within the system. This capability will be achieved by allowing the user to query several files and selectively assemble on an output work tape the resultant information. The data will then be presented to the user in the format he specifies. For example, if the user has three files containing 25X1 B With regard to the means by which inputs to the UIF are to be obtained, it is evident that a separate indexing and transcription process is not required if the CHIVE one- time indexing concept is implemented. In other words, the plan is to exploit the document retrieval system in order to build information files. On the other hand, once the SYSTEM FILES Unsynthesized Information Files 5.5.5. - 110 - Approved For Release 2000/05ALtRtiA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CI1-W1-03952A000100050001-7 data has been put into machine readable form, the informa- tion file inputs might be derived either by (a) automatic duplication of portions of the index records during their input processing into the Master Index, or (b) by periodically querying the Master Index. Figure 5-2 illustrates these alternatives graphically. CHIVE proposes to follow the latter path for the following reasons: - It will create less of a burden on the machine processor which would otherwise have to examine every incoming record to determine if it contained data relevant to a particular information file. - There is no real requirement to update the informa- tion files at the instant that the data is entered into the machine. - By requiring some external action to be taken before data is transferred to an information file, management control is enhanced. To facilitate file building, standing queries will be written, punched, and entered into the system. Thus, when the information analyst wishes to add new data to an information file, he can merely call for the pertinent query by name and the computer will make the necessary search and load the data into the relevant file. The content of records in a UIF may consist only of a specified set of fixed elements of information which SYSTEM FILES Unsynthesized Information Files 5.5.5. Approved For Release 2000/05/30 : CIASIffeRM03952A000100050001-7 Approved For Release 2000/05/30: CIA-RDP78-03952A000100050001-7 Index Record Figure 5-2 UIF FUN BUILDING _ALTERNATIVES Index Record Processor DP Processor Master Index File Unsyn. Info. Files Master Index Human-triggered Info. File Building - 112 - Approved For Release 2000/05/30: CIA-RDP78-03952A000100050001-7 25X1B 25X1B 25X1B Approved For Release 2000/05/30 : CgkBPT8-03952A000100050001-7 appear once in each data record or a combination of fixed and repetitive elements of information including some fields of variable length. For example, in a file elements in a record contained in this file might have a fixed number of characters or, alternatively, some may be fixed while others (e.g., "function attended") may be fields of variable length. Similarly, in the case of a travel file Document references, while not vital to information files which ordinarily do not require consultation of the documents from which the data was originally extracted, can nevertheless be included in UIF records where required by citing the pertinent document control number. Similarly, the security classification of a record in such SYSTEM FILES Unsynthesized Information Files 5.5.5. - 113 - Approved For Release 2000/05/30 : CIAMS76103952A000100050001-7 Approved For Release 2000/05/3g?KTRDP78-03952A000100050001-7 files is easily denoted since the record should typically carry the same classification as the parent record in the Master Index. In both cases, the transferral of the document control number and security classification from the header portion of a Master Index Record to a record in a UIF can be accomplished automatically at the same time that the content data is duplicated for storage in a UIF record. Provision will be made to automatically identify those records in the Master Document Index whose content has been extracted for a given information file so that the same records need not be searched later. The system will also allow the analyst to specify both an "active" and a "history" file of information for any one functional area if it seems desirable to save the digital records representing a file for some indefinite period of time. Outputs from the UIF will largely consist of periodic listings of the complete contents of a UIF arranged in various sequences. Such listings will be used to service customers as well as information analysts within CHIVE who will analyze and modify the listed records which will then be converted back to machine language and input to SYSTEM FILES Unsynthesized Information Files 5.5.5. - 114 - Approved For Release 2000/0MAZREIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : &&9?:F78-03952A000100050001-7 Summary Information Files. Some of these listings will no doubt be required on a regularly scheduled basis, e.g., a leader appearance listing published weekly. Others will be demand products issued irregularly as the need arises. In addition to the provision of hard-copy machine listings of digital UIF meeting the the records an entire file for browsing purposes, a file itself can be queried for records request specifications, in which case only satisfying the request will be output. This mode of man/file interface will probably be used less than hard-copy browsing; but, unlike the CHIVE Vocabulary Control Files discussed in section 5.5.4., the means by which the UIF records will be made available to the human in the system will not be exclusively through a hard-copy representation of the file contents. 5.5.6. SUMMARY INFORMATION FILES (SIF) Summary Information Files (SIF), like the Unsynthesized Information Files, can be classed as formatted in nature since their specifications can be pre-defined and the data elements making up their content can be handled as logical SYSTEM FILES Summary Information Files 5.5.6. - 115 - Approved For Release 2000/05/30 : CIA-BERRIE-D3952A000100050001-7 Approved For Release 2000/05/4d&RDP78-03952A000100050001-7 entities in the system for purposes of input, query, and output processing. Pertinent input data is organized by major subject and formatted for ready retrieval and tabulation by content. Summary Information Files consist of semi-evaluated data relative to specific classes of events or named objects. In format and content they are indistinguishable from UIF files, differing only in the fact that redundant, and, usually, contradictory information has been removed from the SIF files through a process of human analysis and synthesis of the raw data originally received. While they cannot be accurately described as containing only "finished intelligence" (if, by definition, this term is meant to apply only to the refined outputs of an intelli- gence research facility), neither is their content truly "raw" and, for this reason, the expression "semi-evaluated" has been used advisedly. Certain of the Vocabulary Control Files which must exceed the boundaries of a typical dictionary in order to adequately "identify" a controlled term can also be properly classified (as pointed out in the introduction to this chapter) as Summary Information Files. These SYSTEM FILES Summary Information Files 5.5.6. - llt Approved For Release 2000/05/315E.WRDP78-03952A000100050001-7 ONO mmw 25X1B 25X1B sviro Approved For Release 2000/05/30 : Clagg-03952A000100050001-7 include the personality and organization/facility identifier files discussed in sections and, respec- tively. Not only are these formatted files whose content is fixed and specific, they contain a variety of summarized, semi-evaluated facts about named objects which, when displayed, can serve a variety of information needs other than that of supporting the document indexer. An example of an SIP would be a Officials File. In reality, a type of organization summary file, since the stored data would consist of a set of facts about significant public and private organs of society within the country concerned and not summary data about personalities as such, the file entries might contain in tabular form information on the name of an organization, its subordination, the names of its officers and perhaps ordinary members of the organization, their individual position titles, and the dates of appointment and/or earliest and latest dates of identification for each. The data would be retrievable on the basis of any of the categories which make up the file so that answers to such questions as: SYSTEM FILES Summary Information Files 5.5.6. - 117 - Approved For Release 2000/05/30 : CIASEERW03952A000100050001-7 25X6 Approved For Release 2000/05/gqff-RDP78-03952A000100050001-7 25X1B 25X1B can be readily obtained. Summary Information Files can, of course, include fully automatic, semi-automatic, and manual data files ranging from the highly structured to the unformatted, narrative type. FIB's Installation Summaries File and the various types of biographic reports on file in BR are examples of manual summary information files which are only partially formatted if at all. BR's Who's Who Card File is a formatted, semi-automated summary file on personalities. The CHIVE system will also produce and maintain manual files of summary information, but the focus of discussion in this section is on the digital, and not hard-copy, summary files planned for the system. The criteria dictating the establishment and maintenance of an SIF file are identical to those for a UIF file, i.e., the set of data elements making up the file must be definable and a "substantial" number of requests for such data must be anticipated. The means by which input and maintenance transactions will be obtained, however, will differ from that of the UIF files. SYSTEM FILES Summary Information Files 5.5.6. - 118 - Approved For Release 2000/05At9kpek-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAKLW8-03952A000100050001-7 Unlike UIF processing, additions to and modifications of SIF files cannot be automatically generated since human judgment is required, if not to identify potentially relevant inputs, to make the conclusive determination that a record should be added to a file or a change made to an existing record. The process by which these functions will be performed will vary depending on the nature of a particular SIF file. Figure 5-3 depicts the various approaches which can be used. As indicated in the figure, the SIF file builder has essentially three options open to him as the means of inputting data to a Summary Information File: (a) he may query the basic index records to documents for data pertinent to an established SIF file (Option 1), (b) he may obtain a printout of a UIF file which serves as the raw data base for an associated SIF file (Option 2), or he may arrange with other information analysts to have all documents containing information pertinent to his needs routed to him (Option 3).* *A fourth option, which has not been suggested because it would lead to completely duplicative document handling, would require the SIF specialist to examine all incoming documents for their possible relevance to a summary data file. Such an approach would never be required as long as the initial document indexing was sufficiently specific to enable the information file specialist to recover the pertinent index records and/or documents by a search of the Master Index. SYSTEM FILES - 119 - Summary Information :files Approved For Release 2000/05/30 : CIA-BWM339%2A060100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Figure 5-3 SIF FILE BUILDING ALTERNATIVES 2ption 1 Search Request from SIF Specialist Index Record Listing Select Re- cords or hrases) Doc 't Image File Option 2 Search Request from SIF Specialist V Listing of UIF Records 14-- Requests for Select Doctts SIF Specialist Review Additions or Changes to SIF SIF File Option 3 Original Docits Screened b nfo Analys Select Doctts SIF Specialist Review - 120 - Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Additional Routine Input Processing 4 _ J Approved For Release 2000/05/30 : ClAg611V-03952A000100050001-7 Options 1 and 2 both provide for the possibility that the SIF specialist may wish to examine certain documents for himself to clarify some fact reported in the records listed for him as a result of his inquiry. It is anticipated, however, that in most instances the listed records will speak for themselves and no reference to documents will be required in the SIF input process. SIF files will be available in both digital and hard-copy (listing) form, and, like the UIF, can include active as well as history files. Source citations in the form of document control numbers may be listed at the end of each summary information record or referenced to each term (element of information) in the record. 5.5.7. SPECIAL PROJECT FILES In the remarks made earlier in this chapter with regard to Special Project Files, it was noted that the limits of CHIVE responsibility for special file building, maintenance, and output processing have not as yet been satisfactorily determined. Furthermore, in a survey of existing OCR files which were either exclusively or in 25X6 part (see Appendix 5.D.), no files were identified which in the CHIVE context would be considered SYSTEM FILES Special Project Files 5.5.7. - 121 - Approved For Release 2000/05/30 : CIAMBEa03952A000100050001-7 Approved For Release 2000/0545CM-RDP78-03952A000100050001-7 as "special project" in nature. For both of these reasons, therefore, it is difficult to specify the characteristics of individual Special Project Files which might be included either in the initial or final CHIVE system. One must anticipate, however, that the need for such special files will be expressed by customers from time to time, and some remarks may be in order as to the features which distinguish these files from other system files and how requirements for such files might be accommodated. Processing demands of a one-time nature which necessitate the input, manipulation, and retrieval of a peculiar set of data will not fall within the Special Project File category since these will be handled like any other request. However, if the system were asked to continue the activity on an indefinite basis, this method of handling would no longer suffice and a special project need would have been established. Special projects will include any files obtained from organizations external to CHIVE which cannot be fully integrated with equivalent CHIVE files and which require machine handling. In this sense the term "special projects" could apply to certain EAM files inherited from OCR, as well as to files acquired from other agencies. SYSTEM FILES Special Project Files 5.5.7. - 122 - Approved For Release 2000/05RTIVRDP78-03952A000100050001-7 mow 25X1 B ow. tonal NNW Approved For Release 2000/05/30 : MRIg78-03952A000100050001-7 Special Project Files will also describe any customer input requirements which cannot be satisfactorily handled by the established system for representing the informa- tion content of documents. The data involved might be largely numeric in character or, if non-numeric, would require the extraction of items of information not planned for inclusion in the system and which, if accepted, would significantly add to total processing time. It is well known that individual members and groups within the CIA customer population have a number of relatively unique and distinct information handling problems which cannot be met by generalized information system attempting to serve only the common interests of the many. Examples of such requirements were uncovered during the earlier fact-finding survey of the DD/I. The following is but a partial list of some of the needs expressed: SYSTEM FILES Special Project Files 5.5.7. - 123 - Approved For Release 2000/05/30 : CSE-eaPT8-03952A000100050001-7 25X1B Approved For Release 2000/05/455cM-RDP78-03952A000100050001-7 In the past when a research analyst, or group of analysts, developed an information control problem which was not being satisfactorily handled either in the appropriate production office or by the central reference system, one of the following courses of action was generally adopted: - The problem was contracted out to some external organization which obtained the necessary source documents and did the input processing and retrieval 25X1A required to respond to the identified need. Examples of this approach include OSI's Project 25X1A and LSD/SI's - A special group was set up within one of the research offices to perform either or both the input and data manipulation functions depending on what was required. The of ORR is a good illustration of this type of problem solution. 25X1A - OCR was asked to expand its operations either by increasing the depth of its indexing or by broadening its document coverage, or both. OCR projects generated by a demand for increasing indexing depth beyond that normally provided by the basic indexing SYSTEM FILES Special Project Files 5.5.7. 124 Approved For Release 2000/00KRETA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAGRg-03952A000100050001-7 systems employed include the SR Test Range Activity File and the BR Travel File. Illustra- tions of projects which required expansion of the OCR document base can be found in the the Library's Science Information Service (SIS), 25X1A the SR PI Reports File, the etc. 25X1B - Mechanical, if not data extraction, assistance was requested either of OCR's Machine Division or of some external machine facility (governmental or private) where manual techniques for data manipula- tion and display were unsatisfactory. Examples File for OBI. Today, the resources available to an analyst faced with an unresolved information processing requirement are much the same--but with one significant difference. If he is willing to prepare the input data of interest to him to the point at least where it can be transcribed into machine-recognizable form, he now has a powerful machine capability available to him to perform a host of operations on the data. The CHIVE EDP System can, of course, increase this capability. The question, however, which remains is whether CHIVE should get involved in "special-project" applications at all, where only a limited set of customer interests are served, and, if so, whether its involvement should be restricted to the provision of EDP support or whether it should also assist in input preparation. - 125 - SYSTEM FILES Special Project Files 5.5.7. Approved For Release 2000/05/30 : CIA-Wlz7L8r03952A000100050001-7 25X1A Approved For Release 2000/05/WW-RDP78-03952A000100050001-7 Currently, the OCS/Applications Division is per- forming a major role in supporting special project requirements of CIA research analysts. In all such projects, however, the data extraction responsibility has been assumed by the customer concerned. It might be argued that to avoid confusion of responsibility, the role CHIVE should play in this area (if any) should be restricted to the assumption of the responsibility for those special projects where, for one reason or another, it seems most efficient to have the central reference organization, rather than research analysts, prepare the input data. This, however, heightens the risk of gradually proliferating the informa- tion processing responsibilities of the CHIVE system to the point where it might become simply a collection of special projects. In the design concept presented here, the responsi- bility for CHIVE's undertaking certain special projects has been accepted, but the duty of preparing the input to Special Project Files is assumed to be the customer's and the report reflects this philosophy. In the final analysis, however, the matter can only be resolved by SYSTEM FILES Special Project Files 5.5.7. - 126 - Approved For Release 2000/05giltalp-RDP78-03952A000100050001-7 owl god Approved For Release 2000/05/30 : Clk-W8-03952A000100050001-7 management decision. If the choice is to include special projects within CHIVE, including the function of data preparation, the approach suggested above may provide a modus vivendi for relating the respective roles to be played by CHIVE and OCS/Applications in the handling of these projects. 5.5.8. REFERRAL SERVICE FILES Current manpower ceilings seriously limit the cover- age of present central system operations. Even if available manpower can somehow be more effectively utilized, the volume of material of potential value is so great that complete coverage would still not be possible. Thus, the only alternative appears to be to develop support from other systems, including centralized as well as personalized (analyst-driven) file activities. To do this, CHIVE must identify information resources available in such systems and determine how best to tap these resources for the Agency consumer. In earlier CHIVE documentation, reference was made to a "support mode" which envisaged not only the referral of customers to persons or files of possible interest SYSTEM FILES Referral Service Files 5.5.8. - 127 - Approved For Release 2000/05/30 : CIA-SIDERET03952A000100050001-7 Approved For Release 2000/05/3g.QUIRDP78-03952A000100050001-7 external to CHIVE, but the actual acquisition of certain machine-language files and supporting documentation which would be searched in-house in behalf of CHIVE customers. The "support mode" concept remains valid in present CHIVE thinking, but in the further refinement of the design a distinction has been made between: (a) files outside of CHIVE's control which are actually available within the system in either manual or mechanized form, and (b) information resources not directly accessible to CHIVE to which customers may be referred. The former have been classified in this report as "Supplemental Files" (see section 5.5.1.), i.e., files neither built by CHIVE nor inherited from OCR, of which Special Project Files may be one class. The latter are now termed "Referral Service Files" and are the subject of this section. The community's information resources are so vast and scattered that even the simple identification of all potential sources constitutes a major problem. For this reason, it is planned to concentrate initially on the identification of the many different human file resources scattered amongst the various service components and production shops within this Agency. Only after this SYSTEM FILES Referral Service Files 5.5.8. 128 Approved For Release 2000/05altRVt\-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIRW8-03952A0001000500017 is done will any attempt be made to obtain descriptions of information resources and repositories in other USIB components. Perhaps the simplest means of beginning to build a referral service capability will be to derive a set of analyst profiles from requests levied against the central system. Using this technique, search terms chosen for query purposes will gradually form the set of subject identifiers descriptive of each analyst's interests. This approach will be supplemented by the circulation of questionnaires to analysts throughout the research (and select service) components of the Agency, which would solicit narrative statements of their areas of substantive knowledgeability, including their personal files or files maintained by their respective sections, branches, or other organizational component. Not all analysts, of course, can be expected to respond. However, experience with similar surveys in other organizations suggests that a response figure of about 80% is not beyond reason. The returned questionnaires will be indexed in the vocabulary of the CHIVE system, including both the ISC classified schedule as well as key words. These descriptors SYSTEM FILES Referral Service Files 5.5.8. - 129 - Approved For Release 2000/05/30 : CIA-Fe5M63952A000100050001-7 Approved For Release 2000/05/36ECRERDP78-03952A000100050001-7 will not necessarily be limited to conceptual-type subjects (although the emphasis will probably be on these kinds of topics) but will, in all probability, also contain on occasion named-object identifiers such as the names of persons, organizations, military installations, etc. It is not anticipated that the responses will include every specific subject heading in an analyst file. However, it is hoped that the principal categories of information contained in such files will be described and this alone would greatly assist analysts in seeking to exploit the Agency's human and documentary resources. In addition to storing information descriptive of the subject matter in which a person or file specializes the data which will be contained in these referral service records will include: - Name of Individual - Organization Identification (component to which analyst or file is attached) - Address of Individual or File (room and phone number) - Descriptive Title of File - Overall Security Classification of File SYSTEM FILES Referral Service Files 5.5.8. - 130 - Approved For Release 2000/0%.3&FRik-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: CIMIrt8P8-03952A000100050001-7 - Releasability - Countries or Geographical Person or File Areas Covered by the - Primary Intelligence Activity Supported by the Person or File (e.g., Missile Photography, Intelligence, etc. OB, Ground - File Storage Medium (documents, 5" x 8" cards, EAM cards, magnetic tape, etc.) Assuming the cooperation of a reasonable number of 25X1B analysts, it is likely that the collected records will be sufficiently voluminous and file order requirements so varied that a machine data base will be needed. It is not contemplated, however, that the Referral Service mr Files will be automatically searched at the time queries are levied against the substantive data files of the CHIVE system. Rather the content of such files will be made moo available in the form of a published Directory of Informa- tion Resources. This Directory would be issued to CHIVE system operators and perhaps, in a variety of classifica- tions, selectively disseminated to Agency consumers. When specifically requested to do so by a customer, CHIVE personnel, in addition to searching the basic files of the CHIVE system, will consult the Directory for the demi SYSTEM FILES Referral Service Files 5.5.8. - 131 - Approved For Release 2000/05/30 : CIA-FallER03952A000100050001-7 Approved For Release 2000/05/36EM1RDP78-03952A000100050001-7 purpose of determining what other files or intelligence analysts might possess information pertinent to a given query. The type of service that will be provided, if and when a potentially relevant resource is uncovered, will vary depending on such factors as the location of the file or the urgency of the request. In some instances, CHIVE information analysts will act as the intermediary between the customer and the other informa- tion resource. In other cases, they will simply refer him to the appropriate system. With regard to the provision of a referral service capability for files outside of CIA, advantage might be taken of efforts currently being sponsored both by DIA and by CODIB to collect descriptions of intelligence data files maintained in an automated form by Department of Defense elements and USIB member agencies, respectively. It is planned that a catalog of such files will be published periodically and may be interrogated on an ad hoc basis. If these external collection programs prove successful, the data resulting therefrom might be merged with the product of the internal file survey to form a relatively comprehensive record of information resources throughout the Community. SYSTEM FILES Referral Service Files 5.5.8. - 132 - Approved For Release 2000/058aRtIt4-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIWP8-03952A000100050001-7 5.5.9. MANAGEMENT DATA FILE The CHIVE Management Data File will contain two types of data: - Data, obtained by computer methods, about the processes performed by the EDP portion of the CHIVE system. - Data, obtained by manual methods, about the non-computer processes of the CHIVE system. The following paragraphs discuss the sources and collection methodologies for these two types of data, Imo the reasons for the dichotomy, and the use of this data to operational management. and mast Collection Techniques As indicated above, the method employed in collecting the data (EDP data or manual data) determines the origin of the data and to a large extent the use of the data by CHIVE managers. EDP data collection refers to an activity within the computer itself. The monitor program system (with its attendant bookkeeping functions) will supervise all computer operations. This is implied under the philosophy of a multiprogramming system. This method of operating provides a natural means of recording: SYSTEM FILES Management Data File 133 Approved For Release 2000/05/30: CIA-KIBUREg3952A000100050001-7 Approved For Release 2000/05/36EMERDP78-03952A000100050001-7 (a) Process times. The computer has timing mechanisms which the monitor can use to record computation, input, and output times as indi- vidual entities, as well as the total time the computer uses to process a transaction. (b) Error rates and types. A variety of errors and malfunctions may abort an operation or degrade the output. Certain of these, e.g., misuse of the language, transcription errors, equipment disorders, and illegal file manipula- tions, may be more readily detected and recorded by the computer programs than by manual means. (c) File activity. It is of significant importance to determine which files or parts of files experience a high rate of use. File system design, program system design, and language structure are just a few of the areas which affect the use of the files and are, in turn, influenced by usage statistics. This dynamic data may be supplemented by such relatively static data as: - Equipment availability - Day, month, and year - Priority of the transaction As this data is recorded (either by the bookkeeping routines within the monitor or by specially produced CHIVE programs as adjuncts to the monitor) it should be entered into a file. This file is essentially a log of CHIVE EDP transactions and their associated management data. The normal method of labelling entries in such a SYSTEM FILES Management Data File - 134 - Approved For Release 2000/05ftibeft-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clk-W8-03952A000100050001-7 file is by "job" or transaction number. Under the multiprogramming mode of computer operations, it is mandatory that each "job" which enters the computer be uniquely identified. This "job" or transaction number provides a natural storage and retrieval device. Manual data collection refers to that activity outside the domain of the computer which collects management data about the processing of transactions. As presently envisioned, the process will begin when a transaction is initiated and will end when the transaction is completed. For example, a query against a file is a transaction which begins with the request and ends when the requester obtains the data and materials which satisfy his request. Between these two events many functions are performed in many organizational elements. The majority of the time-consuming and error-prone functions are performed by people. Data regarding these functions may be conveniently collected by manual techniques. It is suggested that data regarding each transaction accompany the transaction during the entire process. If feasible, a standard form should be used. Examples of manual data are as follows: SYSTEM FILES Management Data File - 135 - Approved For Release 2000/05/30 : CIA-RDEvai0B952A000100050001-7 Approved For Release 2000/05/3gWIRDP78-03952A000100050001-7 - Name of Requester - Requester's Organization - Name of Analyst - Analyst's Organization - Type of Transaction - Transaction Number - Dissemination Code - Time Received and Time Released (by each organiza- tional unit which handles or is responsible for the transaction) - Organizational Identifier (for each component which handles or is responsible for the transaction) Not all of these are applicable to each transaction. However, the last two items--times and organizations-- must be supplied for each component and each transaction for two reasons: (a) To account for each transaction and its location in the system. (b) To provide a complete file of data for process evaluation. Storage, Retrieval, and Processing The EDP data, due to the collection method, is naturally stored as a file of data within the CHIVE EDP system. As such, it can be processed and retrieved through SYSTEM FILES Management Data File - 136 - Approved For Release 2000/05AtiMA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA?IW-03952A000100050001-7 the use of the CHIVE query language. It is suggested that initially no language capabilities be added for this specific purpose since the number and nature of reports on machine processes which CHIVE management will require is not completely predictable. The manual data collected in the system should initially be stored, retrieved, and processed by manual methods. As the system is used and evaluated, the file of manual data and the number of management reports will increase. At some point, this data must be processed by the EDP part of the CHIVE system. For this reason, it is important to design the manual data forms so that, as volume increases and operational procedures become firm, the data may readily be input to the computer and integrated into the EDP management data file. When this point in system evolution is reached, all manually collected data regarding CHIVE operations will be retrieved and stored by initiating a transaction. Thus, data about the processing of the Management Data File is recorded in the Management Data File and constitutes a resource which management may use to study its own evaluative and analytic activities. SYSTEM FILES Management Data File - 137 - Approved For Release 2000/05/30 : CIA-MIM63952A000100050001-7 Approved For Release 2000/05/3gMhDP78-03952A000100050001-7 Reports and Their Use In discussing reports and their content, a distinction should be made as to when, during the evolution of the system, the reports are needed. This is particularly true in the case of those reports drawn from the EDP management data file prior to the incorporation of the manual data. The purpose of reports based on data collected by EDP methods is to assist the CHIVE analysts and designers in improving, correcting, and modifying the EDP portion of the system. During the initial stages of operational testing t will be necessary to examine EDP operations carefully in order to eliminate bottlenecks and optimize equipment usage. Certain reports will be highly specialized, e.g., an analysis of disk storage use over some period of time, and will not be necessary as a regular product. Of continuing interest will be reports which provide management with an insight into the amount of time used on the computer and its various components. This has long-range implications regarding computer hardware acquisition. Reports derived from the manually collected data will vary in frequency and detail as the system gains operational SYSTEM FILES Management Data File - 138 - Approved For Release 2000/056tkW-RDP78-03952A000100050001-7 SECRET Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 acceptance. In any new system, there will be imbalances which must be adjusted if the best results are to be obtained from the available personnel and equipment. The parameters which can be measured within the system are primarily concerned with rates and volumes. It is suggested that forms be designed and procedures instituted which will provide managers with raw data on how long a transaction stays in each component. This is the first step toward the elimination of delay points in the system. Shifting of manpower and new procedures will undoubtedly be necessary. This in turn will prompt another round of reports and analysis. And so on. Of interest to manage- ment in terms of long-range changes to the system will be reports on sources and types of transactions. Such reports are generated by the present system and will be produced by CHIVE. Data on the number of cards produced, number of file accessions, number of references generated, and number of pages delivered will also provide managers with the necessary background for making adjustments in the processing of transactions. After all management data has been combined in the file maintained by the EDP system, reports can be SYSTEM FILES Management Data File - 139 - Approved For Release 2000/05/30: CIA-14Mq3952A000100050001-7 Approved For Release 2000/05/3C6ECRERDP78-03952A000100050001-7 generated on a regular or demand basis with much less expenditure of manpower. The nature of the reports will probably vary little after the shakedown period is completed. However, the volume of data which must be manipulated dictates an EDP mode of report generation. SYSTEM FILES Management Data File - 140 - Approved For Release 2000/056Lt.a4!1-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA98-03952A000100050001-7 Chapter 5.6. SYSTEM FLOWS AND TRANSACTIONS This chapter provides a more detailed view of system flows and transactions, i.e., the more dynamic aspects of the data processing activity, including some descriptions of illustrative tasks. The document image storage and delivery portion of the system is covered in outline only, leaving the more definitive treatment of this subject to Volume VI. Similarly, only passing mention is made of the EDP design since it is fully discussed in Volume VII. 5.6.1. DOCUMENT INPUT Referring to Figure 5-4, the input to the system will be described. The principal categories of incoming documents will consist of (a) textual-type documents received in all source classifica- tions ranging from Unclassified to T/KH, (b) select documents (principally SI Teletype) SYSTEM FLOWS Document Input 5.6.1. - 141 - Approved For Release 2000/05/30: CIA-MMQ3952A000100050001-7 25X1B Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cli&-Rig-03952A000100050001-7 received in machine language (as well as hard copy), (c) graphic images in the form of ground photography and films, (d) maps, and (e) machine language (ML) index records prepared by external organizations according to CHIVE rules and formats. Graphics and maps will continue to flow to GR and the Map Library Division (ML) through their existing acquisition channels. The only significant change in their oper- ations will be that they will employ the CHIVE vocabu- lary in their indexing or cataloguing operations, and will transmit a copy of their index transcript sheets to CHIVE for conversion into machine readable form and entry into the Master Index File. CHIVE in turn will return to them a printed version of their index records for entry into their manual files where this seems desirable. Documents selected by the information analyst which are available in machine language and have a formatted header and title (e.g., SI Teletype) will bypass indexing and transcription steps and go, in their machine language versions, directly to the EDP SYSTEM FLOWS Document Input 5.6.1, - 143 - Approved For Release 2000/05/30 : CliSELSBM-03952A000100050001-7 Approved For Release 2000/05/SMWRDP78-03952A000100050001-7 System where the necessary conversion to CHIVE format will be performed. The hard copy versions of the documents will be sent simultaneously to microfilming for processinj into the microimage store (Master ImaIe File) Other machine language receipts, consisting of abstracts of foreign scientific and technical literature, bibliographic records, and formatted in- formation extracts pertaining to named-object data appearing in open sources, may likewise be input directly to the EDP System. Printed versions of these receipts, however, may be passed to information analysts within the system who will thereby be afford- ed the opportunity to review their content, and, if desired, delete the corresponding machine record from the EDP file. Since the source documents will not accompany these ML inputs, no photoprocessing will be required. The remainder of this section will deal with the principal input flow process depicted in Figure 5-4, i.e., that relating to all-source textual documents. SYSTEM FLOWS Document Input 5.6.1. - 144 - Approved For Release 2000/05/3?EatiRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : 1Ig78-03952A000100050001-7 Upon their receipt in the mail room, these documents will be counted, batched by type, and assigned document control numbers where required. The batches will then be forwarded to a dessemination unit where the documents will be disseminated to other offices as well as to CHIVE. Documents to be dis- tributed to CHIVE will be divided into two categories; (a) reports for which CHIVE has a repository responsi- bility, and, therefore, must be kept regardless of substantive content (hereafter referred to as "R" documents); and (b) non-repository ("NR") documents whose retention value can only he determined after examination by an experienced intelligence information analyst. "R" documents (constituting the vast majority of incoming receipts) will be addressed to the appro- priate CHIVE subcomponent, but will flow initially to a centralized Header Indexing Group which will index the bibliographic data on the documents. ,Once this operation is completed, the documents will be trans- SYSTEM FLOWS Document Input 5.6.1. - 145 - Approved For Release 2000/05/30 : CSUM1778-03952A000100050001-7 Approved For Release 2000/05/SECM-RDP78-03952A000100050001-7 paitee(1 directly to the Document Delivery System for image processing into the Document Image File, while the header index would be sent to the EDP System for conversion to machine language. The "R" documents whLob, in all probability, would be the ones most Then re uested v Agency- customers in the period aebiately following their receipt, will (by this process) find their way quickly into the document store where they will be available for retrieval. Following image processing, they will be forwarded to the CHIVE analytical desks marked on the documents Lor content review and indexing where warranted. NR" documents will bypass the centralized Header indexing Group, being forwarded by the Dissemination Unit directly to the analytical components within: the CHIVE geographic divisions. Hore a further redistribution of some of the "R" is well as ?NR" documents might take placeif the iLLial dissemination was not sufficiently precise. Lo any event, the:: ultimate recipient if both types SYSTEM FLOWS Document Input - 146 - Approved For Release 2000/05/geater-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClWEREIT8-03952A000100050001-7 of documents will be an information analyst specializing in an area or topic within area. His responsibility, relative to the "R" documents, will be to determine whether content indexing is warranted in addition to the header indexing already performed. If not, he will destroy the documents and send a notice to the EDP System that no content index will be forthcoming. If the documents, however, do warrant content indexing, he will mark the parts of the documents which he wants reflected in the index, and will pass the marked docu- ments to a Content Indexing Group serving his Division. (The activity of these individuals is described below.) "NR" documents will likewise be examined by infor- mation analysts and will either be destroyed or marked for some form of indexing. If indexing is required, the documents will be sent first to header indexing clerks functioningat the division or desk level. They will prepare header transcript sheets, like their counterparts in the centralized Header Indexing Group. Where content indexing is not required but storage is SYSTEM FLOWS Document Input 5.6.1. - 147 - Approved For Release 2000/05/30 : CIASECRET-03952A000100050001-7 Approved For Release 2000/05/36EMTRDP78-03952A000100050001-7 desired, the "NR" documents will be sent to the Document Delivery System for microfilming, while their corresponding header transcript sheets will be passed to the EDP System. The remaining "NR" docu- ments (and transcript sheets) which were to be content indexed will be forwarded to the Division's Content Indexing Group where they will rejoin the select "R" documents discussed above. In the Content Indexing Group, semi-professionals known as content indexers will prepare content data transcript sheets by extracting and formatting the data identified for them by the information analysts. A selected portion of this work will be inspected and revised if necessary. Corrections and changes will be written on the data sheets. Once the content data transcript sheets have been prepared, the marked-up copies of the "R" documents can be destroyed since an image of these will already be available in the Document Delivery System. The indexed "NR" documents, however, will be forwarded SYSTEM FLOWS Document Input 5.6.1. - 148 - Approved For Release 2000/05/39EatfRDP78-03952A000100050001-7 IMO awl omit IMMO Approved For Release 2000/05/30 : Cl00171-03952A000100050001-7 to the Document Delivery System for processing into the Master Image File. Content data transcript sheets for both "R" and "NRudocuments will be sent to a Data Transcription Group where they will be copied by typists.* The typed index entries, after sight verification, will then be fed to the EDP System for machine processing. Within the EDP Subsystem, a Page Reader will convert the clear-text header and content indexes into machine language. Following this operation, punched Work Cards will be generated by the computer from a portion of the header data record which will be used in the Document Delivery System (see below) in the preparation of the microimage store. The complete digitalized records of the header and content indexes will be processed by computer programs which will check the records for format and certain types of content errors and add them to the pertinent system files. *Header data sheets can presumably be typed by the header indexers who prepared them. SYSTEM FLOWS . Document Input - 14 9 - Approved For Release 2000/05/30 : CIASIRCRE13-03952A000100050001-7 Approved For Release 2000/05/fMURDP78-03952A000100050001-7 In the Document Delivery System, documents to be kept in hard copy for reasons of length, image quality, or other will be shelf-filed in an area contiguous to the microimage file according to their meaningful document control numbers. The remaining documents will be routed to a microfilm section. There they will be photographed, and, assuming the storage medium selected is the 35 mm. aperture card; the resultant product will be an aperture card with the document batch and serial numbers eye-visible in the aperture. After these numbers are punched into the aperture cards, the aperture cards will be mechani- cally collated on these numbers with the deck of Work Cards prepared by the computer from the header data records to the same documents. Following collation, other data punched in the Work Cards will be reproduced and interpreted for the Vital Materials Repository (VMR) and NSA as appropriate. Lastly, a master set of the cards will be filed in document control number sequence in the Master Image File. SYSTEM FLOWS Document Input 5.6.1. - 150 - Approved For Release 2000/05/4ECIMRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CligreF8-03952A000100050001-7 5.6.2. DOCUMENT RETRIEVAL Referring now to Figure 5-5, the recovery of information from the files will be discussed. The retrieval process will ordinarily begin with a customer external to CHIVE originating a request for data ether on a form designed for this purpose, by lepnone contact, or by personal visit to the system. He will be put in touch with an information analyst working on the geographic/topical area of concern. The information analyst will be familiar with the current reporting, having screened incoming documents to determine what should be indexed, and will also have had extensive training in the indexing vocabulary, the logical files available within the system, and the query language required to conduct the computer search. After ascertaining the clearance level of the customer, the degree of sensitivity desired in the search, and the heterogeneity of the document base to be explored (e.g., "search document and photo SYSTEM FLOWS Document Retrieval 5.6.2. - 1 1 - Approved For Release 2000/05/30 : CIPMREZ-03952A000100050001-7 25X1B Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cl/frarr78-03952A000100050001-7 indexes, but not maps or films"), the information analyst (assuming a machine search is required) will translate the request into a set of commands using the formal language developed by CHIVE (see section 7.A,). To prepare the necessary search criteria he will consult the various Vocabulary Control Files-- e.g., MOFIF, ISC, etc.--in order to derive the proper terms on which the search should be conducted. This research might also reveal whether certain inherited files would be worth interrogating (see section Having determined what descriptors to employ in the search, he will obtain a request number from a central control point and proceed to fill out an inter-leaved set of request forms on which he will identify himself (as well as his customer) by name and address, cite the file (s) to be interrogated, detail the logic and priority of the search, and define the output format required. One copy of his request statement will then be sent to thereuest control point to be added to the file of open requests. Assuming, however, that some inherited files must also be searched since the SYSTEM FLOWS Document Retrieval 5.6.2. - 153 - Approved For Release 2000/05/30 : CIASBOREI-03952A000100050001-7 Approved For Release 2000/05/AE.CERRTRDP78-03952A000100050001-7 date span of the -request encompassed the ?period prier to the initiation of the CHIVE system, the information analyst may be required to take one or more of the following additional steps: a. Eeeemine hard copy files of cards or docu- ments co-located with his organization component. Reeeest the retrieval of hard copy records (c.e., AIRA, one-name cards, etc.) from the system's centrally-located, master document cellection. c. Consult uith other information analysts familiar with the contents, vocabularies, and record formats of machine files in- heritedeby CHIVE and obtain their assistance (here rcuired) in preparing the special request forms to interrogate said files. The formulated machine requests will be typed en,-, sight verified, and than transmitted to the Page .e.ealee via the pneumatic tube system. For those requests to be passed against the EDP files, the eomputer will check for such things as the complete- ness of the recruest statement and validation of the terms composing the query. All requests will then Le queued for processing against the pertinent SYSTEM FLOWS Document Retrieval 3.5.2. - 154 - Approved For Release 2000/05/4tderRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClAS-ER-03952A000100050001-7 inherited and. CHIVE-built files.* Searches of unconverted EM files will be con- ducted as at present, with the output taking the form of existing machine listings which cite documents, personality dossiers, installation num- bers, or photo accession numbers relevant to the request. For files converted to EDP and the CHIVE- built Master Index File, the product of the search will also be a listing, albeit in a different form. On the first page(s) of the listing will appear the identity of the information analyst levying the request, the request itself, and the list of docu- ment control nuMbers which satisfied the search criteria. On succeeding pages will appear, depend- ing on the output format requested, either the complete "hit" index records or select elements thereof. (Output of a statistical count of the number of documents which matched the search *The periodicity of searches may differ between these files, i.e., inherited files may customarily be searched only once a day while the CHIVE-built files will be searched on a demand basis. SYSTEM FLOWS Document Retrieval 5.6.2. - 155 - Approved For Release 2000/05/30 : CIAM1RE8103952A000100050001-7 Approved For Release 2000/05/W8gRDP78-03952A000100050001-7 prescription, without the records themselves, is also possible if the information analyst so desires.). Codes appearing in the records would be translated into clear text for ease of understanding by the information analyst and customer (if the latter also reviews the listing directly). The information analyst will study the various machine listings received to determine the relevance of the retrieved records to the search prescription, and, particularly in the case of inherited file out- puts, will consult with other information analysts familiar with the contents and vocabularies of such files as required. In a certain percentage of cases the output records may, themselves, answer the request. If so, the retrieval activity will end with the ,information analyst transmitting the desired information by mail or phone to the customer. On the other hand, the response might have been such that he will wish to re-enter the request with .mproved criteria. SYSTEM FLOWS Document Retrieval 5.6_2. - 156 - Approved For Release 2000/05/Rckft-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cl1-rtR-03952A000100050001-7 When the index record output is satisfactory but, in itself, does not supply the answer sought, the information analyst may order the pertinent documents from the Document Delivery System before transmitting .he results of the search to his custom- er for review. If so, he will encircle the appro- priate document numbers appearing on the first page of his listing and send this page to the Document Delivery System. Where inherited files, however, are involved he may be ordering personality or instal- lation dossiers, as well as documents, and will, therefore, follow a. slightly different procedure. Graphics and map index records uncovered during the _initial search will be transmitted to the customer who will order these items for himself. Dossiers, following their retrieval from the file, will be forwarded directly to the information analyst erequesting same. A replica, rather than the file copy of all other documents, however, including those recovered from the existing Intellofax and SR SYSTEM FLOWS Document Retrieval 5.6.2. - 157 - Approved For Release 2000/05/30 : CIAMMI03952A000100050001-7 Approved For Release 2000/05/36gMTRDP78-03952A000100050001-7 collections as well as from the microimage and hard- copy files of CHIVE, will be prepared before being transmitted to the analyst. The information analyst will review the output from the various document files, and, after removing those documents which do not appear to be pertinent, will transmit the response to the customer. Alterna- tivtely, the information analyst may be asked to respond to the inquiry by phone, memorandum, completion of a customer's response form, or by the preparation of a narrative report (e.g., a biographic summary). In the latter case, he would obviously have to supply information rather than documents, which might ne- cessitate a more sophisticated analysis and synthesis of the materials at hand. Lastly, the information analyst may update certain of his identifier records, as well as dossier files, to reflect the results of his analysis (see section, or send a marked copy of his report (if it deserves retention) back through the input process for indexing and storage in the Master Image File. He will also return any master cards or SYSTEM FLOWS - 158 - Approved For Release 2000/05/39E;atTRDP78-0)@/13alMOSY7a1 Approved For Release 2000/05/30 : Cl1-ggi-03952A000100050001-7 dossiers to their appropriate files, and report the closing out of the request by completing his copy of the request form. The latter will be sent for processing into the Management Data Files. 5.6.3. INFORMATION FILE BUIMING, MAINTENANCE, AND RETRIEVAL As has been pointed out, the CHIVE system, like the existing central reference operation, will require a variety of dictionaries and other support tools (given the general title of Vocabulary Control Files in this report). In addition, it will maintain sub- stantive files of information either in unsynthesized or summary form. Since the procedures for building such files as well as retrieving data therefrom will differ substantially from the document indexing and recovery process, they are reviewed here separately. Moreover, these files, unlike the Master Index records, will require continual maintenance, i.e., the deletion of obsolete or useless data as well as the correction of or addition of information to, existing records in SYSTEM FLOWS File Building 5.6.3. - 159 - Approved For Release 2000/05/30 : CIAMM-03952A000100050001-7 Approved For Release 2000/05/3tRUCTRDP78-03952A000100050001-7 the file. The Master Index File, on the other hand, will require little maintenance at the sub-record level as such--only the addition of new records to the file and the periodic retirement of segments of the file to a less accessible storage medium. Vocabulary Control File Maintenance Vocabulary Control Files (e.g., MOFIF, MLD, etc.) will be consulted by content indexers as well as header data indexers in order to select the ap- proved term or code for representing a subject or named-object mentioned in a document.* These files, initially, will be represented in listing form although some alternative reference medium will be intestigated. If the indexer finds no suitable entry for the topic mentioned in the document, or if the entry is erroneous or incomplete, he will prepare a File Maintenance Transcript Sheet on which he will specify the changes to be made to the file in question, *The maintenance of the personality identifier file (Master Dossier Index) is excepted from this discus- sion since, as the reader will recall from section, names will not be "identified" during the input process. SYSTEM FLOWS File Building - 160 - 5,6 3.1 Approved For Release 2000/05/3SELVEIRDP78-03952A000i00050001-7 Approved For Release 2000/05/30 : ClA5k6Pg-03952A000100050001-7 using a portion of the same command language employed in the retrieval of records from the Master Index File. The File Maintenance Transcript Sheet will be Passed to a dictionary editor who will be responsible for reviewing all changes made to this specific vocabu- lary control file. He will insure that the proposed transaction is legitimate and proper, and, after enter- ing the proposed changes by hand in his master listing, will forward the transcript sheet to the Data Tran- scription Group for typing. After the transcript sheet has been copied and any necessary corrections made, it will be processed in essentially the same manner as the Document index Transcript Sheets, that is, the forms will be convert- ed to machine language by the Page Reader and the resultant output fed to the EDP System for updating the pertinent machine files. A record of the changes made will then be printed out in the various arrange- ments required, and returned to the dictionary editor as well as all indexers using the particular vocabu- SYSTEM FLOWS File Building - 161 - Approved For Release 2000/05/30 : CIA-ERWF03952A000100050001-7 Approved For Release 2000/05/311E:CdiETRDP78-03952A000100050001-7 lary control file affected. The frequency of preparation of these printed supplements to master listings, as well as the frequency with which the master listings themselves will be rerun, will vary depending on the number of changes occurring over a given period of time. The initial period of CHIVE operation will permit time for some experimentation to arrive at the most satisfactory procedure. UIF and SIF Processing As indicated previcusy, formatted information files consisting of logical data units either in unsynthesized or summary form may be initiated either by: (a) analysts external to the CHIVE system having a pressing and continuing need for the retrieval of select facts (as distinct from documents) pertaining to a given subject or function; or (b) by CHIVE information analysts reacting to the accumu- lative effect of specific request patterns. Require- ments of this nature, since they will increase both the human and machine burden, will be reviewed by managers at the branch or higher level to determine, SYSTEM FLOWS File Building - 162 - Approved For Release 2000/05/39EatTRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClOOFF8-03952A000100050001-7 the anticipated load on the system and its capacity to respond to same. Accepted requests for the establishment of UIF or SIF files will be assigned to one or more infor- mation analysts conversant in the subject matter in- volved, for initiation of the input as well as main- tenance and retrieval processing. Assuming the data is to be stored in digital files, the information analyst responsible for the file will consult first with a specialist assigned to the EDP System known as an EDP File Analyst. The latter will be throughly familiar with the internal operations of the EDP System and, in particular, the method used to estab- lish new digital files. His duties would be analagous to those of an individual in the Planning Staff of the Machine Division/OCR, i.e., he will design the format and record. structure of the machine file re- quired by the information analyst and see to it that the file is actually established. In general, the approach of the area information analyst will be to use the document retrieval system - 163 - SYSTEM FLOWS File Building Approved For Release 2000/05/30 : ciASEGREIT-03952mooloomoo1-7 25X1B Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clgag-03952A000100050001-7 to help build the required information files. If the file, however, is to have the characteristics of an Unsynthesized Information File (see section 5.5.5, above), the actual involvement of the infor- matior analist in the input process may not be great since, presumably, the data requested is already re- flected in the content of document index records (i.e., the UIF would be built directly from re- arranged elements of index records).* Where this is indeed the case, the information analyst will periodically direct the computer to take such action by calling for the appropriate standing query and. record generation job to be run. SIF files, on the other hand, will require more activity on the part of the information analyst since they will consist of evaluated, summary records about named-objects or events. These can only be *If the data is not already being captured, then the request must be classified as a"special project" which would require a procedure all its own. SYSTEM FLOWS File Building - 165 - Approved For Release 2000/05/30 : CIAMMIN03952A000100050001-7 Approved For Release 2000/05/3gaKRDP78-03952A000100050001-7 generated (as suggested in section 5.5.6.) by the analysis of the output from a UIF- file, from the Master Index File, or by the processing of the in- coming documents themselves. Assuming the SIF is to be built from data in a UIF, the information analyst will, review the listed product from a UIF, comparing it with a listing of any records already stored in the SIF. If he decides to make a change to the SIF either by adding new data, deleting what was there, or by replacing old information with new, he will prepare a File Maintenance Transcript Sheet (similar, if not identical, to that used to update vocabulary control files) on which he will describe the transactions to be performed. This form will follow the usual path to typing, thence to the Page Reader, and finally to the EDP System for computer processing. The retrieval of data from either the SIF or UIF files might be initiated for a variety of reasons, the principal ones being as follows: a. To provide a listing of changes to the master file in order to update the infor- SYSTEM FLOWS File Building - 166 - Approved For Release 2000/05/acMRDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIAW/M03952A000100050001-7 mation analyst's printed version of the file. b. To provide a listing of the complete master file either for reference use by the infor- mation analyst* or for periodic publication and distribution to interested customers. c. To search, in response to a customer's request, for a specific fact or correlation of facts which could not be readily derived by human browsing of the printed records. Whatever the reason for initiating a retrieval transaction the process will be virtually the same as that followed in the retrieval of document index records (using the same retrieval language), with the exception that no inherited files should be involved in the search and no documents will ordi- narily need to be retrieved from the document image store. Schedules can, of course, be set up for the levying of standing queries which would cause the listing of all or a portion of a file on a periodic basis without any action being required on the part of the responsible information analyst. *The listing will be the primary mechanism for analyst-SIF communication. SYSTEM FLOWS File Building - 167 - Approved For Release 2000/05/30 : CIA9KEIREEI-03952A000100050001-7 Approved For Release 2000/05/SECRETRDP78-03952A000100050001-7 5.6.4. TASK TABLES FOR SYSTEM TRANSACTIONS Examples of the step-by-step procedure by which some of the system transactions outlined above might be carried out using the equipment, file organization, program organization, and operator procedures described elsewhere in this report are provided below. Obvi- ously, there are a variety of procedures that might be used to perform any of these tasks. 4hat is sug- gested here must, therefore, be regarded as tentative and subject to modification as procedures are worked out in detail during Phase III. With regard to the method of presentation, it should be pointed out that written descriptions of even the most routine human activities make difficult reading at best. Anf.this is no less true of a data processing operation, especially when couched in the language of the systems analyst. Secondly, it is a fact that if flow charts were prepared of many current central-reference operations, the resultant products would also appear relatively complex. Yet, somehow, humans manage to carry out the operations involved. SYSTEM FLOWS Task Tables 5.6.4. - 168 - Approved For Release 2000/05/1%616RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClA44-03952A000100050001-7 Lastly, it should be recognized that some atypical problems .are covered in the task tables which would not ordinarily be encountered in the average trans- action. These, necessarily, further complicate the narrative discussion. The tables which follow have four columns. The first column (STEP) contains the number of the oper- ation. The number is used in the body of the table to reference deviations from the normal sequence of operations. The phrase, "go to step 10," will tell the reader that the next operation in the sequence is step 10. The second column (AGENT) identifies the person or equipment which is chiefly responsible for carrying out the operation. The third column (LOCATION) shows where most of the operation is carried out. The fourth column (OPERATION) has one or more sentences for each operation which describes what takes place in the operation. These are either processing operations, in which some action is taken on the data covered by the task table, or they are decision operations in which a question is asked and SYSTEM FLOWS Task Tables 5.6.4. - 169 - Approved For Release 2000/05/30 : CIA-EPTRET03952A000100050001-7 Approved For Release 2000/05/36MKTRDP78-03952A000100050001-7 the consequences are given for the two or more possible answers'. These consequences are usually in the form of "go to statements. The statement, "STOP," is the last statement in the OPERATION column for a particular task and indicates that the task is completed. SYSTEM FLOWS Task Tables 5.6.4. - 170 - Approved For Release 2000/05/3@gatiRDP78-03952A000100050001-7 STATSPEC Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Next 3 Page(s) In Document Exempt Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : aC078-03952A000100050001-7 Table 5-3 OVER-COUNTER DOCUMENT SEARCH Step Agent Location Operation 1. Requester Will vary Communicate available biblio- graphic identifying data on document (s) wanted by phone, mail, or in person to Docu- ment Delivery System, and , indicate response priority. 2. Informa- Document Prepare request form if not tion Delivery already made out. Control System Clerk 3. Informa- Document If control number is available tion Control Clerk Delivery System for the requested document, send one copy of the request form to the search unit respon- sible for the particular col- lection or sub-file in which the document would be stored; if the control number is not available, go to step 10. 4. Document Document If the document would ordinarily File Delivery be in the Microimage File, search Clerk System the motorized card file for the document control number cited and proceed to step 5; if the document would ordinarily be in the Hard Copy File, go to step 18, - 175 - Approved For Release 2000/05/30 : Clfirffe1VB-03952A000100050001-7 Approved For Release 2000/05/38EMIRDP78-03952A000100050001-7 Step Agent Location Operation 5. Document Document If the document is found, remove File Delivery document, replacing it. with an Clerk System "out" card, and send document with request form attached to reproduction; if document is not found, and it is in a Category for which the system has a re- pository responsibility, forward request to Hard Copy File searchers and qo to step 18. 6. Reproduc- Document Prepare paper copy of document tion Delivery on appropriate image-processing Equipment System equipment. Operator 7. Reproduc- Docament Transmit paper reproduction of tion Delivery document plus request form to Euuipment Operator System request receipt point, and re- turn master image to appro- priate files section for refil- ing. 8. Informa- Document Deliver copy of document (if tion Control Clerk Delivery, Jyste,ALL found) to requester. Otherwise notify requester that document is either still in transit or not available in CHIVE .(and why). If requester wishes, hold the request for a second search after a suitable time interval. - 176 - Approved For Release 2000/055birElp-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : eRg78-03952A000100050001-7 Ste- Agent Location Operation 9 Informa- Document Record temporary or final corn- , tion Control Delivery J.y; pletion of action on request form and transmit form to Data , , Clerk Transcription Group for typing - and subsequent insertion (via , , the Page Reader) into the ' , Management Data Files. End of , Over-Counter Document Search. . STOP. , 10. Informa- Document Telephone, or send copy of tion Delivery request form to, EDP System. Control System Clerk ii. Informa- Computer If a priority request, deliver tion Control Center to console operator; if not priority, send to key punching , Clerk and go to step 16. , 12. 'Computer Computer Key the request into the corn- Operator Center puter using the inquiry console.* 13, Computer Operator Computer Center Using the document identifying handles provided by the re- quester (e.g., post, airgram number, jPRS number, date, or other), search the header data portion of the Master Document Index File and print out the corresponding document control numbers. *Cross reference listings, arranged in various sequences, will also be available for consultation and may be used in preference to machine queries to recover document control numbers where this approach would be equally effective. - 117 - Approved For Release 2000/05/30 : CIAME78-03952A000100050001-7 Approved For Release 2000/05/36EMTRDP78-03952A000100050001-7 Step Agent Location Operation 14. Computer Computer Transmit results of printout Operator Center to Information Control Clerk. 15. Informa- Computer Telephone or transmit request tion Control , Clerk Center form with list of document con- trol numbers to Document De- . livery System. Go to step 4. 16. Key Computer Key punch search specifications 17. Punch Operator Computer Center Computer and transmit cards to operations section to await batch proces- sing, Insert the request into the Operator Center 1 computer and go to step 13. 18, Document File Document Delivery Search the appropriate. segment of the -Tard Copy File. If docu- Clerk System ment is found, remove document, , with an "out" card, send document with request form attached to reproduction, and go to step 6. If document is not found, so indicate on re- quest form, send request form back to receipt point, and go to step 8. , 178 Approved For Release 2000/059Lecet-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Cg-RBg8-03952A000100050001-7 Table 5-4 GENERATION AND INPUT PROCESSING OF FORMATTED INFORMATION/INDEX RECORDS PREPARED UNDER CONTRACT* Step Agent Location Operation :. i Informa- tion Contractor Receive and log in the periodical, monograph, or other publication , Control to be exploited. Clerk 2. Informa- Contractor Obtain code designation (if a tion serial) from an official list Control and enter same on a routing .: Clerk sheet clipped to the publica- , 3. Informa- Contractor tion. Sort and distribute publications tion to appropriate translators de- Control pending upon language or content . Clerk 4. Trans- Contractor of publication, Scan content of publication for labor data of interest to CHIVE and determine elements of informa- tion to be extracted. *This table illustrates the procedure which might be followed where the following conditions prevail: (a) CHIVE can influence the automation of data at the source- (b) the elements of information to be extracted lend themselves to a highly formatted record structure. Information of this type which enters the central reference system now, but only in hard copy, includes the Political and Scientific Biographic Cards from JPRS, Bibliographic Cards from the MIRA contract at the Library of Congress, abstracts of scientific articles from FDD, etc, - 179 - Approved For Release 2000/05/30 : CISEEIRE78-03952A000100050001-7 Approved For Release 2000/05/?kaETRDP78-03952A000100050001-7 Step Agent Location Operation 5. Trans- lator Contractor Type formatted transcript sheet for Ach article, monograph, or other, containing the pertinent information required. 'Enter . . data in English in the appro- priate columns or spaces pro- vided, and in the coding con- vention required where this does not require dictionary consul- tation. For the latter (e.g., organization names), enter descriptor in clear text. Type "remarks" - type information, the abstract body (if a scien- tific article), and similar un- formatted text at the end of the index record. 6. Trans- lator Contractor Clip transcri9t Sheet to publi- cation and transmit both to coding group co-located with the Contractor or internal to CHIVE'. 7. Content Contractor Add codes, where required, on to Indexer or CHIVE transcript Sheet in addition to clear text after consulting per- tinent CHIVE dictionaries. 8. Content Contractor Return publications to file and Indexer or CHIVE send transcript Sheets to typists. 9. Typist Contractor or CHIVE If typed product is to .be read by CHIVE's Page Reader, type entries in form of hard copy; otherwise, generate paper tape as well as hard copy on Flexowriter-like device and go to step 11. - 180 - Approved For Release 2000/05/4tcllerRDP78-03952A000100050001-7 IP Approved For Release 2000/05/30 : AgaT78-03952A000100050001-7 Step Agent Location Operation , 10. Page Reader CHIVE Read typed copy and feed machine- language product to computer. 11. Computer CHIVE Process records into Master Docu- ment Index File. , . , 12. Computer CHIVE If CHIVE area desk most concerned ' with input records generated by ' . contractor does not desire to ' review additions made to the files, , ' . input process is completed. End of Input of Formatted Index Records Prepared under Contract. STOP. If opposite is true, print out (on a periodic basis) a hard . , . copy listing of new records enter- ing system, transmit listing to . appropriate CHIVE area desk, and go to step 13. 13. ' informa- Lion CHIVE Scan output listing for unwanted items. Analyst , 14. Informa- tion Analyst CHIVE Prepare a File Maintenance Tran- script Sheet containing the usual job specifications (e.g., trans- action originator, classification, file to be addressed, date, etc.), the numbers 3f the unique records to be add:ressed, and the operation (presumably a "delete") to be performed. - 181 - Approved For Release 2000/05/30 : CISMET8-03952A000100050001-7 Approved For Release 2000/05/WW-RDP78-03952A000100050001-7 Step Agent Location Operation 15. Informa- tion Analyst CHIVE Send transcript sheet via typing and Page Reader to EDP System for processing. 16. Computer CHIVE Delete unwanted recorda from the pertinent file.* *An alternative approach to that taken in steps 13-16 would have the information analyst responsible for the file make use of a remote display device to screen additions to the file and make deletions thereto. Indeed, such a device could be introduced much earlier in the input cycle as the means by which codes would be added to the records and any unwanted entries deleted before file updating is actually undertaken by the computer. - 182 - Approved For Release 2000/05gedat-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : 6FNCAg78-03952A000100050001-7 ten 1. Table 5-5 INFORMATION ANALYST ACTIVITY RELATIVE TO A.N .A.LL-SOURCE, ALL -FILE SEARCH FOR A NAMED PKSONALITY Agent q!ation Anaayst ';aalyst Infor- itation Analyst Infor- mation Analyst Location Operation -All vary - 183 - Obtain available identifying data (e.q., name, citizenship, occupa- tion, affiliation) on personality wanted by phone, mall or in per- son from requester. If request has been levied on right party, accept same if request has been levied on riqht area desk but wrong Information Anal7st (-because, on this desk, 1-..h re is more than one analyst and each specializes in a differ- ent topic), transfer request to correct individual. Obtain request number from cen- tral control point an::.1 enter in First section of interleaved. re- cfuest form no elementary data needed for logqinrj purposes, I .e., name of rerTuester, date, name of analyst handling request, etc. Send one copy of request form to control point for filing with. other ''opon" requests. Approved For Release 2000/05/30 : CISEalbe78-03952A000100050001-7 Approved For Release 2000/05/AEWRDP78-03952A000100050001-7 Step Agent Location Operation S. Informa- tion Analyst C.G.D. Search Master Dossier Index list- ing for references to inherited as well as CHIVE-built dossiers. If an entry for the personality is found, extract dossier number and date dossier identifier record was last updated. . Informa- tion Analyst C.G.D. Enter in the query statement section of one copy of the inter- leaved request form the specific search parameters to be used in querying the CHIVE-built Master Index File. For example, if the Name Group Table is to be used, enter single spellings of both surname and personal names; if the name group feature is to be bypassed, enter the specific variant spellings to be included in the search; if FNU's are not wan4-.er1, so specify; if a dossier is avaiMble on the personality, exclude unwanted references already on file in the dossier by specifying that the date of preparation of any document index record containing the desired name should not be of a lesser value than the date the dossier identifier record was last up- dated. Also list any other factors 'which will serve to limit the scope of the search e.g., citizenship, general or specific occupational category, date of birth range, etc. - 184 - Approved For Release 2000/05/MM-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : MCRN78-03952A000100050001-7 [Step Agent Location Operation . ' , ' . . . . , 7. 8. 9. Informa- tion Analyst , , ' , ? . . ? ' , Informae ? tion Analyst Informs.- tion ? Analyst C.G.D. . C.G.D. C.G.D. ? Assuming the Special Register (SR) name index portion of the Detail File to Comint Reports has not been integrated with the CHIVE Master Document Index, complete the copy of the inter- leaved request form used for searches of the SR Detail File consulting (as necessary) with an Information Analyst familiar with the vocabulary and file structure of this inherited file system. Refer to the printed version of the Name sGroup Table to help select the variant name spellings to be searched in this file, and also include any variant spellings required if the transliteration system employed in this file is unique. If a dossier was discovered on the personality in step 5, enter its number on the dossier re- trieval copy of the request form. Forward the completed request form resulting from step 6 Ithrough typing and. Page Reader to the Computer Center for re- trieval of the pertinent index ,records from the CHIVE-built Master Index File; forward the request form resulting from step 7 directly to the Computer Center for manual retrieval and subse- quent listing of the relevant name records from the punch card - 185 - Approved For Release 2000/05/30 : CReRE178-03952A000100050001-7 Approved For Release 2000/05853CRETX-RDP78-03952A000100050001-7 Step Agent Location Operation file, inherited from SR; forward the dossier request form to the hard copy section of the Docu- ment Delivery System for re- covery of the dossier desired. 10. Informa- tion Analyst C.G.D. Telephone or communicate in some other fashion the details of the request to the Graphics Register (GR) for manual retrieval of photographs on the individual wanted from the inherited GR Per- sonality Photo File. (Photos on the person processed subsequent to the initiation of the CHIVE system will be uncovered, in- itially in the form of index records, in the computer search of the Master Index File refer- red to above.) 11. Informa- tion C.G.D. While awaiting receipt of the listed index records from the Analyst Master Index File and SR Name File, as well as the arrival of the hard copy dossier and photos, investigate any self-indexed card or document files on per- sonalities inherited from BR which may be located either with the area desk or in the central hard copy files of the Document Delivery System. Also examine any Supplementary Files (e.g., Who's Who publications, commercial indexes, etc.) avail- able at the area desk. - 186 - Approved For Release 2000/05atR@Ifk-RDP78-03952A000100050001-7 4110 Approved For Release 2000/05/30 : CgRg8-03952A000100050001-7 Step, Agent , Location Operation . , . ? , , , " . , . . ? 13. , . . . . - . 14. Informa- tion Analyst Informa- tion Analyst Informa:- tion Analyst C.G.D. . e.G.D. ? ? . , . , C.G.D. Upon delivery of the index list- ings frnm the Master Document Index and SR lame File searches, iLeLne eie references printed out to determine whether they indeed refer to the person sought. Consult again, if necessary, with an Information Analyst familiar with, the SR system to I interpret the output from the SR file. Assuming the request will not be rerun with improved criteria, identify the documents desired by encircling the appropriate document numbers appearing on the first pages of the listings. (Alternatively, the listing may be on a two-part form which will allow the Information Analyst to keep a carbon copy of the index record listing after using the original as an order for docu- ments.) Transmit the document orders to the Doeument Delivery System, and any photo control num-Hers to GR, for retrieval and reproluction of the items desired. *It is assumed, for the purposes of this table, that all material available on the personality being searched must be examined before a response can be made to the requester. For this reason, the search cannot end with the retrieval of an index record or card from a manual file, - 187 - Approved For Release 2000/05/30 : ClikEME78-03952A000100050001-7 Approved For Release 2000/05/AMKRDP78-03952A000100050001-7 ' Step Agent Location Operation 15. , Informa- tion Analyst C.G.D. ' Assemble all material collected from the various document re- positories (i.e., hard copy dossier . . _ eproductions of documents from the CHIVE.Mastef Image File, inherited Comint Document File, and GR Person- ality Photo File : . . original items pulled from self-indexed card or document files . . . and reference works from the Supplementary Files). :Remove those items which, after analy- sis of the documents themselves, prove to be unrelated to the person in question, and prepare the response in the manner re- quested by the customer. End of All-Source Search for a Named Personality. STOP. - 188 - Approved For Release 2000/05fttailk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClOOF7T8-03952A000100050001-7 Chapter 5.7. FILE CONVERSION S.7.1. INTRODUCTION Of the many types of extant central reference files which might be candidates for full or partial conversion to the CHIVE system, two are of primary concern. These are the document index and document image type files. In the former category are such files as the following: - SR Detail Index File (Comint) - SR Detail Index File (PI) FIB Active InstallationIndex File BR Dossier Index File IRS Document Index File - GR Ground Photo Index Inherited document image files include: - IRS Document File (includes aperture cards and hard copy) SR Comint Document File - BR One-Name File - FIB .Active Installation File (includes cards and folders) - BR Dossier Folder File There are, of course, many other types of central reference files in addition to those listed above, including some already in machine language. FILE CONVERSION Introduction - 189 - 5.7.1. Approved For Release 2000/05/30 : CIA-gpen-I33952A000100050001-7 Approved For Release 2000/05/3WARTRDP78-03952A000100050001-7 Most of these, however, are either information files of such short-term interest that there would be little reason for converting the existing records, or are vocabulary control type files which, while they might be used to build analogous CHIVE indexing and retrieval tools, would not be converted per se. The discussion in this section, therefore, will cover only index and image files, in that order. 5.7.2. DOCUMENT INDEX FILES Reasons for Conversion One of the most important reasons for converting tha inherited files to the CHIVE system would be to , create a truly centralized source of reference data and information for the Agency. Conversion of the existing document index files to magnetic tape under the CHIVE system would provide a means of establishing effective data systems management. The conversion of the inherited files would result in a reduction in the total number of document index files that would have to be maintained. In addition, conversion of these files would tend to FILE CONVERSION Index Files - 190 - Approved For Release 2000/05Abalf-RDP78-03952A000100050001-7 mei 011111 mstsi mommi mot mirso Approved For Release 2000/05/30 : ClkW8-03952A0001000500017 simplify the operating procedures of the document indexing and retrieval system. By converting, only one set of procedures would be needed as opposed to a set of procedures for the inherited files and a different set of procedures for the CHIVE-built files if conversion were not undertaken. Further- more, a reduction in the total number of personnel in the document indexing and retrieval system and a reduction in space should be obtained by converting the inherited files. Degrees of Conversion There are at least three different degrees or types of conversion that are possible. The first is a direct conversion and is probably the simplest and least expensive. Direct conversion means simply that the card image would be converted directly to tape. This type of conversion would not reduce any of the duplicative information existing in the card files. Moreover, it is the least desirable because it would provide the least amount of flexibility. FILE CONVERSION Index Files - 191 - Approved For Release 2000/05/30 : CIA-FOISe4tET3952A000100050001-7 Approved For Release 2000/05/36MEIRDP78-03952A000100050001-7 The second type of conversion is to convert the card files to the CHIVE format. This would eliminate any redundancy existing in the card files by pulling all data that was indexed on any particular document into one logical CHIVE record. This type of conver- sion is more desirable since it would provide good flexibility and would eliminate the built-in redun- dancy of the existing card files. The third type of conversion would be a complete conversion, both syntactic and semantic. The syntactic as-Deets of the change would be similar to that de- scribed in the preceding paragraph. The semantic or vocabulary conversion, however, would re7luire a con- siderable amount of intellectual participation by analysts from the respectiveareas where the inherit- ed files originate. This type of conversion would be the most desirable and most flexible, but it would also be the -mot complex and difficult to accomplish. FILE CONVERSION Index Files - 192 - Approved For Release 2000/0WthaA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIWER*8-03952A000100050001-7 SR Detail Index File Study Of the various document index files described in Appendix 5.D., only one has been looked at in any detail to ascertain the conversion possibilities. That file was the SR Detail Index File--the study being performed to determine the advisability and. Air feasibility of converting the file from cards to magnetic tape or to a direct access device. Some of 04.04 the findings of this preliminary study are presented mw below, Whether these are representative of similar conclusions that might be reached vis-a-vis other inherited document index files after investigation f their individual conversion potential, one cannot say. Further study of the entire problem will be required during Phase III before any final recom- NNW arid mendations can be made. The following are the data that were collected from the SR study. The number of cards, in millions, that would have to be read to convert all of the Detail FILE CONVERSION Index Files - 193 - Approved For Release 2000/05/30 : CIA-SEMISI03952A000100050001-7 Approved For Release 2000/05/306WRIDP78-03952A000100050001-7 Index File is as follows: No. 1 File-Subject/Commodity No. 4 File-Area 7.2 4.0 No.'s 2,3,6,7,8,9 Files-Organi- zation and Personality 4.1 15.6 This means that 15.6 million cards would have to be read to acquire all of the data in the current Detail Index File. This data applies to conversion to tape or conversion to a direct access device. Both approaches are discussed in the following sections. Conversion to a Magnetic Tape File For the first part of this study, it was assumed that the Detail Index File would be converted to one long file ordered on series-document number, with all data pertaining to any one document constituting a logical record. The file size converted to tape would be approximately 930 million characters. This would result in approximately 40 tapes for the master file, with that many as first backup also. This indicates that at least 80 tapes would be required at any one FILE CONVERSION Index Files - 194 - Approved For Release 2000/05atatt-RDP78-03952A000100050001-7 !al VIP Approved For Release 2000/05/30 : Cl/W6P8-03952A000100050001-7 time to represent the file on tape. Assuming a thousand cards per minute input rate with 20% allowed for manual handling, this results in 307 hours of 360/Mod 30 machine time to read the file in, This is n-Juivalent to approximately 1.3 months of Mod 30 time (eight hours per day), Assuming the read-in is performed on extra shift, the minimum cost would be $3,000. In addition 30 to 35 hours of 7090 or 360/Mod 60 time would be needed for sorting., merg- ing, and file building. This cost would amount to approximately $14,000 . Programming and analysts costs are estimated at $10,000. Therefore, an initial cost or conversion would, at a minimum, cost about $27,000. It would. take a minimum of three hours to read a tape file of this size. An additional half-hour per day would be required for input request, sorting of input and output, output processing, output and maintenance. It was assumed that the Mod. 60 would. be used to do the search processing. This would amount to approximately $200 per hour. Assuming a once-a-day search, '1:..he approximate monthly machine rental to per- form the maintenance and retrieval of the SR Detail FILE CONVERSION - . 195 - Index Files ?Approved For Release 2000/05/30 : CIA-FEBERE13952A0B100050001-7 Approved For Release 2000/05/MMTRDP78-03952A000100050001-7 Index File would be $15,400. This is approximately. two-and-a-half times the present EAM rental of SR's: Machine Branch. Turn-around time on requests would suffer by converting a large file of this nature to tape. The SR personnel contacted indicated that a 24-hour turn- around on all requests would be unacceptable. They further indicated- that approximately 20% of the re- quests handled by SR require a two-hour-or-less re- sponse time. These priority requests are spread throughout the file, not just in a selected portion of the file. The amount of space presently occupied by the SR Machine Branch (card files and EAM gear) is ap- proximately 4,300-square feet. A reasonable value to place on this would be about $4 per square foot, per year. Assuming that 3,000 square feet of this area could be saved by -conversion, this would result in an effective savings of $12,000 per year. FILE CONVERSION Index Files - 196 - Approved For Release 2000/0NeRetA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClA041-03952A000100050001-7 Conversion to a Direct Access File Slightly different ground rules were chosen for this technique than were used on the "long tape file." Instead of trying to form one logical record from all the cards existing in the Detail File which originated from any, one document, the existing file structure was assumed to be transferred to the Data Cell. Also, it was assumed that a directory or access file of a very simple nature would be maintained to enhance re- trieval on this file. It was further assumed that the IBM 360/Mod 60 would be used to build the file and perform the operational activities required of the file. The file would reside on a 2321 Data Cell which has a capacity of 400 million characters on line stor- age. However, the cells on a Data Cell Drive may be changed much in the same manner that tapes are changed on tape drives or disk packs on disk drives. Only one Data Cell Drive, which can have a maximum of ton cells on line, is required. The converted Detail Index File would occupy approximately thirty cells assuming about 75%, packing. FILE CONVERSION - 197 - Index Files Approved For Release 2000/05/30 : CIA-BreR6103952A001MQVA01 Approved For Release 2000/05/35EMTRDP78-03952A000100050001-7 The read-in of the file, assuming a thousand cards per minute reading rate and 20% handling, would take 307 hours on the 1402 attached to the Mod 60. The data could be read on to tape as an interim measure to save some rental on the Data Cell. How- ever, some of these savings may be absorbed by addi- tional programming costs. Assuming the Mod 60 would be operating in a multi-programmed mode, the cost )f initial conversion would be as follows: Reader (1402) $ 1,600 Channels 200 Tapes 3,000 Data Cell 200 CPU 100 Analysis and Program- 10,000 ming $15,100 As was mentioned earlier, the structure of the file would be the same as exists presently in cards Therefore, no sorting for the input conversion is needed. Retrieval on the file would take advantage of the directory to reduce the number of records that must be read to satisfy a request. A rough estimate of the average number of cards accessed from the exist- FILE CONVERSION Index Files - 198 - Approved For Release 2000/05/?kle-RDP78:8362iDdIAY0050001-7 Approved For Release 2000/05/30 : Clagg-03952A000100050001-7 ing file is in the range of 60 to 70 thousand per request. Therefore, 100 thousand cards per request was assumed as a very safe estimate for the direct access file. Assuming 10 requests per day (based on current usage), this results in approximately one million cards being processed per day. The average time of 137 microseconds per card was estimated for card processing. This results in approximately 0.83 hours per month CPU time. CPU time for re- trieval and maintenance is 1.83 hours plus about 10% for handling which equals approximately two hours per month. This results in approximately $300-400 per month rental for the Mod 60 (for everything except the Data Cell). A range of costs are provided instead of more stable figures because of the dif- ficulty in estimating for a multi-programming environment. Estimated use of the Data Cell is approximately 20 hours per month for retrieval and 54 hours per month for maintenance if the entire file is passed each maintenance run. These two functions result in FILE CONVERSION Index Files - 199 - Approved For Release 2000/05/30 : CIA-NeREP3952A000100050001-7 Approved For Release 2000/05/3WRIRDP78-03952A000100050001-7 approximately $1200 a month rental. Summary The comments in this summary generally apply to both parts of the study except where specifically stated otherwise. The following table of data was provided, with some modifications by SR personnel, from the Report: 25X1A File Request Rates Searches/Mo. Searches/Day Requests/Mo. Requests/Day No. 1 32 1.5 167 7.6 No. 4 62 3.0 443 20.0 No. 8 24 1.+ 41 2.0 No. 7 14 0.6 88 4.0 No. 6 11 0.5 33 1.5 No.'s 2,3,9 73 3.3 1076 50.0 215 10.0 1848 85.1 The table shows, as the headings indicate, the average requests per month and day. It should be noted that 90% of the requests against the No. 4 (Area) and No.'s 2,3,9 (Personality) files are selected by manu- ally browsing the files. This means that 57% of the SR requests are handled manually. Further, from these facts, it is seen that the conversion to tape or direct FILE CONVERSION Index Files - 200 - a. 3 Approved For Release 2000/05artalt-RDP78-03AzAuuth00050001-7 Approved For Release 2000/05/30 : Clfr'W8-03952A000100050001-7 access file would effectively replace an EAM system that is handling an average of only 93 requests per month. This usage rate is very low. Even if the total request rate were used, it would still be a low usage rate for a computer driven file. The last statement is made for two reasons. First, if the actual number of requests (from a computer file point-of-view) were 215 per month, it would be highly questionable whether this would be large enough to warrant conversion. Second, the original 215 "requests" do not actually represent that many requests from a tape or direct access file standpoint. To explain--a sheet of paper entering the SR machine area containing instructions for searching a. file may ask for references relating to pipes, paper, and cars. These parts are treated as three requests, not one, even though they all would go against the same file. However, this would repre- sent only one request against the file from a tape or direct access point-of-view. Therefore, the total request rate of 215 per month would have to be divided FILE CONVERSION Index Files - 201 - Approved For Release 2000/05/30 : CIA-SE1RET03952A000100050001-7 Approved For Release 2000/05/36EUETRDP78-03952A000100050001-7 by some factor to reflect how many requests this would represent in a tape or direct access system. Data on what this factor should be is not available. at this time. On the basis Of these findings it is recommended that the total Detail File not be converted to magnetic tape. On the other hand, conversion of the Detail File to Data Cell storage appears to be economically feasible. The costs of performing the conversion and doing the required retrieval on a Data Cell attached to an IBM 360/Mod 60 are reasonable. Also, the turn- around time on a request is satisfactory since it would only take a little over five minutes to read and process the required 100,000 records to answer a request. This should leave adequate time for coding, and outputting the request. The decision to convert this file, however, cannot be based on these technical considerations alone. The usage rate must also be carefully ap- praised. Finally, it is important to remember that this conversion problem is but one of many CHIVE FILE CONVERSION Index Files - 202 - Approved For Release 2000/05abiqyk-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clagg-03952A000100050001-7 implementation tasks which must be addressed during the next 18 to 24 month period. 5.73. DOCUMENT IMAGE FILES A comprehensive list of existing document image files is contained in Appendix 5.D. along with a capsule description of the function and activity characteristics of each. Also included for most files is an appraisal of each file's susceptibility to being segmented according to geographical area as a means of transition to the creation of an all-source document file. This section will discuss the conversion alter- natives and recommend a posture for concurrent oper- ation of inherited and CHIVE-built document image files. It is felt that the approach presented will constitute a basis for orderly implementation of a new central document reference facility. It is appropriate, first, to look at the reasons why conversion to a single document system should be considered. The overriding argument for such a step is to eliminate the multiple reference points that an FILE CONVERSION Image Files 5.7.3. - 203 - Approved For Release 2000/05/30 : CIA- 3952A000100050001-7 Approved For Release 2000/05/3ggaDP78-03952A000100050001-7 analyst must currently consult and present to him a central reference point where a comprehensive response to his request can be provided. A further incentive for conversion to a central document system would be intra-Agency standardization of: - File media and techniques - Microfilm processing and reproduction equipment - Hard copy quality and format Conversion to a central repository and reproduction facility also presents a potential for reducing oper- ating costs by combining similar clerical efforts, and by facilitating the use of more advanced proces- sing devices. Assuming then that there are advantages to be derived from converting to a centralized document reference facility, let us consider to what degree this could reasonably be accomplished. Of about 25 document image files which are candi- dates for conversion (files enumerated in Appendix 5.D.), many can be excluded from consideration as candidates for conversion. A policy decision has been made to, FILE CONVERSION Image Files 5.7.3. - 204 - Approved For Release 2000/05/SKIZELT-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIMW8-03952A000100050001-7 the effect that only textual documents are to fall within CHIVE repository responsibility. This im- mediately excludes all graphic files (i.e., photo, film, slide, and map files) which are to remain the respective responsibilities of GR and the Map Library. Another major group of files are the dossiers Which are subject-oriented folders relating to per- sonalities, organizations, and installations. These files are maintained and referenced by information specialists who generally act as intermediaries between the consumer and the files. It has not been demon- strated that this type of information reference service can be improved by conversion of the existing files to another storage medium. Consequently, for the present it will be assumed that these files, which are primarily under the cognizance of BR and FIB, will be retained in their present form. The foregoing exclusions restrict the discussion, then, to document image files currently maintained by the Library (Intellofax) and SR. These files are FILE CONVERSION Image Files 5.7.3. - 205 - Approved For Release 2000/05/30 : CIA-IRBER103952A000100050001-7 Approved For Release 2000/05/306KROP78-03952A000100050001-7 characterized by direct reference activity by the consumer, and, in most cases, respond by furnishing the consumer with a document. Primarily, they fulfill a document retrieval function rather than an infor- mation retrieval function, and, as such, are prime candidates for initial implementation as part of a centralized document reference service. Other files may prove suitable for incorporation into such a facility, but they should be evaluated on an ad hoc basis after a nucleus system has been established. Our recommendatiOn, therefore, is that an all-source document reference facility consisting of document image files within Intellofax and SR be a design goal for the initial system. It should be pointed out that the document system i5 largely independent of the CHIVE computer/indexing effort and consequently could be implemented prior to placing the EDP system on an operational basis. The centralization goal could be attained either in one step or on a modular basis. Either all incoming documents from the two systems could be incorporated, up FILE CONVERSION Image Files 5.7.3. - 206 - Approved For Release 2000/05/nagy-RDP78-03952A000100050001-7 111.111. RIP Approved For Release 2000/05/30 : ClOEFF8-03952A000100050001-7 into the new system, or some portion of each (such as Chicom materials) could be assimilated into the CHIVE-built system. The latter approach offers the advantage of limiting the volume during an initial shakedown phase. The question remains as to how such an all- source document reference capability could be instituted. Essentially, it involves the problem of somehow combining two diverse inherited systems and integrating these with a third, new CHIVE-built system. As a fundamental tenet, total conversion of the existing document image files to the newly adopted file medium is not warranted or practical. The in- herited files are very large in volume, having been accumulated over a number of years. Conversion to virtually any new system would require a copy of the document to be completely re-photographed and re- processed into the new file medium. Some partial conversion to the new system might prove advisable for any segment of the file where high reference activity, over a long term, can be anticipated. FILE CONVERSION Image Files 5.7.3. - 207 - Approved For Release 2000/05/30 : CIA-BeeRE-1)3952A000100050001-7 Approved For Release 2000/05/AECAURDP78-03952A000100050001-7 However, because of the low activity rate of the total file, the cost of converting records which will never be active should be avoided. The recommended posture, therefore, is that inherited files will not be con- verted from their current form but will merely be co-- located within a-single area along with the CHIVE-built files. The appropriate processing equipment will be installed within this same area and a single reference point will be presented to the consumer. Requests will be serviced through the appropriate systems, and responses furnished through a single distribution point ,.ffnere the proper enforcement of security restraints will be administered. The inherited files will be retained for reference purposes only and will not be augmented. All new items introduced into the file will be assimilated into the CHIVE-built system. It is recognized that the recommended approach perpetuates existing files and techniques while intro- ducing one additional document system to operate con- currently. Nonetheless, this approach seems to be FILE CONVERSION Image Files 5.7.3. - 208 - Approved For Release 2000/05Aftralf-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clk-RFA-03952A000100050001-7 the only feasible way to cut over to a single, standardized document system and also eliminate the extreme cost and effort associated with a large- scale retrospective conversion. Experience has shown that there is a bias of reference activity toward more recent materials which would effect a gradual phasing out of the inherited systems with the growth of the CHIVE-built document file. FILE CONVERSION Image Files 5.7.3. - 209 - Approved For Release 2000/05/30 : CIA-IneRn3952A000100050001-7 Approved For Release 2000/05/30 : CIAWIK03952A000100050001-7 Chapter 5.8. COMPUTER INTERFACE 5.8.1. GENERAL The EDP portion of CHIVE will perform the follow- ing functions: - Build and maintain files - Create sub-files from existing files - Search files and retrieve data from them - Display data The techniques chosen to implement these functions provide a built-in flexibility that will also allow revisions in the definition of the content and struc- ture of CHIVE-built files. In a computer based system, special effort must be oe'oted to inputting data, searching for it, re- organizing it, and subsequently displaying it. An integral -.part of the EDP system is a command language that allows these types of manipulation. It is recog- nized that "unlimited" flexibility is allowed if the user can be persuaded to use machine language. More practically, a set of commands is provided that COMPUTER INTERF7V?E - 211 - General Approved For Release 2000/05/30 : CIAME15-03952MWM050001-7 Approved For Release 2000/05/1ECOKRDP78-03952A000100050001-7 permits personnel other than programmers to use the EDP system. The CHIVE command language is fully described in Appendix 7.A. The language allows the user to direct the performance of the four functions mentioned above. Full use of the commands requires good knowledge of the indexing procedures, logic, and the content and, structure of the records and files to be manipulated. It is planned that only information analysts, diction- ary editors, and, to some extent, content indexers, will be trained to use the language. The responsibilities concerned with defining new files and modifying existing file definitions will be assigned to the EDP file analyst. (See section 5.2.3. Lor further description.) The EDP file analyst must be trained to a level similar to that of a programmer, since he must be able to specify files to the system, initiate jobs for the machine operations personnel and participate in subsequent check-out. 5.8.2. COMMAND LANGUAGE The command language permits the information COMPUTER INTERFACE Command Language 5.8.2. - 212 - Approved For Release 2000/0513%EateDP78-03952A000100050001-7 WA Wok p. , Approved For Release 2000/05/30 : CIAW8-03952A000100050001-7 ana ysts to direct the EDP system to provide desired :esults and.. products. The first consideration of the user 13 to build and maintain files. The usual file maintenance o-oerations are provided. They are: - Adding new data to a file - Changing existing data - Deleting existing data The user can control the file maintenance operations in either of two ways The first way is the usual one of specifying a unique record identification and then having the desired maintenance perforMed on that record. The second ,4ay is to specify logical condi- tions t'hat coulJ TIllalify a sinrIle record or many records ithin a file for the specified maintenance operation. For example, it may be desired to change the names of all factories named. the Stalin Works to In such a ease it is only :accessary to sst up the test condition with a replace command. The desired changes are made without requir- ing the user to hnow in advance the unique identifi- cations of all of the records involved in the trans- COMPUTER INTERFACE Command Language 5.3.2. - 213 - Approved For Release 2000/05/30 : CIA-IneREP3952A000100050001-7 Approved For Release 2000/05/3g?U-TRIDP78-03952A000100050001-7 action. The second concern of the user is to search the files. The CHIVE command language provides basic search operators -and logical linkage. The available operators are: and, or, not, greater than, less than, and equal. In addition, a "scan" operator allows searches for a contiguous string of characters in a value field. Notation is provided for specifying that the character string can be in any position within the value field and in some relative position. For example, it may be desired: to find all occur: ences of the character string ACZN22 no matter liere it occurs in the value field or only when it is the first six characters of a value. Another capability provided by the command lan- guage is to allow indirect searches. Here we mean that the user can specify the results of one search to be used as arguments in a subsequent search. An example would he: "What universities or colleges were attended by engineers working at radar plants in Country A?" Atfirst search is necessary to deter- COMPUTER INTERFACE Command Language 5.3.2. - 214 - Approved For Release 2000/05tRizty?-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA- VON" -03952A000100050001-7 mine the names of engineers associated with radar plants in country A. A second search can then be made to associate these engineers with schools. The command language allows the researcher to specify that the names of the engineers be automatically used as input arguments to the second search. Thus the problem involved with routing an intermediate machine output to an information analyst, setting up a second search, and then submitting it to the system are elimi- nated. New files can be created by preserving the results of extensive searches of large document files. In addition, the capability of restructuring records is provided by the HIT processing commands of the CHIVE language. These commands allow a user to manipulate records after they have been found to satisfy search criteria and before they are transmitted to an out- put file. The control available permits saving for output all or specified portions of the original .records. In addition, computations can be specified and the resulting values can be appended to the new COMPUTER INTERFACE Command Language 5.3.2. - 215 - Approved For Release 2000/05/30 : CIA-FSEEREV3952A000100050001-7 Approved For Release 2000/05/WWRDP78-03952A000100050001-7 output records. The resulting files can in turn be searched and updated in the same manner as any other system data file. The command language also governs printing and displaying data. Section 7.11. describes output proces- sing in detail and Appendix 7.C. shows samples of the types of reports provided by the EDP System. To specify a report it is only necessary to use the print command and then to state the name of the file, the sort sequence, and the output format desired. The format type includes such parameters as number of lines. per page, width of printed portion of page, top and bottom literals, pagination, etc. The current report capability is felt to be adequate at this stage of the CHIVE development. Additional features will be pro- vided only after actu-1 need is established in an operational environment. r 0 0 FILE DEFINITIONS AND THE EDP FILE ANALYST The CHIVE command language allows manipulation of data in existing files and also permits a way of creating sub-files which can in turn be processed by the EDP system. These features directly concern COMPUTER INTERFACE - 216 - File Definitions Approved For Release 2000/05/?kkii-RDP78-93%52.A000100050001-7 Approved For Release 2000/05/30 : Clagq-03952A000100050001-7 the information analyst. The tasks and procedures associated with changing file definitions and adding new files to the system are the responsibility of the EDP file analyst. The CHIVE EDP programs are controlled by external descrip- tions of the data files to be processed. The data descriptions taken collectively are called File Format Tables. Each table describes a file and its consti- tuent elements. If it is desired to process files other than those currently defined it is necessary to add new table descriptions to those already in existence. The File Format Tables contain all the informa- tion about an item that is required to process it. Included are the terms allowed in a record, term groupings, which terms are used as identifiers, addres- sing parameters, occurrence data, bow stored, and con- tent legality parameters. Extensive revisions can be made to the tables. In addition to adding new files, terms can be added to or deleted from an existing file. Legalities can also be changed. It is important to note that revisions of this type do not require any COMPUTER INTERFACE - 217 - File Definition Approved For Release 2000/05/30 : CIA-8eeRaiD3952A00A19035.0001-7 Approved For Release 2000/05/3(REBODP78-03952A000100050001-7 maintenance to the EDP programs. The external file definition concept requires a special maintenance system. There are two main functions involved: the first concerns generating file format tables, and the second involves restruc- turing existing file data records. File format tables are generated from descriptions supplied by file analysts. Some types of table revision will result in producing a table that is inconsistent with the existing file. In this case, the existing file is processed so that its item structure reflects the new table revisions. After this Step it is possible dm for the EDP system to operate correctly on the revised file with the new file format table. 5.3.4. SUMMARY The CHIVE EDP System can be viewed by the informa- tion analyst as a tool for manipulating data. In order o get at this information, he must learn the rules and Procedures attendant with the CHIVE command Ian- juage. Forms will be designed to aid and guide in transcribing the commands. The EDP system is designed COMPUTER INTERFACE Lillitaw. 5 . 4. - 218 - Approved For Release 2000/05/Ackty-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : Clk8g8-03952A000100050001-7 to allow random transactions which will obviate to some tent the scheduling of input to the machine. Output will be 7;ufficiently identified so it can be routed -loacI7. to the information analyst. It is recognized that the interaction of the man and machine is never smooth. For this reason two remote consoles will be included in the initial system. These consoles will permit experimenting, in an operational environment, with the problems of direct communication between the information analvst and the EDP System_ They should be 'helpful in expedi- ting icarch processing, reducing pa;-)r outpat volumes and in simplifyin the problem of routing request,73 to and from. the computer. COMPUTER INTERFACE Summary 5.3.4. - 219 - Approved For Release 2000/05/30 : CIA-SIBERBT03952A000100050001-7 Approved For Release 2000/05/30 : CIRW8-03952A000100050001-7 Aopendix 5.A. THE ORGANIZATIONAL PROBLEM This appendix describes the reasoning Ifihich led CHIVE to recommend the geographic organization of input and retrie -al personnel with additional topical specialization or certain priority countries. In it, various alternative organizational configurations are described and their advantages and disadvantages dis- cussed.. A formal report on the CHIVE Indexing Experi- ment which led to some revision of the organizational concept recommended here--namely, the removal of the coding responsibility as such from the information analyst's area of concern--will be published in the near future as an additional appendix to this Phase II moo Report. 5.A.1. ORGANIZATIONAL OBJECTIVES mart In considering the overall problem of how best to organize the functions to be performed and personnel to carry out these functions in a future storage and Jaw retrieval system, it appears logical to address oneself first to the primary objectives of the contemplated ORGANIZATIONAL PROBLEM - 221 - Objectives Approved For Release 2000/05/30 : CIA-SEriteT03952A606100050001-7 Approved For Release 2000/05/3g?KTRIDP78-03952A000100050001-7 system and to derive from these a subset of organi- zational-or management requirements which, if met, could assist in the attainment of the ultimate system goals. A particular organizational and management framevork, of course, cannot by itself insure the achievement of a system superior to that now in existence. On the other hand, it is equally clear that despite all the advantages of EDP hardware (including stored program logic, speeds, etc.) and new developments in the information retrieval state- of-the-art, these tools alone are as yet insuffi- cient to provide any major breakthroughs, and indeed have inherent disadvantages as well as advantages which, in the final analysis, must be taken into account. For this reason the efficient organization and employment of personnel takes on added significance. In fact, it may well determine whether a major step forward is possible. The principal CHIVE system design objectives which have been discussed in some detail in earlier ORGANIZATIONAL PROBLEM Objectives 5.A.1. - 222 - Approved For Release 2000/05Atek-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIWI?F/8-03952A000100050001-7 documentation may be summarized for the purposes of this discussion as follows: Objectives derived from user needs 1. Broder document coverage 2. Increased indexing specificity . More exhaustive indexing 4. Capability to answer more complex questions S. Reduction of retrieval time 5. Single-service point Common system vocabularies 3. All-source output capability ObAectives d.erivoi needs Micro-storage medium 10. Increased transcription speeds 11. Increased file utilization 12. More efficient use of available manpower w/o unacceptble degradation of system performance 13. Reduction of index and support file query time 14. Reduction of manual labor involved in preparing 'system outputs (research aids, acquisition lists, etc.) ORGANIZATION-\L PROBLEM Objectives - 223 - Approved For Release 2000/05/30 : CIA-Faha1t3952A000100050001-7 Approved For Release 2000/05/30S:Egla1/4411DP78-03952A000100050001-7 15. Improved communication with customer 16. Increased index record lengths so as to reduce file proliferation 17. Improved evaluative tools for management Some of the above are themselves organizational objectives for CHIVE, e.g., items 6, 8, and 15. Other listed objectives, if they are to be achieved, have implications at least for the organizational side of the total system design effort as well as for other design tasks. Combining the former with some deductive reasoning about the latter which is oriented towards the personnel and. management impli- cations thereof, it is possible to form a list of what might be called CHIVE organizational require- ments. This list follows, and it is important to this discussion since it sets the goals in terms of which various alternative organizational configur- ations are compared. Oblectives Influencing CHIVE Organizational Structure I. Specialization with minimum processing ORGANIZATIONAL PROBLEM Objectives 5.A.1. - 224 - Approved For Release 2000/052tAtft-RbP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIRW8-03952A000100050001-7 duplication Encourage specialization on the part of information analysts to the extent possible so as to improve the quality of inputs and relevance of outputs to customer needs. At the same time minimize duplicative processing activities--i.e., multiple readings of the same documents, expen- diture of intellectual time in term selection, transcription, etc. 2. Minimum customer contact points Facilitate direct interface between the user seeking information and the information analyst -aost knowledgeable on the problem. Provide a coordination capability where required, but organize analysts so as to reduce need for same. 3. All-source service from any point Organize system so that requester, if he so desires, can receive all pertinent information from whatever source that bears on his search problem. 4. Close comTrcunication between input and query handlers ORGANIZATIONAL PROBLEM - 225 - Objectives Approved For Release 2000/05/30 : CIA-SEITRET03952ACf00406050001-7 Approved For Release 2000/05/M:CaRTRDP78-03952A000100050001-7 Enable person querying system store to be thoroughly acquainted with processed inputs. Similarly, keep indexers informed of requests being handled by the system. Ideally, input and query processors should be one and the same. 5. Close communication between system operators and users Operators should be fully cognizant of intel- ligence needs and priorities of research analysts. This is especially important in theCIA appli- cation where the breadth of customer subject interests and responsibilities and the volume of the data base are so large as to prevent equal attention being given to all subjects or source's. 6. Document control--first priority The primary responsibility of the central reference system, i.e., to establish a basic retrospective search capability for all positive intelligence documents of immediate or potential interest to the Agency, must not be diluted by the additior :rf special tasks which, if permitted ORGANIZATIONAL PROBLEM Objectives 5.A.1. - 226 - Approved For Release 2000/05gtAlfk-RDP78-03952A000100050001-7 VMS Approved For Release 2000/05/30 : Clardg-03952A000100050001-7 to grow unrestrained, would prevent the achieve- ment of fundamental goals. Elemental priorities must be established and adhered to, and personnel organized in a fashion to bar the drift toward serving- specialized user interests. 7. Job satisfaction Morale of the central reference personnel must be maintained to reduce turnover and attract high-quality persons to the staff. Information analysts positions should afford opportunities for career growth and offer sufficient intelleC- tual challenge to interest professional employees. 8. Flexibility in personnel allocations New processing requirements and shifts in intelligence interests and priorities should not unduly upset the central reference operations and organizational structure. Requirements for retraining should be minimal if standard vocabu- laries, input, and retrieval systems prevail throughout CHIVE. Ideally the shift of one or more persons to more pressing tasks would not completely destroy an existing activity assuming ORGANIZATIONAL PPOBLEM - 227 - Objectives Approved For Release 2000/05/30 : CIA6113M103952A60.0100050001-7 25X1B Approved For Release 2000/05/3V:WRDP78-03952A000100050001-7 the assignment of more than one person to a given subject or geographic area to begin with 5.A.2. ALTERNATIVE FIRST-LEVEL ORGANIZATIONAL CONCEPTS Keeping in mind the above-listed objectives for organizing the central reference personnel and acti- vities, what kind of organizational configuration would appear to offer the best hope of meeting most if not all of these aims? In this section we will review some of the possible alternatives without necessarily considering all variant approaches which might theoretically be envisaged. The focus here will be on the initial; or first-level, organizational breakdown. In a subsequent section we will address the problem of how to manage activities within the rough organizational framework selected. 5.A.2.1. Alternative A - Retention of Present Configuration Under this concept the existing structure of OCR would be accepted as is. Input and querying would be organized by subject (Biographic Register, and Intellofax), by ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.1. 228 Approved For Release 2000/0SPERM-RDP78-03952A000100050001-7 -4 4 apt Approved For Release 2000/05/30 : CIRWW8-03952A000100050001-7 subject within source (Special Register), and by information carrier (Graphics Register and Map Library), Specialized. EDP systems could be developed .which would be tailored to the needs and desires of each Register or Division which might well employ different vocabularies, input and output processes, document storage media, etc. Alternatively, all systems might be required to adopt common file formats, dictionaries, pro- grams, document storage and delivery systems, and. so forth in order to simplify management understanding and control of processing activities and reduce design costs. The principal advantages of this approach are operator and management familiarity with ad- ministering such a system, the availability of trained personnel and established operational procedures, the avoidance of any drastic reshuf- fling of personnel and slots with all the atten- dant problems associated therewith, and the assurance of continuing a level of system per- ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.1. 229 Approved For Release 2000/05/30: CIA-16150M3952A000100050001-7 Approved For Release 2000/05/3?WRDP78-03952A000100050001-7 formance at least as high as that which it now. obtains. In summary, the retention of the exist- ing configuration is attractive because it would be the easiest to implement, and because we know it works even if the efficiency and quality of its performance is perhaps less than might be desired. The major reason for not following this route is that, while the risks are less, the system will always be constrained by the organizational struc- ture within which it must operate. Thus the potential for real improvement will be limited. Specifically, it would be impossible to make any real progress toward achieving objectives 1-3 above and limits severely what can be accomplished on objective 8. Redundant reading and analysis of collateral documents could scarcely be avoided and the trend toward all-source information files might foster duplicative processing (already initiated by FIB's exploitation of Comint materials) in the SI area as well. Semi-duplicative document repositories, such as now exist in FIB, BR, the ORGANIZATIONAL PROBLEM First-Level ConceptS 5.A.2.1. - 230 - Approved For Release 2000/MtRON-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : ClaW3-03952A0001000500017 Intellofax System, and to a minor extent GR, would probaly persist because of the difficulty of identifying in advance which repository will choose to keep a given document. Customers seeking to exploit all the subsystems would still be faced with the necessity of interrogating each system separately unless an inter-system reference group were provided or the system contacted assumed the responsibility of querying all others. Either of the latter potential solutions, however, would interpose request "interpreters" between the customer and the ultimate respondent with consequent ill effects to the communication process. In brief, while Alternative ?i is appealing because of its familiarity, its inherent disad- vantages are sufficient in number to influence a search for something better if such can be found. 5.A.2.2. Alternative B - Single, All-Source Document Re -ieval System: Separate Biographic Information Facility Bet.een the extremes of a completely central- ized, all-source, all-topic storage and retrieval ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 231 - Approved For Release 2000/05/30 : CIA-F8IBEREI3952A000100050001-7 Approved For Release 2000/05/3gcelkIRDP78-03952A000100050001-7 system and the existing decentralized configur- ation of OCR many variations and alternative combinations can be conceived. That which has attracted the most attention perhaps is the concept of merging Intellofax, the Special Regis- ter, and the Foreign Installations Branch but leaving the Biographic Register as a separate activity. Proponents of this approach (some of whom would also except FIB from the merger) gen- erally point to the "unique character" of the BR operation, its "analytical" responsibilities, its production of finished intelligence, the fact that it is not a document retrieval system at all but rather an inforTe,ation file, and so forth. Most of those favoring this compromise ap- proach are somewhat vague on the organizational details. Some, apparently, would establish an all-source BR, removing the responsibility for personality control of Comint materials from the conjoined Intellofax-Special Register operation, ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 232 - Approved For Release 2000/OgNRRA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : C1W-W8-03952A000100050001-7 Others would not oake this transfer of responsi- bility arguing, inter alia, that most BR customers are not cleared for Comint anyway. Some would retain the all-source FIB system as a separate file as weIi, presumably with installation index- ing remaining a part of the Intellofax-SR document input activity. The redundant analysis of docu- ments common to each of these systems has either not been considered by those who have recommended this approach or has been aecepted as a necessary evil. Of those favoring Alternative B or some vari- ation thereof, most do so in the belief that there are indeed advantages to be gained from the all- source, approach, integrated indexing, system standardization across OCR, common vocabularies and other reference tools, and other CHIVE goals. Most would, therefore, adopt CHIVE 's system recom- mendations if biographic data handling at least were excluded. 7:71:1at appears, however, to disturb people the ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. -233 - Approved For Release 2000/05/30: CIAMLF1715103952A000100050001-7 Approved For Release 2000/05/30SMOMP78-03952A000100050001-7 most about the prospect of including biographic intelligence in a centralized system is the index transcription problem. It is pointed out first of all that, while the necessity for filling out transcript sheets has long been accepted by Intello- fax and SR analysts, it would not be readily ac- cepted by BR personnel who, in recent years, have employed a file system (sometimes referred to as a "Collectanea" by Jocumentalists)* which requires no transcription at all. Second, there is the fact that any transcription requirement, no matter how limited, would diminish the number of person- ality references which could be processed by BR since it would necessarily add to processing time. Third, there is the argument, freluently expressed, that BR's need for multiple access points to per- sonality data has fallen off steadily over the *This term refers to any file system that used the general approach of lifting sections from a single source document, reproducing these excerpts, and physically filing them under each of the categories or key words of interest. ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 234 - Approved For Release 2000/058RRtit4-RDP78-039521000100050001-7 Approved For Release 2000/05/30: Cli1W78-03952A0001000500017 past several years following the assumption of eaAmunity responsibility for political person- alities. 7:rAy have more than name control over files, the reasoning goes, if the majority of rc!quests are for specific named individuals? The transcription argument might, indeed, just.fy leaving BR outside the central system concept were it not for the fact that following such a course helps none at all to resolve BR's storage and retrieval problems. Examined real- istically, it appears clear that there are only two fundamental ways of processing biographic or any other kind of information: (a) by creat- ing an index to documents containing the pertinent information (which index is then screened prior to the recovery of the documents themselves) or (b) by filing (and, if necessary, reproducing) the documents under the terms which constitute the desired search parameters (i.e., by estab- lishing a "self-indexed document collection). If the choice is totake the index path then ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 235 - Approved For Release 2000/05/30 : CIA-FeleRF03952A000100050001-7 Approved For Release 2000/05/3SECRUkDP78-03952A000100050001-7 certain elementary requirements must be met if retrieval from the system is to be successful. In the case of large personality record col- lections it means the index must carry sufficient identifying information about the personality to enable the searcher to distinguish between personalities bearing similar names. The more identifying information extracted from the docu- ment the better, but at the price of increased: transcription time. Alternatively, the more ab- breviated the index the less the transcription burden, but at the cost of more irrelevant docu- ments retrieved. The "collectanea" (or self-indexed document file) approach offers the user a reverse set of advantages and disadvantages. On the one hand, it virtually eliminates the function of having to transcribe words from documents. On the other hand, it vastly increas(s he physical storage require- ments of the syst A by virtue of the fact that each document must be multiplied by as many file ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 236 - Approved For Release 2000/05gt:RrEyt-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIASM-03952A000100050001-7 25X1A 25X1A headings as one chooses to store the document Hander. Since no system has unlimited space, this usually means that the means of access to the document collection are severely limited in comparison with document index systems In addi- tion, the filing problem is exaggerated by the xplorion of LThe original document population (witness I.Ez'H3 assignment ofil file ciers fulltlme to Its; central biographic card file andE cLerIc U dossier system)- The point of this brief detour into the of -.,),ansanality data handling is to make clear that nothing is really gained by leaving OR oatsi6e the central system framework unless it has airaftl been concluded that biographic data -will not bo controlled by an index per se. r!',ven this '...,;oulj not necessarily dictate the ei:clusion ryf us , 1.ra,inie process in, 31.11A-,::IL would be .oerfcIcy possible for the input analyst, after indexing the remainder of the document's content, Is hae the document or selected pages ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 237 - Approved For Release 2000/05/30 : CIA-MR6103952A000100050001-7 Approved For Release 2000/05/11EMETRDP78-03952A000100050001-7 therefrom reproduced and filed (,,n hard copy or microimage form) under the personality names of interest. If, on the other hand, the decision is to index biographic information then there are certain very real benefits in integrating this index activity with the representation of other subjects discussed in documents. for the remaining arguments deployed in the cause of keeping BR outside the integrated processing activity, they have little bearing On the manner in which biographic data should be ored and retrieved. Rather, they relate to le a_alytical functions to be performed, i.e., interpretation, correlation, synthesis, etc., after the raw material has been recovered from the files. Admittedly this intellectual process could be carried out by a separate group altogether, as indeed often occurs when a customer (e.g., a scientific intelligence analyst) chooses to review and interpret the basic documentation himself. But it can also be performed, perhaps equally well, ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.2. - 238 - Approved For Release 2000/05/?keik-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIRW8-03952A000100050001-7 Lrr persons who also index and retrieve biographic information. 'Thichever path is chosen it need not affect where and how documents are processed. 5.A.2.3. Alternative C - Co-located Organizational 'oafiguation A radically different organization concept from those discussed thus far, one which deserves at least brief consideration, is the notion of decentralizing document processing in the Agency by di persin(j the activity amongst the research and production components. Among the arguments for upgrading the so-called "analyst files" versus attempting to improve the central reference system are the folloing: - Analyst files will continue to be main- tained whatever is done centrally. Since they are a major information retrieval resource vihy not make them even more effective and efficient? - Providing analysts with manpower support in the form of information assistants hysically co-located with research per- sonnel in the production offices would relieve the analyst of most of his file maintenance problems and enable him to devote more time to research. ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3 - 239 - Approved For Release 2000/05/30 : CIA-FEtant3952A000100050001-7 Approved For Release 2000/05/3gUaDP78-03952A000100050001-7 - Analysts could more readily control what goes Into the files thus reducing input chaff and providing semi-evaluated re- trieval. - Full-time information specialists could' index more material than analysts can process into their files today thus im- proving the breadth and depth of coverage. In the decentralized as in the centralized system approach, it is possible to think of many wayT, in which the processing activity might be organized. The following, however, are perhaps the most logical alternatives: a. Decentralized input and files/central directory of files Under this approach OCR would virtually disappear with the exception of the Library, FDD, and possible the Graphics Register. Analysts would continue to process materials into their own files but might be provided some machine assistance in the areas of file manipulation, storage, and reproduction. In addition, a master profile or directory of analyst files would be created and maintained - 240 - Approved For Release 2000/0*EZRW-RDP78-03952A000100050001-7 ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. Approved For Release 2000/05/30 :gictIA3P78-03952A000100050001-7 at some central location. Analysts with a search problem would consult the directory, determine which file(s) to peruse, and then either exploit the file directly or work through the analyst who maintains the file. Personnel formerly attached to OCR could either be assigned to the research analysts as information assistants where they would perform the bulk of the input and retrieval activity, or the research analyst population might be increased by converting the slots to intelligence production positions. ago b. Decentralized Input/Centralized Files tool ftig maintaining decentralized analyst files, mow research analysts and/or their information egg assistants would be required to transcribe their indexing in such a fashion that a This scheme would be much the same as the above in that input processing would still be performed on a decentralized basis. The difference would be that, in addition to warmi0 ftgir onwii ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. - 241 - Approved For Release 2000/05/30 : CISEEIRET5-03952A000100050001-7 Approved For Release 2000/05/30EKTIDP78-03952A000100050001-7 record thereof could be passed to a central storage and retrieval facility. Similarly, reproductions of the documents they wished to store or the pertinent cita- tions thereto would be sent to central storage. Adoption of this approach would' greatly increase search specificity over the directory technique and greatly simplify the problem of gaining access to the data files themselves. c. Decentralized input and files for select subjects/centralized input and files where interests overlap This system is perhaps best represented in the real world by NSA where files of restricted interest are co-located with the most appropriate customer offices, while files of interest to many are maintained centrally. d. Centralized input and files/information specialists co-located with research components This system would continue the central refer- ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. - 242 - Approved For Release 2000/04WERGIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : AC:078-03952A000100050001-7 ence activity without prejudice to decen- tralized analyst files, but representatives of the central system would serve on permanent or rotational assignments in the customer offices. Their function would not be to index material for analysts, nor to actually search and retrieve material from the central system, but to improve communications between the analysts and the central storage and retrieval operation. They would provide advice to analysts on the reference services available to them, transmit their queries to the proper components, identify unnecessary and/or duplicative data files, inform the central service of current intelligence priorities and anticipated retrieval needs, and in general insure that both sides of the house achieved a full understanding of each other's problems, capa- bilities, and requirements. There is much that is attractive about all ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. - 243 - Approved For Release 2000/05/30 : afr-CRW78-03952A000100050001-7 25X1A Approved For Release 2000/05/46GMRDP78-03952A000100050001-7 the above alternatives primarily because all Provide better user definition and control of what the Agency should be retaining in its record collections, and because all provide a means for the analyst to exploit potentially useful files maintained by others. With the exception of stem 3ad., however, which appears to offer some significant advantages which might well be tested on a limite( basis, all suffer from one or more of the following disadvantages which - sufficiently serious to recommend the rejection of the decentralized organizational concept as a practical solution: re Elimination of part or all of the existing central processing activitivies would inevi- tably give rise to increased record keeping by Agency analysts. Indexing by these analysts would be highly duplicative and inefficient because of overlapping interests amongst Agency components. Even today the duplication of analyst file activity is. sufficiently widespread to cause some ? to seek ways in which the situation might be ameliorated. In a recent study* one re- search analyst reported that "the files of The Analyst's Inbox in the DWI Area: Help or Hindrance?, 30 June 1964, OTR/IPC, Confidential. - 244 ORGANIZATIONAL PROBLEM - Approved For Release 2000/05/igdre-RDP78-q3 A. 2 . 3 . V6ib *166514/2P:='" Approved For Release 2000/05/30 : gk-kirTID78-03952A000100050001-7 several offices within OCI and ORR practi- cally mirror each other, if not in totality, then at least in certain subjects." Among the reasons for this situation, the same analyst observed, is the failure of manage- ment to properly define the exact responsi- bility of the analyst beyond his geographic area, the necessity for the analyst to be aware of the "big picture," fear of requests from Agency officialdom whether they fall within the analyst's assigned mission or nor, physical distance from other potentially useful files, etc. Whatever the truth of these remarks (and all were noted during the mow CHIVE Fact-Finding Survey of the DD/I), any enlargement of the analyst's filing responsi- bilities would result in a corresponding increase in duplicate files. mr - It would be virtually impossible to establish and maintain inter-analyst consistency in indexing, and to enforce adherence to standard rules and practives. The many components in- volved, each responsible to a different line of command, would make coordination and management most difficult. - Analysts regard file maintenance as a necessary evil. Any suggestion that they expand their input activities, especially if it requires them to prepare index records in a fashion mai which can be "captured" for storage at a central location, would meet with great resistance. - Analysts select only a small percentage of ono incoming documents for filing. This fraction of collected intelligence infor- mation ordinarily reflects a current ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. - 245 - Approved For Release 2000/05/30 : C1EAM78-03952A000100050001-7 Approved For Release 2000/05/36ECRE-RDP78-03952A000100050001-7 problem bias or that material pertinent to an analyst's production assignments for the coming year. Moreover, some information which would be filed by an analyst with less experience on the job would be ignored by the more senior type who has already stored such information in his head. Unfortunately, the analyst's cranium, although a well-recognized part of the Agency's institutional memory, is not easily accessed by information seekers and is lost when the analyst leaves the Agency. - Analysts almost universally state that they I:'ant and need a central system for retrospective search and file back-up. They do not feel that their own files, nor even the sum of all files of all research components even if they could be made readily available to them, would fully satisfy their requirements. - The possibility of co-locating select central reference files with the primary users, as suggested in 3.c. above, is practical only for intelligence organi- zations having clear demarcations of sub- ject and area responsibility. Regrettably, no such pattern prevails in this Agency, as pointed out in the study referred to above. - Agency reference responsibilities to other USIB components, whether imposed by DCL) directive (e.g., biographic) or the result . of tradition and historical precedent, could-he met only with great difficulty if the centralized file concept were abandoned. Interface problems of inde- ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.3. - 246 - Approved For Release 2000/05W0W-RDP78-03952A000100050001-7 imp Approved For Release 2000/05/30 :WligP78-03952A000100050001-7 scribable complexity would inevitably arise. In summary, there appears to be no accept- able alternative to a central reference system for a consumer population as large and complex as that represented by the DD/I and other CIA and non-CIA components. 5.A.2.4. Alternative D - Centralized, Geographically Organized Configuration Assuming the organizational objectives listed on pages 224-227 are indeed the controlling parameters in selecting a management framework for a future information storage and retrieval system for the Agency, it is difficult to con- ceive of any better way of organizing the person- nel involved than by grouping them initially by geographic area. While this would not overcome all operational problems that can be envisaged, of all the systems considered it comes nearest to meeting the requirements outlined above. In a geographic organizational arrangement there would be, perhaps, five major geographic ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 247 - Approved For Release 2000/05/30 : Gi&CF&K78-03952A000100050001-7 Approved For Release 2000/05/3EalliTRDP78-03952A000100050001-7 divisions reporting directly to a single manager, presumably at the Assistant Director level. Most of the existing central reference repositories (i.e., BR, FIB, SR, and DD) would be abolished and their personnel transferred to the new geo- graphic components. Previous area assignments would be taken into account in relocating per- sonnel. Documents would be disseminated to the geo- graphic divisions by an external dissemination group which would also handle, dissemination to the research offices. These documents would in- clude all materials of whatever classification, format, or mode of presentation. International, documents (those dealing with subjects or events occurring in more than one country) would be routed to each of the geographic desks concerned when the application of area expertese in the indexing process seemed justified by the char- acter of the subject matter dealt with in the ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 248 - Approved For Release 2000/05aft*frRDP78-03952A000100050001-7 Approved For Release 2000/05/30 :WP78-03952A000100050001-7 document. The majority of documents, however, would be processed by one desk only. A single master file would be maintained of all documents indexed by the central reference system. Most requests would be levied directly on the geographic unit having responsibility for the area of concern. Occasional requests would have to be coordinated between the divisions when more than one country was involved, but this would be the exception rather than the rule. The respondent, under the new configuration, would be familiar with reporting from all sources on the matter of interest to the customer, and could thus insure that the data retrieved reflected the full response potential of the system. The proposed configuration would lose the advantage of source specialization in processing and would pose occasional problems of geographic overlap in document indexing and query coordination. However, these disadvantages are not felt to be ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 249 - Approved For Release 2000/05/30 : csEeM78 -0 3952A0 0 01 00050001 -7 Approved For Release 2000/05/3EMTRDP78-03952A000100050001-7 serious. The system would come very close to achieving all of the organizational goals set forth earlier as the following review of said objectives demonstrates: a. Processing duplication There would be a minimum amount of re,4. dundant reading and expenditure of intel- lectual effort in input processing since the majority of documents would be com- pletely processed by the nerson to whom they were sent. While the international document problem will arise, there are fewer international documents than there are documents dealing with multiple sub- 'pm TIF jects (i.e., persons, organizations/instal- OP lations, commodities, etc.). Nor must Infor- mation Analyst specialization necessarily be surrendered. Instead of concentrating on biographic, installation, or other data; they could specialize in certain topic areas of interest to intelligence--e.g., military, ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 250 - Approved For Release 2000/05/30.? C1A-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 :%4P3P78-03952A000100050001-7 economic, political affairs, etc.--within the country to which they are assigned. In addition, the extant duplication of document files would be eliminated with concomitant benefits in terms of storage space, reproduction loads, and filing require- ments. B. Customer Contact Points Analyst inquiries normally relate to a particular geographic area of the world, although the information sought is frequently diverse in character and not restricted to any particular collection resource. Under the configuration proposed, there would ordi- narily be no need for the requester to inter- rogate more than one component of the system since the organization of service personnel would mirror the manner in which user organi- zations are themselves organized, i.e., by topic within country. ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 251 - Approved For Release 2000/05/30 : CRelZPV8-03952A000100050001-7 Approved For Release 2000/05/39EaRURDP78-03952A000100050001-7 C. All-source service One of the principal advantages of geo- graphic organization is that, in addition to the establishment of all-source files, , there is an extra benefit to be derived from the bringing together of information analysts who have specialized source hackground. This pooling of knowledge will make for more in- formed reference personnel and will help remove gaps and ambiguities in the data files and authority lists developed in separate source environments. d. input-output communication The geographic organization of central reference personnel does not, in itself, assure or encutbr communication between input and. query handlers. Ra-,:her, this is affected by the communication processes built into the system, and by the extent to which personnel specialize in the various functional areas of innut and output processing. These ORGANIATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 252 - Approved For Release 2000/0ki5130-? CLA-RDP78-03952A000100050001-7 =k Approved For Release 2000/05/30 : gcREF'78-03952A000100050001-7 matters will be discussed in the next section. e. Operator-user communication 7fhile it may seem that geographic organi- zation oer se offers no inherent benefits over the present central reference configur- ation in terms of insuring better communi- cation between information and research analysts, in fact the information analyst in the proposed system, by virtue of the fact that he has access to a wider variety of sources and shares a subject/area assignment similar to that of his research counterpart, should be more cognizant of the later's resources an6 problems and, therefore, be able to offer him better service This, to be sure, is not enough, given the separate physical and operational environments in which each operates, and for this reason experiments such as locating certain Infor- mation Aai.ytr3. in the research components ORGANIZATIONAL PROBLEM ,First-Level Concepts 5.A.2.z. - 253 - Approved For Release 2000/05/30 : CIARE1.pIVI3-03952A000100050001-7 Approved For Release 2000/05/3.FNRDP78-03952A000100050001-7 should be tried as well. Processing priorities Geographic organization at the upper management levels cannot prevent information persolinel from .being assigned to respond to na.crow inteiests. vithin the geographic f3iy.isons an tiranizationai structure reflect- ing p:;Jocesing concerns (eq., document con- L.:col vs. ocial file projects) might help doing what, but since person- always n T,hifted around it is manage- . 1: eontrol which, in the final analysis, dtermine the direction and continuity c);f: jbL satisfacLion it would appear that the system ptoposed a richer and more meaningful environ- tont or Lhe information specialist than now available to him in the majority cvf: 6C1. registers. He would not be assigned ono :ianction only as, for example, the input ORGANIZATIONAL PROBLEM First-Level Concepts - 254 - Approved For Release 2000/05/3SECCREPDP78-03952A000100050001-7 Approved For Release 2000/05/30: gagF78-03952A000100050001-7 analyst in the Intellofax System 7 he would have access to a greater variety of docu- mentary materials; he would be able to specialize in a substantive area of intelli- gence concern; he would have contact with users of the information store and thus gain some appreciation of the problemto which his effort was addressed; and, not least important from the Agency's point of view, he would be better able to assume a research position if the opportunity arises for him to make such a move--as it often does. h. Flexibility Common system standards and procedures across the geographic division, as well as the increased. availability Personnel on any geographic area should lessen the problems entailed in re-allocating personnel to accom- modate changes in user needs. In a sense, the bringing together of, all persons working ORGANIZATIONAL PROBLEM First-Level Concepts 5.A.2.4. - 255 - Approved For Release 2000/05/30 : CliglalE78-03952A000100050001-7 Approved For Release 2000/05/36EMTRDP78-03952A000100050001-7 on the same country--persons now scattered amongst the various OCR registers--is ana- lagous to the establishment of a medical clinic composed.of specialists in various suject areas versus the continuation of individual medical practice. The assemblage of these various skills increases overall flexibility and assures the highest quality services Before concluding this section of the discus- sion, some additional facts may be worth noting. In a report to the Critical Collection Problems Committee of USIB, the DireCtor/SCIPS observed that "information processing activities, as contrasted with collection or research, generally are not oriented to area or country organization." How- ever, he went on to point out, "most of 'Os '4CY information handling activities surveyea [b SCIS] are concerned with peripheral descripti data rather than the substantive content of the informa- tion items and are, therefore, organized on a ORGANIZATIONAL PROBLEM First-Level Concepts 5. A 2 . 4 . - 256 - Approved For Release 2000/05SSCF4B1i'-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 :511f13P78-03952A0001000500017 functional basis rather than a geographic coverage basis." A different situation exists, he said, "where the process is dependent upon substantive content such as . . . deep indexing." In the latter case "then the lowest organization level is more apt to be structured on a geo- graphic area basis, like collection and research activities are prone to be."* In fact there can be little doubt that the processing of multi-source documents by geo- graphically-organized personnel will work. Within our own agency we have the Analysis Branch of the Document Division organized on this basis to process inputs into the Intellofax System. As is well known, the system deals with a wide _variety of intelligence report series and other documentary media. BR and FIB are similarly ar- ranged and, though they confine themselves to restricted subject areas, are faced with an even 25X1A ORGANIZATIONAL PROBLEM Filst-Level Concepts 5.A.2.4. - 257 - Approved For Release 2000/05/30 : CEIMBP8-03952A000100050001-7 Approved For Release 2000/05/3UMTRDP78-03952A000100050001-7 greater diversification of documentary inputs, including books, periodicals, newspapers and even photos. Outside CIA there is the DIA Jocument storage and retrieval system which reeieves inputs from all USIE agencies and inclexes peraons, organizations, locations, as well as other subj cts. Thus, the issue is not ,:ilhether input processing organized on geographic lines will work, or whether a multiplicity of doctiment types can be handled by a single organization-, but what the tradeoffs are versus some other approach to the problem. 5.A.3. ORGANIZATIONAL ALTERNATIVES WITHIN A GEO- GRAPHIC DIVISION The preceding section was addressed to the issue of the first-level organization of the central proces- sing activity. The problem, however, does not end Iere since, even if the geographic division concept is excepted, each-geographic division would be so large that some division of personnel into more manageable Atinistrative units would be required. ORGANIZATIONAL PROBLEM Geographic Division. 5.A.3. 258 Approved For Release 2000/05/Seatii-RDP78-03952A000100050001-7 Approved For Release 2000/05/30: Agg78-03952A000100050001-7 Referring back to the organizational o7pjectives listed earlier it appears that if th (Jec7fadic arrangement makes sense as the first cat, it would likewise be the preferred approach at every succeeding management level within the organization until the country level itself is reached. For example, if it did not seem desirable to group persons by document source or by the subject matter in documents they were assigned to store and retrieve because of the effects this would have on processing overlap, interface with the customer, capability for providing all-source service, and so forth, then it would make equally little sense to permit them to creep back into the system, although at a lower level, if the effect on the system's performance was still the same. The geographic concept begins to break down, however, when the volume of activity (input as well as requests) on a single country is characteristically so great that a relatively large number of information analysts must be assigned to the same country. It would be possible, of course, to have both the docu- - 259 - ORGANIZi\TIONAL PROBLEM Geographic Division 5.A.3. Approved For Release 2000/05/30 : ClArFaE/T3-03952A000100050001-7 Approved For Release 2000/05/MCSKRDP78-03952A000100050001-7 ments as well as the requests distributed indiscrimi- nately amongst these analysts, but specialization is always advantageous if it can be achieved at minimum or no cost to other system goals. 3ince not enough is known at this point about the input/output traffic that can be expected on every country in the world, nor 74hat the manpower require- ments and constraints will be on the CHIVE system, it is impossible to state with any degree of certainty where a division of personnel within a given geo- graphic area will be required. For some areas it seems logical to predict that an analyst will have complete responsibility for a country, e.g., one of ' emerging states in Africa which is of little conseauence in international affairs and, therefore, engenders little in the way of intelligence reporting or analyst interest. On the other hand, many informa- tion analysts will be required for the larger countries such as the USSR and China and thus the organization! of these analysts !becomes a matter of serious concern. The most reasonable alternative ways of grouping ORGANIZATION\L PROBLEM Geograohic Division - 260 - Approved For Release 2000/055EICIEWRDP78-03952A000100050001-7 mot Approved For Release 2000/05/30 :4?9413P78-03952A000100050001-7 personnel assigned to one country would seem to be the following: .A-3.1. Organization by Document Source Adoption of this approach would mean that separate groups of analysts would be established for each major document category. These cate- gories might be the open literature, collateral intelligence reports, Comint and T/KH, etc_ The principal advantage to be gained from this method of organization would be the availability of personnel trained on a document source basis, It would have stronger selling power if the indexing systems used were to differ by source. noway the latter will not be the case. Its disadvan- tages are that almost every request would have to be coordinated among the different source-oriented units since customers ? would customarily want more than one source searched; Information Thalysts would operate in different worlds and none would ; have a complete picture of reporting in his :parti- cular area of concern; the tendency would be to ORGANIZATIONAL PROBLEM Geographic Division 5,A.3.3. - 261 - Approved For Release 2000/05/30 : Cl/SMEN8-03952A000100050001-7 Approved For Release 2000/05/30613aDP78-03952A000100050001-7 :qaintain separate rather than integrated all- nourec ;Alas: and the multiple service-point ?roblem would remain. On balance, it does not seem to be a desirable approach. 5.A.3.2. Orr-anization bv Function This syntem would allocate to certain infor- mation analysts assigned to a country the respon- sibility for indexing all documents received on their area, to others the responsibility for answering all requests on said country, and pos- saav to a third group the task of maintaining' "special project files and establishing and periodically updating information files consist- ing of summarized data about a particular persOn or group of pnrsons, installation, or activity.' The notion of distinguishing input from retrieval personnel is not a new one. Libraries have traditiOnally followed this approach in separating tile cataloguing from the reference librarian function. Many EDP-supported informa- tion retrieval systems have also chosen this ORGANIZATIONAL PROBLEM Geographic Division', 5.A.3.2. - 262 - Approved For Release 2000/05SECRE1'A-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 :kE1931gP78-03952A000100050001-7 route, the original 433-L system of SAC being a prime exanpie in which so-called "query special- ists" (as distinguished from "coding specialists" and "file modification specialists") were to - handle all searches directed against the system. The advantages of separating personnel by the functions named are the following: - It heightens the job satisfaction of those assigned to the output end of the activity, thus reducing personnel turnover and en- abling the system to recruit and retain higher-quality personnel. - Persons unqualified to deal effectively with requesters can be separated there- from with less embarrassment to management. Similarly, persons who have neither the ee interest, background, nor temperament to become effective indexers can be given ? assignments more in keeping with their qualifications. - New personnel can be trained more quickly if the job responsibilities are more narrowly defined. This will reduce the total amount of unproductive time expended by the system, a matter of no small signi- ficance if the turnover rate is reasonably high. - By encouraging specialization the quality of the system's performance is enhanced. It permits processing to go on undistObed 07GANIZATION2\L PROBLEM Geographic Division 5.A.3.2. - 263 - Approved For Release 2000/05/30 : CIAKEW8-03952A000100050001-7 Approved For Release 2000/05/305:FeaDP78-03952A000100050001-7 by request interruptions with some con- sequent increase in operational efficiency. By formally separating the document storage and retrieval responsibility from special and general-purpose information file main- tenance, system functions would be better defined and management would have a clearer picture of their investment in either area. This Nsould bar the often unnoticed drift of centralized retrieial systems toward. increased special f. se-building activities to the detriment of establishing a basic retrieval capability over the documents entering the system. The principal disadvantages of functional separation are: - Query specialists would be unfamiliar with the inputs to the system except those they retrieved as the result of searches levied against the files. As a result they would tend to lose touch with current intelli- gence reporting unless some mechanism was provided for them to read select incoming documents, review the product of the in dexer activity, or other. In addition, all persons who index documents as well. as answer requests retain a great deal of information in their heads which is never reflected in the index representation of documents. Subtle though this advantage' may be, it makes for more effective service to customers in ways too numerous to mention. And it is most difficult to acquire this knowledge through any other mechanism than participating in the input process itself. - Input specialists would have little ap- preciation of customer needs. Being barred - 264 - ORGANIZATIONAL PROBLEM Geographic Division. Approved For Release 2000/05WRga-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : thAWFT78-03952A000100050001-7 from dealing with requesters, they would not know that subjects to stress in their input processing, nor how to distinguish the significant from the insignificant. - The inevitable tendency would be to con- sider the query specialist a cut above the indexer to the detriment of the input person's morale. As experience has shown in OCR, the request handler would be re- garded as having the more interesting job primarily because, having contact with users, he could understand better .that contribution the entire activity was making to the intelligence mission. Those indexers who were unable to make the change from input to request handling ? ? because no vacancies developed would ultimately take positions elsewhere. Those who remained would tend to repre- sent the less capable and imaginative until, ultimately, the entire input staff would take on these characteristics. This approach would conflict with the mode of operation in most OCR components. With the exception of the Intellofax 3ystem, most CYC.11. systems have chosen to have the same individuals handle queries who handle input to the files. Both the Special Register as well as sections of the Bio- graphic Register have actually operated for varying periods of time on a functional basis but reverted back to the integrated configuration. Certainly, the majority of experienced OCR staff members would prefer to have information analysts operate in both modes and would resist the other approach. - Peak request or input loads would require the temporary assignment of personnel to - 265 - ORGANI1ATIONAL PROBLEM Geographic Division 5.A.3.2. Approved For Release 2000/05/30 : ClaPflitE48-03952A000100050001-7 Approved For Release 2000/05/?gcgaRDP78-03952A000100050001-7 the duty which was not their prime responsibility. Indexers who performed the retrieval function would thereafter be able to claim, and rightly so, that they were able to do the job otherwise they would not have been called on in the first instance. This would tend to weaken management's argument for con- tinuinj the distinction. As can be seen, while a good case can be made for either configuration, we tend to favor not making a formal division of central reference- personnel along functional lines. 7ihile there : will, J..nevitaly, be some persons in the system, whose functions will be more or less unique, and others who because of personality or other limi- tations willbe confined to a restricted area of operations, these will be the exceptions rather, than the rule and, in the latter case at least, would not be reflected in the formal organi- zational structure. .A.3.3.? Organization Named Named Object This configuration would organize the infor- mation analysts by the major classes of data stored ORGANIZATIONAL PROBLEM Geographic Division - 266 - Approved For Release 2000/05SWIM-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 :5VETP78-03952A000100050001-7 and retrieved by the system. For example, within the USSR Division there might be a Personalities Branch, an Organization/Installation Branch, and a Subject/ommodity Branch. The Special Register is divided on this basis today and, in a sense, the collateral repositories of OCR, i.e., BR, FIB, and the Intellofax System, are reflections of the same concept except on a larger scale. We know that this approach will work since has been proven over many years of operating experience. Furthermore, by introducing this kind of division at a much lower operational level (namely the country desk) than is the case today, many of the ills of the existing system such as conflicting vocabularies, overlapping document files, diverse input/output procedures, and so on might well be eliminated. It also offers the advantage of immediately identifiable manpower trained in these particular areas and, in addition, permits a high degree of analyst specialization. 71hat makes this solution unattractive? The ORGANIZATIONAL PROBLEM Geographic Division 5.A.3.3. - 267 - Approved For Release 2000/05/30: CIAW&RfET-03952A000100050001-7 Approved For Release 2000/05/30 ? QA-RDP78-03952A000100050001-7 SECRET principal objection is, of course, the fact that it would be a rare document that would not have to be read aad indexed by all three groups. While Comint materials would be less troublesome in this regard, collateral documents and open literature are not typically oriented to any single type of named object. Attempts to coordinate the input effort so as to reduce duplication would be extremely difficult to implement, and document dissemination would in all likelihood take the form of dissemination of the same documents to all three points. Finally, there would remain the problem of coordinating the response to queries. A significant proportion of the requests would relate to all three subject areas and require a coordinated response. In summary, while this configuration is pre- ferable in many ways to the existing central refer- ence organization, it would be less efficient and ?economical than what might be desired. That there may be a better alternative was suggested earlier, - 268 - ORGANr4ATIONL PROBLEM Geographic Division 5.A.3.3. SECRET Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 ? 411111 IMP , Approved For Release 2000/05/30 : Wifri'78-03952A000100050001-7 anflit will be the subject of the next section. T32\.3.4. Organisation2i3 Topic The major selling points for a topic approach to the organization of the central reference acti- vity beneath the country level are (a) that it corresponds more closely than any other configur- ation to the kinds of requests we can anticipate will be levied on the system; and (p) that, while it does not eliminate entirely the problem of the multi-subject document, it would seem to confine the problem to reasonable bounds. If the former statement is accepted, then organization along topic lines would lessen the need to coordinate the search activity in order to provide the customer with a complete response to his query. Similarly, if documents tend to relate to a single, though broad, subject area of intelligence concerr or example political affairs, scientific and technical intelligence, military activities, or economic matters), then the need for multiple - routing of documents should be diminished and - 269 - Approved For Release 2000/05/30 : CIPM041-03952A000100050001-7 ORGANIZATIONAL PROBLEM Geographic Division 5.A.3.4. Approved For Release 2000/05/305.E&READP78-03952A000100050001-7 processing duplication minimized. A preliminary examination of documents enter- ing the current system, as well as a review- of aueries levied on the system by analysts, indi- cates that both do tend to concentrate on one or another of these basic subject areas. This is not too surprising since these are the classic divisions of strategic intelligence, and collection as well as production organizations within the intelligence community reflect this fact. It also appears that there is a reasonable balance of documents as well as queries in each of these topic areas such that there would not be a pre- ponderance of personnel assigned to any one field. As to whether it might be desirable to fu7ther refine the topical breakdown within poli- tical affairs, economics, etc., this would depend on the number of information analysts assigned to any one topic. Additional subdivisions are clearly possible and could be advantageous in that they would permit increased analyst specialization and - 270 - Approved For Release 2000/05116CREA-RDP78-03952A000100050001-7 ORGANIZATIONAL PRO3LEM Geographic Division 5.A.3.4. Approved For Release 2000/05/30 : gc-VETP78-03952A0001000500017 lessen the span of control problem for super- visors. On the other hand, these benefits might ultimately be offset by the inability of the system to separate documents cleanly on the basis of these increasingly narrow subject categories. Documents dealing with two or more major topics would, of course, be received by the system. However, this need not cause any undue concern. Multi-processing of a single multi-subject docu- ment by different topical specialists is less important than multi-processing of an international document by geographic specialists. Such documents would be directed to the unit which seemed princi- pally concerned for complete indexing even when the choice seemed rather arbitrary. If it appeared that the information reported seemed of more than average significance, this would not preclude an information copy of the same document being routed to another unit. Research analysts should prefer to deal with topic-oriented information specialists since they ORGANIATIONTiL PROBLEM Geographic Division 5.A.3.4. - 271 - Approved For Release 2000/05/30 : CI1SME7T3-03952A000100050001-7 Approved For Release 2000/05/3?geBaDP78-03952A000100050001-7 would find them better able to understand their search problems. Indeed, such information specialists might in time become more factually 3mowledgcab1e than their customers since they would have fewer extraneous responsibilities and could concentrate their exclusive attention on the subject at hand. IMO ORGANI-ATIONAL PROBLEM Geographic Division - 272 - Approved For Release 2000/09:61KROVRDP78-03952A000100050001-7 - OM involved included 16 indexers, 4 senior indexers, 3 Approved For Release 2000/05/30: ?g78-03952A000100050001-7 Appendix 5.B. PIREE,IMININRY EVALUATION OF THE CHIVE INDEXING EXPERIMENT 5.3.1. SUMMARY DESCRIPTION OF EXPERIMENT A joint OCR/CHIVE indexing experiment was con- ducted, from about 15 November 1954 to 15 January 1'2:35. Approximately two months training preceded the indexin-4 phase of the'experiment, while the query and evaluation phase is expected to extend through my. The personnel ,Jleric. typists, and 3 project monitors. The d.ata 25X6 consisted, of some 5,000 all-source documents on 25X6 col-k during the period. 1 July - 7. 1 30 September 1,7.64. , The experiment washeld to test certain or-Tlani- 6,; zational and indexing techniques -proposed by CHIVE. 7 .1 Specifically, it -oas desired to test the following .major concepts: ? 7 7 That with aderJuate supporting tools, a person can satisfactorily index all of the, information contained in documents, i. n people, -organi- zations/instaliations, areas, subjects, etc. EVLa:',TION OF EXPORIMENT SupilLary 5.E.]. - 273 - Approved For Release 2000/05/30 : Cl) '-03952A000100050001-7 Approved For Release 2000/05/3WIETRDP78-03952A000100050001-7 - That all-source materials (including Col- lateral, SI, and T/KH) not only can be proces- sed and retrieved in one integrated system but that certain advantages will accrue there- from. - That personnel organization by geographic area and, if necessary, by topic is feasible and desirable. That the CHIVE indexing approach will provide at least as many entry points to documents as that now obtainable from the sum of the indivi- dual indexes and other controls established in the various registers of OCR. - That header data (bibliographic) indexing can be performed by clerical personnel with a minimum of guidance. To.test these concepts, an experimental Branch was established. The Branch was organized into four topical sections: Political, Economic, Military, and Scientific and Technical. Each section was headed. by a senior indexer. More than half of the OCR person- nel assigned to the project had some previous indexing experience, but less than half were currently full- time indexers. Each section was allotted personnel who had experience in working with SI materials, 25X6 background, or familiarity with the Intelligence Subject Cpcle. Some of the individuals had more than 274 25X1A EVALUATION OF EXPERIMENT Summary Approved For Release 2000/066SMA-RDP78-03952A000100050001-7 25X6 25X6 25X6 Approved For Release 2000/05/30 :gakTP78-03952A000100050001-7 one of these attributes. Unfortunately, few of the indexers had previous topical specialization similar to that employed in the experiment. The indexing tools used during the experiment included: - The Intelligence Subject Code - A listing of on whom the Biographic Register maintains dossiers - The Special Register Manual - The NIS Gazetteer - The Special Register Code Book Supplement to the ISC - The CHIVE Indexing Manual - Miscellaneous dictionaries and other reference works. The Intelligence Subject Code and the SR Code Book Supplement were used to index subjects and commodities. The BR dossier list, SR Organization Manual, and the NIS Gazetteer were used as authorities for entering people, organizations, and place names--that is, EVALUATION OF EXPERIMENT Summary 5.B.1. - 275 - Approved For Release 2000/05/30 : CIWERV8-03952A000100050001-7 25X1A Approved For Release 2000/05/3RWRIDP78-03952A000100050001-7 whenever a significant person or organization was encountered, the indexer had to refer to the dossier list or organization manual, find the correct entry, and enter the code assigned by BF or R. All place names were checked in the NIS Gazetteer for the correct entry form. The CHIVE Indexing Manual con- tained the explanation of the indexing techniques, the method of transcription, and some preliminary indexing rules and procedures. The data base was All-source and consisted of , Collateral intelligence reports, translations, the FBIS, newspaper articles, Comint, T/KH materials, and miscellaneous other series. Each document category was represented in proportion to the total documents currently re- ceived in that category during a year. All of the documents concerned tions with other countries. For documents which contained multi-country/ subject content, the rule was established to index that material which would normally be processed by an operational Consistency was dif- 25X6 25X6 25X6 EVALUATION OF EXPERIMENT - 276 - Surninary Approved For Release 2000/055KRER-RDP785:13:952A000100050001-7 Approved For Release 2000/05/30: 65cligT178-03952A000100050001-7 obtain here because the indexers had slightly different interpretations as to what a 25X1A would process. It was decided not to apply any selection criteria, but to index all of the information concerning As a result, many low- level personalities and installations, as well as fragmentary subject matter, were indexed which would not be captured in an operational system. No selection criteria were applied because it was felt that realistic criteria could not be established prior to the experiment and that artificial criteria' would affect the experimental results. It was further felt that great indexing depth would aid in estab- lishing future criteria--that is, that the experi- ment would. show that redundant indexing of many sub- jects is unrealistic. However, despite the lack of criteria, an indexing consistency test following the experiment showed that each indexer tended to apply his own criteria based on his views of what was important. 25X6 EVALUATION OF EXPERIMENT SUM:Clary 5.B.1. - 277 - Approved For Release 2000/05/30 : CI4KIXE1T8-03952A000100050001-7 Approved For Release 2000/05/39BapROP78-03952A000100050001-7 The documents were broken out into the four topical categories mentioned above. Each senior controlled the flow of material to his indexers thus assuring that each processed a variety of sources. Upon completion of the indexing, the seniors re- viewed the transcript sheets for accuracy and logic. However, many errors were not caught because neither the indexers nor the seniors were as well versed in the system as would be desirable in an operational system. In fact, it would be fair to say that it was not until the end of the experiment that the indexers and seniors were beginning to gain confi- dence in what they were doing. In addition, severaL of the indexers were not suited to the task and would have to be given other assignments in an operational' system. Following review by the seniors, the documents and transcript sheets were transmitted to the three typists for header data transcription. One of these clericals acted as a senior for resolving problems. In addition, an OCS system analyst who had planned - 278 - Approved For Release 2000/0MRRA-RDP78-03952A000100050001-7 EVALUATION OF EXPERIMENT Summary 5.B.1. Approved For Release 2000/05/30: CniS8-03952A000100050001-7 the header data transcription task oversaw this phase of the operation. The documents were then filed by a CHIVE accession number, and the tran- script sheets were transmitted to key punching. Computer processing resulted in a print-but of index records which contained errors. These listings were reviewed by one of the project monitors and fina corrections were made. - PRELIMINARY FINDINGS The final results of the experiment await the conclusion of the query phase. However, prelimi- nary findings relating to indexer reactions, se- lection -problems, indexing times, etc., can be described, and these are perhaps the critical factors affecting the organization of the proposed CHIVE system, 5.B.2.1. Personnel Considerations The personnel involved in the experiment were college-graduate professionals and less than half had worked in jobs that involved full-time indexing. Even those with an OCR indexing background, had - 279 - EVALUATION OF EXPERIMENT Preliminary Findings 5.B.2.1. Approved For Release 2000/05/30 : CIAWfil-03952A000100050001-7 Approved For Release 2000/05/3EGRINRDP78-03952A000100050001-7 worked or allied. taskssuch as querying or diction- ary building, or had served as experts on some aspect of indexing. In this experiment they did nothing but Index and found the tools and rules with which they eori+, I 0 0 0 0 0 0 0 0 0 0 0 0 0 114 00000000000000000000000000000000000000000000000000000D000000001 , 73 4%411 4101112131515q(617141 avnz 2525262/2020353i32133435361383350515753555555571459505125.155554647585960610/6365655567586970/1/2/3751575/115/ 1111111111111111111111111111/1 111111111111111111111111111111111111111111111111 1 ' 21122222227222222222222222222222222222222222222 2 22222222222222122222 2 22222 2 2 : 1 333 3 33 3 3 3 33 3 33 1 33 3 3 3 3 3 Vfrsin intdEprineA wn r27P.T., 334 31 )3333 3 3 3 3 33333 3333 3 3 3 : 3 ; 14 4 4 4 4 4 4 4 4 4 4 4 4:4 4 4 4 4 4444444444444444444444444444444444444444444444444444444444d t 1 55555555555555555555555555555555555555555555555555555555555555555555555555551 , 666666666666666663666666G666666666666666666668666666G666666666 1 1 1 7 7 1 1 1 7 ) 1 1 117 1 1 1 7 1 7 7 7 1 7 1 7 7 7 7 7 1 7 7 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 1 7 7 7 7 7 7 7 7 7 7 7 7 6 3666666666666 E 1 7 7 7 7 7 1 1 1 1 1 11/1 8888888888888:88888888888888888888888888888888888 8 8 8 8 8 8 8 88 a s 88888888888888888i 999999999999 9:9 999999999999999999999099999999999999999999999999 5 9 999999999999F. 11 4 5 , 4 5 W1112.114.50 '41.0252 72 225'24252221183031473311363631839545152535555565754505051:.25155555451u536441626.165 4 ;4.54.6474/. r,.1.,. .5N,1.1, - 514 - Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 4 a 1 2 3 4 5 6 7 9 CONFIDENTIAL Sag Approved For Release 2000/05/30 : CIA-l -03952A000100050001-7 Figure 5.D-15 JOB 3 (KWIC) ELEMENTS OF INFORMATION Col. Field Name 1 File 2-15 Document Identity, Series & No. 16 Year Date of Information** Content Description The numbers (1-4) which identify the exploded punch card records. Numbers are suppressed in printout. Clear-text transcrip- tion of the document series and the docu- ment number. Series and number are sepa- rated by a blank space. Year of publication of the report taken from the documentation. The numbers 0 to 9 are com- bined with over punches to develop 2-digit year on printout. X overpunch for 4 No overpunch for 5 n overpunch for 6 X overpunch for 7* 17-18 From Month 0-12 19 Year as in 16 above 20-21 22 * * To Month 0-12 Year as in 16 above Corresponding CHIVE Field None Card 03 - Doc/ Series identifi- cation no. includes series, no. & year. The digraph sub- ject portion of the series is also carried in field 02-02. 01-07 first subfield 01-07 second subfield All year fields for Job 3 are one column and a 2-digit year is developed in the same manner for all. No information date on PI publication date appearing on the date line is entered in cols. 20-22. Approved For Release 2000/05/30 : Cl - 515 - 03952A000100050001-7 NciDENTA Approved For Release 2000/05/ Figure 5.D-15 (Cont'd.) Col. Field Name 23,37, 51,65 24-36 38-50 52-64 66-78 79 80 Subject seg- ment Keyword Code Clear aext aiig150001-7 Content Description 3 or 0 are used to show whether or not the word following is a keyword and, therefore, to be printed in the alpha list of keywords which comprise the index. 3=Keyword. 0=Non-Keyword* The clear-text words taken from the docu- ment. Some but not all are dictionary controlled. a. SI Reports. The number 2 identifies Cuban reports. The number 3 identifies UAR reports. b. PI reports. One of 6 codes C,K,S,T, Z,N used to indicate the security channels in which the document is being handled. For multiple channels, highest indicator is used. Distribution control symbols 1-7. Corresponding CHIVE Field 01-14 01-02 and 01-03 No one-to-one relation with CHIVE code. 01-04 No onP to-one equivalency. CHIVE code could be easily expanded to accommo- date these entries. * All Keywords are dictionary contr-lled. See sections 5 and 6 for sample pages from China area book and Job 3 dictionary. - 516 - Approved For Release 2000/0 tk-RDP78- Approved For Release 2000/05/30 :CI 6 952A000100050001-7 ZW014 CONFIDENTIAL Figure 5.D-16 FIB TOWN/CITY INFORMATION CARD FORMAT Col. 1-3 1. FIB Country Code 4-6 2. FIB Political Subdivision Code 7-30 3. Location Name 31-32 4. 200 Chart Series 33-35 5. B. E. - WAC Number 36-40 6. B. E. - Town Number 41-42 7. Degrees (N/S) 43-44 8. Minutes (N/S) 45-46 9. Seconds (N/S) (South "X") 47-49 10. Degrees (E/W) 50-51 11. Minutes (E/W) 52-53 12. Seconds (E/W) West "X") 54-55 13. Date of Latest Information (Yr) 56 14. Source Code 57-64 15. AMS Chart Number 65-69 16. Location Identification Code 70 17. Town Card Indicator ("X") 71 18. Town Information Indicators 72-74 19. Cat. Design. Code 75-80 20. Town "C" Code - 517 - Approved For Release 2000/05/30 : CIAArati103952A000100050001-7 CONFIDENTIAL Approved For Release 2000/0-5 .114. a r 6.91. -RDP78-03 Figure 5.D-17 FIB INSTALLATION INFORMATION CARD FORMAT Col. 1-3 1. FIB Country Code 4-6 2. FIB Political Subdivision Code 7-11 3. Location Identification Code 12-35 4. Installation Name 36-40 5. B. E. Installation Number 41-42 6. Degrees (N/S) 43-44 7. Minutes (N/S) 45-46 8. Seconds (N/S) (South "X") 47-49 9. Degrees (E/W) 50-51 10. Minutes (E/W) 52-53 11. Seconds (E/W) (West "X") 54-55 12. Date of Latest Information (Yr) 56 13. Source Code 57-64 14. FIB Identification Number (Firm #) 65-70 15. Installation Identification Code (ICC) 71 16. Installation Use/Assoc. Indicators 72-74 17. Cat. Design. Code 75-80 18. Installation "C" Code Approved For Release 2000/05 ri\i\lor? -RDP78-03952A000100050001-7 cc) Approved For Release 2000/05/30 : CIA?8-03952A060100050001-7 Figure 5.D-18 FIB LOCATION CROSS REFERENCE CARD FORMAT Col. 1-3 1. FIB Country Code 4-6 2. FIB Political Subdivision Code 7-11 3. Location Identification Code 12-14 4. "See" 15 5. (Blank) 16-35 6. Location Cross Reference Name 36-40 7. (Blank) 41-42 8. Degrees (N/S) 43-44 9. Minutes (N/S) 45-46 10. Seconds (N/S) (South "X") 47-49 11. Degrees (E/W) 50-51 12. Minutes (E/W) 52-53 13. Seconds (E/W) (West "X") 54-69 14. (Blank) 70 15. Cross Reference Card Indicator("12") 71-80 16. (Blank) - 519 - Approved For Release 2000/05/30: CIA-Rtionair3952A000100050001-7 Approved For Release 2000/05/30 ? CIA-RDP7 SECRET.- 00100050001-7 s Figure 5.D-19 FIB ICF COORDINATE CARD FORMAT Col. 1-7 8-28 1. 2. Sequence Number Location 29 3. Country Code (Target "X") 30-31 4. Country Code 32-35 5. Political Subdivision Code 36-40 6. (Blank) 41-42 7. Degrees (N/S) 43 8. (Blank) 4A-45 9. Minutes (N/S) 46 10. "N" or "S" 47 11. (Blank) 48-50 12. nglgrees (E/W) 51 13. (Blank) 52-53 14. Minutes (E/W) 54 15. "E" or "W" 55-58 16. "APPR" (If Approximation) 59 17. Irclank) 60-63 18. WAC Number 64-69 19. (Blank) 70 20. Control "X" 71-72 21. (Blank) 73 22. Control "X" 74-75 23. (Blank) 76 24. Town Folder Indicator 77-79 25. (Blank) 80 26. Card Type "1" ,CONFIDENTIAL ..$,EqET Approved For Release 2000/05/30 :-CIA-RDP78-03952A000100050001-7 sEcieT CONFIDENTIAL Approved For Release 2000/05/30: CIA-RDP78-03952A000100050001-7 Figure 5.D-20 FIB ICF CITY CROSS REFERENCE CARD FORMAT Col. 1-7 8-28 29 30-31 32-35 36-38 39 40-63 64-79 80 1. Sequence Number 2. Location 3. Country Code (Target "X") 4. Country Code 5. Political Subdivision Code 6. "See" 7. (Blank) 8. Location 9. (Blank) 10. Card Type "2" - 521 - Approved For Release 2000/05/30: CIA-RDP78:b3952A664F44*NTIAL Approved For Release 2000/05/305,60"kbP78-0MAIDIKIN?O?QQ01-7 rinutiV I IAL Figure 5.D-21 FIB ICF NAME CARD FORMAT Col. 1-7 8-28 1. 2. Sequence Number Location 29 3. Country Code (Target "X") 30-31 4. Country Code 32-35 5. Political Subdivision Code 36-63 6. Firm Name 64-67 7. Plant Number 68 8. Status "X" 69-75 9. Firm Number 76 10. Plant Folder Indicator 77-79 11. Industrial Category Code 80 12. Alpha - 522 - Approved For Release 2000/0 keTA-RDP78-03 EoRAIAL Approved For Release 2000/05/30: i 8-03952/QV Figure 5.D-22 FIB MODEL-TYPE/BROCHURE INDEX CARD FORMAT Col. 1-20 21-59 1. 2. Model Type/Series Descriptive Name 60 3. ?Tech. Material Indicator 61 4. Tech. Material Language* 62-63 5. Industry 64 6. Category Code iICC 65-71 7. Dossier Number (Firm #) 72-73 8. Date (Month) 74-75 9. Date (Year) 76-78 10. Country Code 79-80 11. USSR Area Code *Admissable Entries are: (1) English (2) Native Language (3) Other Approved For Release 2000/05/30 : CIA- )0___NTIDENTIAL i3952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Figure 5.D-23 PUNCH CARD CHARACTERISTICS of the IRS Document Index File (New) E-f 4.1 ? Subject Code st Subject Modifier E-4 Code 4 a a Clear Text E-4 4 o Subject 0 4 Organization E-4 ? Abbr. Place Name 0 ul Area, Code Source Code 4 4 A Document No. A 4 H Pub. Date 8 4 Classification Code * NOTE: Numbers indicate action codes. These are literal entries. - 524 - Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Figure 5.D-24 PUNCHED CARD CHARACTERISTICS OF THE IRS DOCUMENT INDEX FILE (OLD) Fields 1 1 - 6 2 7 -1112 3 4 13- 14 5 15- 18 6 19-21-23- 20 7 27 8 2C 9 26-32 Punch N pos. ,Data Subject Code Subject Modi- fier (Action Code) Area Code Classification Code Source Code Locator No. Related Area Code Related Area Code Pub. Date Control No. 031 , . - 525 - Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Figure 5.D-25 PUNCHED CARD CHARACTERISTICS OF THE FILM INDEX FILE 1 Fields 1 1 - 6 2 7-1012 3 11- 4 1314-1E20 5 6 19- 7 2122-2E27 8 9 Punch Data pos_ Subject Code ONMOMM Area Code Text (Language) AMO Code Type Code Mi Holding Agency Code Pub. Date (Yr) MOW Classification MI Code Title No. alliMil (Control No.) Availability WM Code - 526 - Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7 Approved For Release 2000/05/30 : CIA-RDp78-03952A000100050001-7 seat(' CONFIDENTIAL CON IDEF, CREt Approved For Release 2000/05/30 : CIA-RDP78-03952A000100050001-7