Chapter 6: Interacting with Collectors
To enhance communication and collaboration between the analytic and
collection communities, to develop an integrated information
needs/requirements process, to develop a rapid data integration
capability, to help guide and inform future collection strategies and
acquisitions within a fast-changing target environment.
- National integrated intelligence requirements process that links or replaces existing community committees and cuts across disciplines.
- Integrated collection management tools and systems in transition or under development to improve visibility among collection managers, facilitate trade-offs, and provide the connectivity needed for efficient collection coordination among the various collection disciplines.
- Integrated strategies for exploitation of open source information.
- Comprehensive, community-wide evaluation program for analysis and collection.
- Analytic community engaged in decisions about requirements for future collection systems.
- Community-wide training to educate analysts about collection systems and capabilities and encourage greater analyst involvement in development of integrated strategies among the various collection disciplines.
- National requirements process running smoothly. Analysts have the tools and ability to task and monitor status of collection requests; improved integration and collection strategies among all collection disciplines are the norm.
- Cross-community evaluation tools provide Intelligence Community managers with data necessary to weigh trade-offs and make hard resource decisions.
- Collection training mandatory at several stages of analysts’ careers and collection-related rotations part of normal career progression.
- Effective open source strategies firmly established.
- Broad ranging, rapid integration and reporting from multiple sources in a virtual, collaborative network.
A close and continuing relationship between the analytic and collection communities is imperative if we are to develop effective collection strategies against increasingly diverse and hard-to-penetrate targets. Analysts must play a more active role in the targeting, evaluation, requirements, and acquisition process. In short, we must:
- Train our analysts in the capabilities of the various collection systems, participate more actively in the development of collection strategies, provide more frequent and effective feedback to the collection community, and help ensure that new systems are designed to respond to likely long-term priorities.
- Implement a comprehensive collection evaluation program to measure how effectively collection meets analytic and customer needs, better balance the demands for global coverage, warning, and crisis support, and make more informed trade-offs among resources devoted to various collection systems.
- Establish an efficient requirements process that better conveys customer needs, improves customer-collection manager collaboration, facilitates development of multi discipline collection strategies, and ensures that future systems are designed to respond to likely long-term priorities.
- Provide our analysts with the resources and tools necessary to exploit the increasing volume of collected materials.
The Evolving Collection Environment
The collection environment has undergone a fundamental transformation in the last ten years and is continuing to change in ways that pose an unprecedented challenge to the US Intelligence Community. Our targets are becoming more mobile and agile, and they are less dependent on the traditional communications infrastructure.
At the same time, the business and intelligence worlds are moving closer together. Commercial companies are breaking into areas that previously were the sole purview of the Intelligence Community, and the intelligence communications and work environment are moving to the internet
The improvements in technologies—and their increasing accessibility to friends and foe alike—are greatly complicating the threat environment. Strategic adversaries, rogue states, terrorist organizations, narcotics traffickers, organized criminals, and other transnational actors, are increasingly exploiting modern technology to operate, communicate, and move about in ways that challenge our collection capabilities.
Open Source: A Special Case
Harnessing open source information is a key challenge today and will be tomorrow because there is so much of it, and because a lot of it is critical to our needs. Traditionally, open source meant the Foreign Broadcast Information Service’s (FBIS) daily catch of press and other media, which provided useful context and color to intelligence analysis. Today, open source material of relevance to analysts working in a dispersed threat environment is dauntingly voluminous, and the Intelligence Community is not keeping up with it.
Exploiting open source information requires a broad, multifaceted strategy: full access to the Internet and skill in using it, state-of-the-art software and analytic tools to mine the Worldwide Web; partnerships with commercial vendors to keep pace with rapid advances in information technology; and a more mobile cadre of analysts who are prepared to engage comfortably—and effectively—with academics, scientists, businessmen, and others with infor mation or expertise relevant to our needs.
Today, the open source community, led principally by FBIS, is making some headway in bounding the problem, but it does not have the resources and is not empowered to meet the challenge. Many other initiatives touch on the open-source problem, but they are neither coordinated nor comprehensive, and they fail to address the problem squarely. The NIPB has made the development of an Intelligence Community strategy for open source a top priority for investment and concerted action over the next few years (see chapter on External Analysis).
Analytic Support to Collection Strategies and Targeting
One of the biggest problems we face as an analytic community today is helping our analysts better understand the collection management process and the capabilities of various collection systems. Yet, the complex world that we must analyze, and the difficult and often technical issues that we must address, demand that our analysts become more actively involved in tasking collection systems and developing integrated collection strategies.
Most components of the analytic community— as well as many of the collection agencies— have made at least a modest effort to bolster their education programs. Training, however, is only a first step in enabling and encouraging analysts to provide more input into decisions concerning intelligence collection. We must establish mechanisms through which analysts can help collectors develop near- and long-term collection strategies. The Community’s record in this area is mixed but improving.
As an analytic community, we need to take a hard look at these activities and how they relate (or should relate) to one another. And we need to identify other programs or processes we should be fostering to develop closer ties between analysts and collectors.
Evaluating Collection: Are We Satisfying Customer Needs?
Evaluation must be an implicit part of any good collection strategy, and the analytic community—as a direct customer of collection and a critical interface with the military, policy, and law enforcement communities—must play an active role. Evaluation needs to take place on several levels. At the micro level, there must be feedback mechanisms to indicate whether specific information needs have been satisfied and whether the intelligence provided was timely and relevant. At the macro level, we need to answer the larger questions: “What does all this add up to; did we collect the intelligence necessary to answer key questions and make sound analytic judgments; have we enhanced our consumers’ collective knowledge about an issue or problem?”
Current collection evaluation tends to occur within, but not across, collection disciplines. Feedback mechanisms vary in effectiveness and generally focus on the micro level. In addition, these efforts by definition consist of collectors overseeing evaluations of their own product. This is not necessarily bad, but it can raise questions about objectivity—particularly when collectors are competing for scarce resources—and it only indirectly provides Community managers with insights into overall or relative performance. In short, discipline specific evaluations are of limited utility to Community managers trying to make decisions about tradeoffs or synergies among collection programs.
The identification of intelligence gaps has sometimes been a byproduct of Intelligence Community studies, but these studies were not designed to provide a systematic assessment of collection performance and customer satisfaction. In addition, the Community has from time to time prepared “lessons learned” assessments of its performance on specific countries or issues. These types of studies have proved useful and have been well-received by national and defense consumers and by Congress.
In the past several years, the Intelligence Community also has begun to address evaluation at the macro level—across the various collection disciplines and across topics—and to develop the kind of quantitative and trend data that should help managers assess collection strategies, customer satisfaction, and allocation of resources more effectively.
Clearly, the Intelligence Community has a way to go in developing a useful and comprehensive evaluation program that cuts across collection disciplines. We must strengthen well-conceived and proven tools already in use but also develop other methodologies to address collection performance. Such evaluations must take the entire collection cycle into consideration so that not only deficiencies in collection are identified but also deficiencies in processing and exploitation of intelligence. As a key customer of collection and as the critical interface between the IC and its consumers, the analytic community must be at the center of the effort.
Managing Collection Requirements and Tasking
The way the Intelligence Community manages collection requirements remains complex and inefficient. National priorities documents provide only a framework of general priorities. The Intelligence Community has no overarching process for integrating and prioritizing information needs and turning them into collection requirements. Instead, we have multiple sets of standing requirements that are specific to each collection discipline, and that take only modest account of potential contributions from other disciplines.
All of the collection disciplines have begun to recognize that success, both as individual organizations and as an Intelligence Community, will depend increasingly on more collaborative management of collection requirements and development of coherent multidisciplinary collection strategies, and that the analytic community can be an asset and an ally in working these issues. Partly for this reason, most have taken steps to improve the accountability and visibility of their requirements processes and systems.
Laudable as many current efforts to improve the requirements and tasking process are, they go only part way toward resolving some fundamental shortcomings. They do not, for exam ple, reduce the complexity and diversity of “front door” entries into the collection require ments world. They also do not provide integrated collection management tools and processes to improve interaction among collection managers, facilitate trade-offs, and provide the connectivity needed for efficient cross-discipline collection.
A National Integrated Intelligence Requirements Process
Developing a requirements process that cuts across the collection disciplines and assessing the level and effectiveness of overall collection efforts remain extremely difficult. The ADCI for Analysis and Production and the ADCI for Collection are working together and with other components in the Intelligence Community to achieve a more coherent and integrated front to-back-end collection management process. During the past year, a number of new initiatives have emerged:
- The two ADCIs have provided initial funding for a collection management system to be used by all of the collection disciplines. The system will provide an integrated collection management capability—enabling information sharing among the various collection communities and users of intelligence data. It should improve interaction and enable collection managers to collaborate directly with planners, analysts, collection system operators, and exploitation and dissemination specialists to ensure that requirements are met rapidly and efficiently. The system will provide users with a capability to track the status of their requirements and to adjust their collection requirements to rapidly changing intelligence needs and priorities.
- The new board was established last year under the chairmanship of the ADCI for Administration and vice-chairmanship of the ADCI/AP and ADCI/C to oversee the development of national requirements for future systems. The board serves as the DCI’s focal point for identifying future intelligence needs within the national security missions and as customer advocates for those needs as they relate to Intelligence Community strategic planning, programming, and acquisition decisions. The board takes the long term view in defining and prioritizing needs, and constructs detailed system requirements documents related to the acquisition of intelligence capabilities.
These efforts are steps in the right direction. The analytic community, however, must now help sustain such activities and work together with the ADCI/C and the collection community to ensure that strategic and near-term analytic priorities inform and drive collection requirements and management. In short, the time is now ripe for a new national integrated intelligence requirements process that will link the activities described above and provide the critical interface between analysis and collection, information needs and collection requirements, and analytic priorities and long-term acquisition strategies.
Strengthening our Processing and Exploitation Capabilities
Technology has greatly increased the amount of information the Intelligence Community is able to collect, but our ability to process and exploit that information has not kept pace—and in some cases has fallen behind previous levels. Previous IC assessments have noted that the revolution in information technology and telecommunications has changed the imagery and signals intelligence worlds from collection constrained to exploitation-constrained environments. We have not been able to take advantage of potentially valuable information because we do not have the resources to exploit it.
Despite the front-loading of analytic manpower in collection organizations, our capabilities in these areas remain inadequate. The problem is due only in part to the personnel cuts of the last ten years—cuts that have affected the entire analytic community. The problem also lies in an aging infrastructure and lack of investment in the technological tools and expertise needed to exploit the emerging global net. We must address this issue as a Community because it will have a significant impact on our ability to exploit fully the current and future capabilities of our collection systems and ultimately serve our policy, military, and other customers.
An Investment Strategy for Enhancing Interaction with Collectors
Analytic involvement in the collection process—including targeting, evaluation, set ting priorities, tasking systems, and processing and exploiting collected information—will require a significantly greater commitment in people and resources than is currently the case. We need to leverage these resources carefully— participating in and strengthening initiatives that promise the greatest payoff. As top priorities, we must:
1. Establish a Community-wide training program to educate analysts on collection management and systems. Our analysts must understand collectors’ needs, capabilities, and constraints to participate more actively in the collection process. Collection training should be made a mandatory part of the career progression for all analysts, and those who develop the requisite collection management skills should be recognized and rewarded professionally.
- Evaluate existing collection training activities in the Community and baseline resources currently devoted to them. Identify best practices and/or centers of excellence. Establish an IC-wide curriculum on collection programs, building—where possible—on training already offered by individual agencies. Start with introductory courses and training for new recruits. Follow with advanced training on collection management and the development of collection strategies. Designate core courses that should be mandatory. Tie to “virtual university” initiative in Chapter 2 (Investing in People).
2. Strengthen the role of analysts and promote collaboration with collectors in the development of collection strategies—including for open source information—and the acquisition of future collection capabilities. Analysts and collectors must interact more closely in determining collection shortfalls, identifying future needs, and developing remedies to better guide collection and respond to intelligence gaps. This area is likely to require major resource investment, and we therefore need to take a hard look at how best to manage our resources. This involves examining existing efforts that use analysts to help in collection targeting and new initiatives designed to improve collection effectiveness through cross discipline planning and development of innovative collection methodologies.
- Develop a comprehensive and focused open source strategy to exploit this growing—and increasingly important—source of information and expertise.
- Conduct a Community-wide review of ongoing efforts to improve collaboration between analysts and collectors. Identify best practices and lessons learned. Review possible new areas for analyst-collector interaction.
- Establish aggressive program in which analysts serve rotational tours in collection agencies and DCI centers specifically to work on collection targeting, strategy, and systems acquisition issues. Look for innovative approaches—including short-term rotations of one-, two-, or three-months’ duration. The objective is to complement collection training, enhance analyst-collector inter action, and provide real input on specific collection problems where most needed. ADCI/AP will work with the ADCI/C and others to identify issues/areas where analytic resources can be used most effectively.
- Benchmark additional analytic resources necessary to support collection-related activity (ADCI/AP action with input from NIPB agencies). Ensure that these activities are tied to any new national integrated intelligence requirements process.
3. Develop better methods for evaluating collection and measuring satisfaction of customer needs. We must have an evaluation system that allows us to assess how well we are doing to satisfy our customers and fill intelligence gaps, and that helps us make informed decisions about difficult tradeoffs between collection platforms and future acquisition capabilities. This evaluative process must also help us accurately determine whether deficiencies are due to collection activities and capabilities themselves or to shortfalls in processing and exploitation of the information collected.
To do so, we need a set of complementary evaluation programs that provide micro-level data on satisfaction of specific requirements as well as macro-level data on performance across the collection disciplines; information on the performance of individual collection systems on both an absolute and relative basis, but also assessments of how we are doing as a Community to address critical needs. In other words, we need both broad-based studies that cut across issues and collection disciplines, and in-depth studies of single issues and individual collection disciplines. Finally, we need longitudinal studies that allow for trend analysis as well as narrowly focused studies that provide valuable "lessons learned" in collecting against specific targets or issues.
- Establish a steering group under the chairmanship of the ADCI/AP to provide over sight of a Community-wide and multifaceted evaluation program. The steering group will oversee/monitor the efforts described below. ADCI/AP will provide an annual update to the DCI, DDCI/CM, and the NIPB on the overall evaluation effort.
- Advance the use of existing agency collection evaluation methodologies on the relative performance of collection systems and plat forms over time. Expand to include other organizations where appropriate.
- Review/build upon the recent effort of the ADCI/AP and ADCI/C in developing a joint annual report on the state of the Intelligence Community. This review should cut across agencies and be based on a consistent, repeatable methodology. It should provide an in-depth evaluation of how well we have performed in meeting customer needs and filling critical intelligence gaps on a series of key issues. It should take into account competing requirements and identify short falls in the collection, processing, and exploitation cycle. It should also provide actionable recommendations to overcome identified shortfalls.
- Appoint blue ribbon panels—ideally a mix of inside and outside experts—to prepare ad hoc "lessons learned" studies on event-driven issues or topics of critical concern to our customers.
- Establish a cross-agency working group to explore types of electronic feedback/evaluation mechanisms in use or planned as part of collection management systems currently under development (e.g., audit trails, site visit measures, pop-up screens, and mandatory versus voluntary evaluation menus). Cross-fertilize with activities underway in analytic organizations as part of production reengineering efforts. Evaluate commercial methods and software that might provide easy and consistent statistics across the Community on customer usage and satisfaction and on demand-to-response ratios for standing and ad hoc intelligence requirements.
4. Develop a National Integrated Intelligence Requirements Process. We must establish a single process to integrate and prioritize requirements across disciplines and mission areas. Without such a process, collectors will be left to integrate and prioritize requirements within their separate stovepipes; the analytic community will continue to have poor visibility into the status of collection; the Intelligence Community will be deprived of a means to make efficient trade-offs across platforms and manage future acquisitions; and non-military national information requirements will continue to compete for collection satisfaction on an individual basis with the vast quantity of military requirements that are fully integrated and prioritized.
This process must allow Community analysts to submit, integrate, and prioritize information needs for all collectors and provide analysts and collectors visibility into the status of col lection through the entire tasking, processing, exploitation, and dissemination cycle. It must include a leadership entity with responsibility for adjudicating conflicts on intelligence priori ties; assuring balanced collection; identifying future requirements; and supporting strategic planning.
- Build on recent studies to map the existing requirements process. Based on these stud ies, develop a framework and processes for integrating and prioritizing intelligence needs.
- Establish an Annual Integrated Information Requirements Plan that will provide prioritized direction to the ADCI for Collection’s annual plan for allocating collection. Marry this plan to software that will allow dynamic updating of priorities.
- Work with existing individual Community collection committees and/or mechanisms to provide: Community-recognized, integrated information priorities to the collectors; substantive guidance during surge situations; and integrated and prioritized future information needs to the new future requirements board.
- In conjunction with the above, monitor and support the pre-acquisition efforts of the integrated IC collection management system and of the agency requirements systems.
5. Ensure adequate resources for the processing and exploitation of collected intelligence. No matter how sophisticated or capable our collection systems are, they remain of little value if we are unable to process and exploit the information collected in a timely and focused fashion. Our capabilities in technical processing and exploitation of information have suffered in recent years from cuts in personnel and lack of investment in infrastructure and expertise-building. Fixing this problem must be a top priority for the entire Intelligence Community.
- ADCI/AP work with NSA, NIMA, and others to ensure that analytic capabilities are addressed in agency-specific strategic plans and annual budget submissions.
- Ensure that investment in analytic tools development addresses needs of analysts in collection agencies, as well as those in other parts of the analytic community. Focus on tools that are interoperable and compatible across the Community. (Implementing actions are addressed more fully in the Technology chapter.)
- Develop a Community-wide capability that will flag events of importance as the information is collected and processed, as well as the necessary tools to allow analysts to integrate the data quickly with information from other multiple sources.
- Study the costs and benefits of establishing a Community analytic center to provide a prototyping or demonstration capability for data integration. Look for ways to leverage existing efforts.