Library

 

Appendix: The Analytic Pathologies Methodology

Appendix:

The “Analytic Pathologies” methodology expands the typical portrayal of the five-part intelligence cycle (planning and direction, collection, processing, analysis and production, and dissemination) to include seven elements in the analytic process:

  • question specification and subject understanding

  • evidence acquisition and situation perception/assessment

  • problem/hypothesis specification

  • decision metrics and procedures

  • analysis, assessment, and answer preparation

  • communication with the client

  • feedback and post-hoc assessment.

More consistent with current research on judgment and decisionmaking, this expansion permits a closer look at the analytic process itself. This approach also makes it possible to capture explicitly the important calibration functions of interactions with clients in framing questions and in delivering answers, and it highlights the importance of explicit feedback for self-assessment. In this approach it is also possible to identify more completely the impediments to analysis and to specify the location and distorting effects of each step in the process.

The Pathologies Map

Layers of Pathology

Finally, the methodology explicitly lays out the key processes and functions (as well as impediments) at three organizational levels: individual analysts, work groups and agencies, and the Intelligence Community as a whole. This layout enables investigators to examine the causes of analytic problems and locate their organizational sources; these problems can occur not only where they directly interfere with the analytic process, but also in the functions which support agency and community analytical processes. With that information, appropriate corrective measures can be identified more easily.

Thus, although the Intelligence Community organizational level provides direction, priorities, and controls budgetary resources—which are important support mechanisms and enablers of output functions—it does not itself directly perform any intelligence functions, except analyses conducted by the National Intelligence Council (NIC), now a part of the office of the Director of National Intelligence.

The effectiveness of the Analytic Pathologies methodology as a diagnostic tool suggests its potential utility when used as a framework for developing corrective measures. The diagnostic effort highlights the importance of properly identifying the organizational level at which impediments impinge upon successful analytic practice, as well as the level at which corrective actions must be taken to restore important functions—and these are often not the same. For example, there are a substantial number of impediments to effective performance by individual analysts, but the appropriate corrective measures need to be instituted at the group or organizational level instead of the analytical level. Different corrective measures also have different time constants of effectiveness to be accommodated, both in planning and in measuring improvement; otherwise, phasing conflicts could easily induce resonant oscillation in processes and control mechanisms.

The explicit differentiation among these elements allowed by this methodology can enable individual and organizations at all levels to develop tools to recognize and use pathology self-assessments. It also allows the development of appropriate measures of effectiveness or indicators of performance targeted at relevant outputs rather than inputs. Thus, extending the Analytic Pathologies framework to use as the basis for developing corrective measures offers promise that it may be able to identify and, therefore, avoid perverse consequences that might result from altering structures and processes while attempting to make them more effective.

 

 

IC Functions and Outputs at Three Levels

Level Key Functions Outputs

Intelligence Community

  • Direction and Leadership

  • Setting Priorities

  • Budgetary Control

  • Policies

  • Oversight

  • Budgetary Allocation

Work Group/Agency

  • Training and Acculturation

  • Work Practices and Habits (TTPs)

  • Rewards and Incentives

  • Review Processes

  • Organizational Knowledge Base

  • Review

  • Production

  • Dissemination

  • Customer Interfaces

  • Work Processes

  • Technical Infrastructure

  • Organizational Culture
Individual
  • Analytic Expertise

  • Domain Knowledge

  • Answers

  • Hypotheses

  • Knowledge Products

  • Increased Knowledge

  • On-demand Expertise

 


Historical Document
Posted: Mar 15, 2007 04:11 PM
Last Updated: Jun 27, 2008 10:00 AM