A National Security Simulation Center
Rachel K. Hanig
and Mark E. Henshaw
The following essay was a winner in the
2007 DNI Galileo Competition, a program that awards authors of papers proposing
innovative solutions to Intelligence Community challenges.
authors argue that creation of a National Security Simulations Center would
strengthen the accuracy and insight of intelligence analysis, improve IC
collaboration, and create a testing ground for new analytic tools and methods.
quality of IC analysis is inconsistent, and the challenges to sustaining a
superior analytic track record look more formidable all the time.
Intelligence analysis too often is like
investing in the stock market—past performance is not an indicator of future
results. The quality of IC analysis is inconsistent, and the challenges to
sustaining a superior analytic track record look more formidable all the time.
The bar has always been set high and is moving higher as policymakers demand
- be “timely”—at least on par with the public media;
- be analytically correct 100 percent of the time while
offering broader strategic views that include longer lists of potential
- be strategically relevant on increasingly complex topics
as the volume of raw information to filter and analyze grows.
This pressure for increased speed,
accuracy, and consistent strategic relevance is one of the primary factors pushing
the analytic corps towards risk aversion and its analytical consequences. Under
the best of circumstances, even the most experienced IC analysts, those with
years of study and experience invested in single accounts, make mistakes by
falling prey to mental biases and mindsets, intelligence gaps, or even “lack of
Given uneven hiring cycles in the IC’s
ranks over the past few decades, it won’t always be the most experienced
analysts making the judgments upon which policymakers might rely.
Initiatives to Improve Analysis: Building Blocks for a Larger Solution
IC has responded to these challenges with three major initiatives. The first
came immediately after 11 September 2001 with a call for more diligent
adherence to analytic tradecraft “best practices.” The problem was and remains
that there really are few standard methods of analysis. Analysts are left
largely to their own devices in developing systems for processing intelligence
and depend on coordination with other analysts to catch the errors.
second and most broad-ranging of these initiatives picked up steam after the
2003 Iraq WMD NIE fiasco. Several solutions, including a number of winning
Galileo papers, focused on giving analysts better access to data before
analysis occurs and promoting better coordination after the fact. Improving the
IC’s data organization and inter-and intra-agency sharing is a necessary but
ultimately insufficient first step.
information sharing and data access are always useful, but information sharing
and data access are not analysis. Even perfect access to perfect information
would be unhelpful if the analytical models used to process it were deficient,
and even perfect coordination among analysts might not be enough to guarantee
the models’ quality. So this begs the question: How can analysts stress-test
the quality of their analytical models, theories, and theses without waiting
for history to prove them right or wrong?
third major initiative promotes the use of alternative analytic tools and
techniques. Again, these are very useful. But the approach is potentially
flawed because many structured analytical tools and techniques are employed as
individual mental exercises. Their effectiveness can still be undermined by
sloppy thinking. Ironically, the analysts who need to use them most desperately
are most likely to use them ineffectively or incorrectly, or just not use them
at all. Nor can we guarantee that the coordination process will catch sloppy
application of alternative analytical tools in all cases since many senior
analysts, though experienced in traditional analytical tradecraft, are no more
experienced in the craft of alternative analysis than their junior
counterparts. Many senior analysts, in fact, prove to be the most resistant to
using such techniques.
three of the above initiatives are critical elements of a larger solution; but
even if all three were perfectly executed, analysts would still struggle to
meet several of the policymakers’ requirements during crises. Quality analysis
cannot be rushed. Strategic insights take time to develop, but when a crisis
breaks, the time for analysts to engage in deep thinking is often past.
[Top of page]
Solution: The National Security Simulations Center
solution that fuses all three initiatives together into a single whole and that
resolves the problem posed by the pressure for analytical timeliness would be
ideal. We propose that one solution is, ironically, both widely known and
little practiced by the IC, simulations.
[Top of page]
can be very effective in stretching analysis and strengthening the
methodological rigor that policy consumers value and expect. The use of
simulations is not new. The US military has used them for years, primarily as
training tools to help troops develop tactical and joint-service coordination
skills. It is unfortunate that the IC has used simulations for the same reason
only intermittently at best—there has never been a central, Intelligence
Community, simulation hub equivalent to the National Strategic Gaming Center at
the National Defense University in Washington, DC, or the Wargaming Center at
the Naval War College in Newport, Rhode Island.
simulations as have been conducted were usually performed under the purview of
individual agencies. However, the intelligence failures of recent years suggest
that the IC should be staging simulations for another purpose: to develop
strategic insights into potential geopolitical developments.
are not predictive, but they can allow analysts to explore key analytic
questions and conclusions in far greater depth than is possible from behind a
desk or in meetings with other analysts. A properly organized geopolitical
simulation forces analysts into dynamic, social, stressful situations that
simulate real-world conditions to expose the participants’ thinking, mindsets,
biases, and assumptions to colleagues and observers positioned to identify
simulations can also peel back the layers of intellectual cruft and weak
analysis to expose insights that might otherwise remain undiscovered—and do it
before real crisis hits, when there is almost no time for analytical
coordination and deep strategic thinking. In a sense, simulations give analysts
better ideas of what geopolitical changes might look like before having to
present their conclusions to policymakers.
[Top of page]
Why a national center?
shows that the preparation and execution of successful simulations are the
product of both structured analytic work and art requiring a large number of
expert people with a large variety of skills. The Intelligence Community would
greatly benefit from a center with a dedicated staff versed in the arts and
crafts of scenario development, construction of simulation tools and
methodology, and subject-matter experts, not to mention the support personnel
needed for such an endeavor.
Director of National Intelligence already has the charter, provided by Congress
in the Intelligence Reform and Terrorism Prevention Act of 2004, Section 1023,
119B, to create national interagency centers that focus on intelligence issues.
The National Counterterrorism Center and National Counterproliferation Center
are two current examples. However, a National Security Simulations Center
(NSSC) would not focus on any single issue that threatens US interests. Not
only could it address threats of all kinds, it could deal with other community
priorities, as seen below.
[Top of page]
NSSC could regularly stage large-scale simulations that would bring together
analysts and managers from multiple agencies. Such simulations would would give
participants opportunities to share information, ideas, theories, and best
practices in structured, realistic environments designed to push the
participants toward common goals.
this sense, the NSSC would function much like the NDU National Strategic Gaming
Center or the Naval War College Wargaming Center. Such simulations would teach
participants how to work together during crises, who to call, and the
capabilities of their IC counterparts. The personal connections developed in
such an environment would be highly useful during real crises, as participants
would better know who to call and would have practiced real-time coordination
with their counterparts.
the NSSC could stage simulations that go far beyond practicing tactical
responses to crisis scenarios. By having analysts participate in the scenario
development process, it would also become a strategic analysis
cross-pollination center. Previously proposed solutions to problems of
community coordination and integration could be field-tested in controlled
environments to determine their practicality and identify their strengths and
[Top of page]
Engagement of outside
simulation’s value rests directly on the quality of both the scenario and the
participants. Backed by the DNI’s authority and resources, NSSC simulations
could recruit high quality participants to lend expertise to scenario
development and to participate in the simulations. It is not unreasonable to
believe that former high-ranking government officials, corporate CEOs, leading
academic thinkers, and other notable figures—including foreign
participants—would be willing to participate in NSSC simulations. Their
involvement would improve strategic analysis across the board and strengthen
the outreach efforts of individual agencies, which now tend to be piecemeal and
ad hoc. This would ensure that outside expertise finds broader audiences and
becomes better aligned with the needs of individual agencies.
[Top of page]
Staying ahead of
media’s rapid response to breaking events leaves the IC at a significant
disadvantage in informing policymakers. The NSSC could help analysts remain
both timely and strategically relevant by simulating as many events as possible
before they happen, thereby buying analysts time that is irretrievably lost
once an event actually occurs. In that sense, properly organized and managed,
simulations could help analysts more quickly provide more informed perspective
addition, as a simulation looked into potential developments, players would be
in position to identify intelligence gaps and to begin developing targeting plans
to fill those gaps.
[Top of page]
Training new IC employees
cyclical nature of hiring in the intelligence community is well documented and
the number of analysts in the IC with less than five years experience has
reached record highs. The NSSC could take the training of new people beyond the
classroom by putting junior analysts into environments in which they could
learn and practice tradecraft without having to worry about making
embarrassing, or career-terminating, analytical errors or having their efforts dismissed
or ridiculed by policymakers.
analysis techniques, which can be difficult to learn and properly apply, often
lend themselves very well to being operationalized within simulations.
Alternative futures analysis, Team A/Team B, and several others are
particularly well suited for use in simulations.
NSSC would also help new analysts learn how to better process the vast amounts
of data available by teaching them how to determine what information would be
most valuable to them and their policymaking customers. And, as they considered
the relative importance of information, they could actually beginning mining
the data they would need in a given circumstance.
[Top of page]
any craft, intelligence analysis, and especially alternative analysis, must
experiment continuously with new tools and techniques. The NSSC would be
ideally suited to serve as a laboratory in which analysts could develop and
field-test tradecraft innovations before deploying them to the IC at large. In
fact, simulations might well point analysts towards new tools and techniques
that might otherwise remain undiscovered, or suggest new uses previously
unconsidered for existing tools. By increasing the frequency of interaction
among analysts focused on specific problems, the NSSC would improve the odds
that innovations could emerge from such social networking. The NSSC could be an
idea factory for experimental tradecraft.
sum, the NSSC could be an organization fit to play many roles in the community.
Which role it would play at any given time would depend on the kind of
simulation chosen for the particular exercise. Tradecraft training, strategic
insight development, and testing of analytical tools and techniques all could
be managed under the single roof of the highly flexible center.
are the one kind of exercise that can tie all other analytical tools and
techniques together, both new and old, while enhancing inter-agency
coordination at the same time. It’s difficult to think of any alternative
concept that even promises a way to enhance IC-wide collaboration and allow
analysts to develop strategic insights and perfect analytical tradecraft, all
in single endeavor. Practice makes perfect, but opportunities to practice all
three activities at once are, to say the least, rare.
[Top of page]
the National Security Simulations Center
outlined justifications for creating such a center, the questions become: What
should the National Security Simulations Center look like and how might it
NSSC would require, at minimum, four key organizational components (see graphic
on following page):
- A Research and
Analysis Staff (R&A)
- Simulations Design
- An Analytical Tools
and Techniques Development Staff (AT&TD)
Sector/Academia Outreach Staff (PS&AO)
[Top of page]
and Analysis Staff (R&A)
primary responsibility of the R&A would be to work with IC subject matter
experts—CIA analysts, NCS officers, and other IC members engaged in analytical
or targeting functions—to identify and craft intelligence questions suited for
scenario testing. This would require R&A to mount in-depth research
campaigns on underlying issue areas to identify three major requirements of
which must be observable and measurable in the real world by the IC; or if they
aren’t observable (and therefore not measurable) could become so through the
implementation of new technologies or collection programs.
the simulation designers could understand in advance where the holes in the
simulation scenario would be and how they could best be addressed.
including social, military, economic, diplomatic, and potential natural
disasters beyond the control of key actors.
a simulation is completed, R&A would be responsible for producing the
analytic product documenting its key findings. Using appropriate analytical
standards and tradecraft, the product would include key findings, warnings and
indicators, and analytic conclusions. These might include strategic projections
and key decision points and discussion of how things might have gone had
different decisions been made. This analysis would all be directed toward
extracting strategic insights that would give analysts and policymakers deeper
understanding of the issues they face.
[Top of page]
Simulation Design Staff
primary responsibility of SD would be to take polished analytical concepts
prepared by R&A and develop simulation scenarios to address them. SD would
devise scenario story lines and geopolitical conditions that would best
illuminate hidden assumptions, insights, and potential outcomes. SD would also
create game mechanics to move players through scenarios. Broadly speaking, this
would include identifying needed government, private sector, non-state and
state roles and organizing players and teams. SD would also be responsible for
creating supporting game materials— maps, manuals, and other accessories—and
driving development of the computer network that would be used to deliver to
players game injects and scenario information and that would provide the means
by which players and teams would communicate with each other and with
a simulation design phase is complete, SD would be responsible for conducting
the live exercise. Those who create simulation scenarios are usually best
prepared to adjudicate players’ actions within those artificial environments.
The skills of scenario designers and adjudicators directly affect the validity
of any simulation’s results. This is not an activity that can easily be taught.
Constructing plausible and useful present and future conditions for a
simulation and then managing the simulation is an art, not a science, and only
time and experience teach it. SD would develop expertise as it created
legitimate environments and judged players moves to ensure that simulation
results would always be credible.
[Top of page]
Analytical Tools &
Techniques Development Staff (AT&TD)
fulfill its mandate as an analytical research center, the NSSC would benefit
greatly from having a separate team of methodologists who could observe
simulations and explore new tools and techniques for addressing the problems
players would confront. AT&TD could be an exceptional IC asset, as it could
be a think-tank mandated to constantly drive analytical methodologies toward
the cutting edge. It could develop and refine new approaches for tackling hard
analytical problems until they were mature enough to be put to work in the IC.
from their respective charters and expertise, AT&TD, R&A and SD could
cooperate to design simulation tools and techniques, with a particular focus on
pioneering methods and software that could be used outside the center by
analysts in small groups at their home facilities. Their work could be enhanced
if the NSSC facility had a charter that, while allowing it to handle classified
information, also allowed experimentation with new computer network
technologies, and allowed for the simulation of 24-hour news media coverage.
potential local and global influence of the media makes it an essential
variable in the simulation environment. Accordingly, an NSSC facility would
need distinct spaces, wired for Internet broadband communications and
teleconferencing, where multiple teams of varying sizes—perhaps a dozen or more
at a time-could play, with at least one dedicated auditorium capable of “hot
wash” sessions, where all participants and observers could participate in
pre-and after-action reviews.
[Top of page]
Outreach Staff (PS&AO)
quality of any simulation, and therefore its analytical results, depends
directly on the quality of its players. While the IC has more than its share of
world-class experts on many subjects, its expertise is dwarfed by that found
outside the IC in other government agencies, the private sector, and academia.
The NSSC could not realize its full potential without taping into those
reservoirs of talent outside the community.
would be responsible for identifying outside experts willing and able to
contribute their time and talents to working side-by-side with IC analysts to
design simulations and to play them out to develop the conclusions. Backed by
the name and prestige of the Office of the DNI, the NSSC almost certainly would
attract leaders from every relevant field, including former and current heads
of state and other high-ranking government officials, corporate CEOs,
technology visionaries, and key academic figures. Their appearance in a
centrally managed simulation would also ensure that their expertise was more
widely shared among all the agencies than possible under present circumstances.
[Top of page]
our core, IC analysts are, first and foremost, investigators and scientists. As
professional intelligence officers we aggressively search for meaning and
strategic understanding of the world and the forces affecting it. We do this to
make sense of the present and to give our nation’s leaders insight, context,
and prescience about the future. However, we have been asked to increase the
quality and relevance of our insight even as the volume of data increases and
the time available to make sense of it decreases.
National Security Simulations Center could be a 21st-century model for
processing and analyzing potential geopolitical developments before they
happen. The center would provide additional ways of exploring why things happen,
why they break, and what geopolitical levers influence global changes. It would
also be a training ground for IC officers to hone their craft. Uncovering
hidden assumptions, identifying new indicators, illuminating alternative
outcomes, and developing and testing new tools and techniques are tasks
inherent in the process of designing and running simulations. As aptly stated
by Peter Schwartz in The Art of the Long View, “The scenario process
provides a context for thinking clearly about the impossibly complex array of
factors that affect any decision.”
what we, as analysts and intelligence collectors, do is going to get harder.
The state of the world continues to become more complex. As a nation, how well
we continue to influence that complexity is directly related to how well we
first make sense of it. The DNI National Security Simulations Center, a
seemingly natural step in the evolution of the intelligence profession, would
go a long way toward helping us to better understand that world and to better
serve our policymakers.
[Top of page]