PROJECT SAFE

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP84-00933R000500120004-2
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 49
• Document Creation Date: December 16, 2016
• Document Release Date: June 28, 2005
• Sequence Number: 4
• Publication Date: February 19, 1982
• Content Type: MF

Science and Technology Advisory Panel
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
19 February 1982
MEMORANDUM FOR: Director of Central Intelligence
birector, Intelligence Community Staff
Deputy Director of Central Intelligence
SUBJECT: Project SAFE
1. I am forwarding the memorandum prepared byl of STAP
on his discussions with Government personnel and oncerning the SAFE
project. I concur with his conclusions, options, and recommendations, based
on m own independent assessments, having discussed the matter with officials
STAT at CIA, and DIA.
2. While STAP through the accompanying memorandum presents two options,
it is our firm belief that the SAFE contract should be terminated at once. In
coming to this conclusion, we recognize the downside dangers of prejudicing
further work on systems that are urgently needed by the analyst. Attempting
to save a flawed system by spending two million dollars a month is not a wise
investment. Much has been learned from the SAFE experience, but a system
whose architecture is based on early 1970's technology is not what will be
needed in the late 1980's. A new start, incorporating the lessons learned
from SAFE, involving intimate interactions between analysts and the systems
architect office based on a contractural arrangement which requires on-site
participation by all parties, and which is monitored on a continuing basis by
outside experts, is a much sounder investment.
3. In reaching this conclusion, we recognized, as is stated in the
memorandum, that considerations other than those of a technical or managerial
nature may make termination politically impossible. Nonetheless we strongly
urge you to consider termination.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY Page i
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
STAT Memo to:
STAT From:
Subject: SAFE
STAT
STAT
STAT
0.0 ABSTRACT
Chairman, STAP
This memorandum analyzes the current status of the SAFE project,
assesses its future, evaluates certain options, and makes-
recommendations about what to. do. The immediate problem is that
the current contractor's iproposal for what will be
initially delivered not only is undefined in many particulars,
but also is simply unacceptable to the Government; to correct
that will surely take both money and time. We must make certain
that the SAFE system provided will be as good as possible.
The deficiencies in the situation are that: 1) the designers at
in which SAFE will work; 2) the design is unfinished in many
details, including some important major decisions, like the
ave never really been aware of the operational environment
distribution of functionality between thel mainframes STAT
and the. user terminals; 3) the user language reflects almost none
of the lengthy detailed work that SAS and CSPO have undertaken --
but note thati Chas recently agreed to accept the Government's
set-of language commands; 4)1 lis responding only to the stated STAT-
requirements, with little appreciation of the demanding jobs that
analysts have; 5) it is not clear whether the hardware will
support both the projected usage and the planned number of users;
6) it is not clear whether the other design decisions are
appropriate.
We recommend that all elements of the design be reassessed by
independent experts. If these evaluations are favorable, the
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY Page ii
Approved For Release 2005/07/12: CIA-RDP84-00933R000500120004-2
appropriate design decisions should be made to correct known.
deficiencies and to resolve the many major 'open issues, and the
contractor should be tasked. to produce a full Block 1 capability.
Enough time should be allowed for this -- we estimate it will
take 18 months more than now scheduled, at least. It will also
take more money. The alternative, we recommend, is to terminate
the contract, though not the effort to provide analysts with
SAFE-like capabilities. Furthermore, we recommend in any event
that the Pilot Mail Operation be extended and enhanced.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
CONTENTS
Page iii
1.0
Introduction
1
2.0
CSPO Concerns
4
2.1 Management
4
2.2 The SUL
6
2.3 System Design
12
2.4 Failure and Recovery
14
2.5 Performance
15
2.6 The Brighter Side
.16
2.7 BUT
17
3.0
Other Issues
19
3.1 Pilot SAFE
19
3.2 Human Factors
21
3.3 How Can We be Sure About SAFE?
22
3.4 The Capabilities of SAFE Hardware
23
3.5 The Intelligence Community and SAFE
23
4.0
What has Affected SAFE?
25
4.1 The Underlying Causes: History
25
.4.2 The Underlying Causes: People.
27
4.3 The Role
28
4.4 The Role of CSPO
29
5.0
Conclusions and Recommendations
30
5.1 Discussion
30
5.2 Conclusions
33
5.3 Recommendations
35
Appendix: Glossary of Acronyms
37
Appendix: Individuals Referenced
38
Appendix: Excerpts from STAP Options Paper (a.ttached)
39
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 : OFFICIAL USE ONLY Page 1
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
1.0 INTRODUCTION
This memo is part of a continuing review of the SAFE project. STAT
SAFE is. entering a crucial period, expenditures Oare rising STAT
to over $2M a month, and the proposed initial deliverable
seems to be unacceptable to the Agency and DIA. It seems clear
that soon certain decisions must be made about modifications and
delays in the contract that will have a large impact in both
Agencies. Here.I discuss the implications of the continuing
problems, especially in the light of the late January 1982
Preliminary Design Review (PDR), which I attended STAT
with the cooperation of the Consolidated SAFE Project Office
(CSPO) I am grateful to CSPO for making that possible
and was impressed by the diligence and dedication of the
government personnel at that meeting.
The history of the SAFE I_ ontract is long and complicated, and
I shall not try to'summarize it here; there are, however, some
pieces of it that must be mentioned:
o By now, the presently projected early blocks will have far
fewer capabilities than had been originally specified. As
costs rose, because of changes, delays,'etc., many
capabilities were delayed, and some were dropped. This was
because the SAFE contract was essentially fixed price,,
although what was negotiated was cost plus fee.
o In the summer of 1981.,-?theI (hardware was selected.
o The Initial Operating Capability (IOC) had always been set
for 1 Jan 83; it is now set at 14 Mar 83, apparently as a
result of PERT projections. The initial deliverable is now
termed 'increment 2.5.'
o In the summer and fall of 1981, it was becoming increasingly
TAT
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 Approved For Release 20 0 1 I ? AI iF1 tI11- FpP=00 t y9338000500120004-2 Page 2
apparent to CSPO that the contract.was in trouble..The
aggregate Award Fee Scoring had fallen from about 75 to less
than 40 in the 5th 6-month period, and the "overall
performance for the 5th period was evaluated as marginal.."*
o In the fall of 1981, partly as a result of. that assessment,
a significant reorganization of thelIstaff was undertaken STAT
("The ritual beheading," according tol head of STAT
the IC staff).
o In the middle-of January 82, CSPO and the Systems Analysis
Staff (SAS) of the Office of Central Reference (OCR)-'learned
was proposing to supply, with the first release of
the deliverable system, a users' command language that
seemed to reflect very few of the deliberations that the
Government had been sharing with the corresponding group on
Language Development The functionality that was seen
in the proposed release seemed dubious, and did not match
what SAS had been discussing.0 called the proposed 2.5 STAT
delivery "abysmal", and CSPO in general regarded it as
unacceptable. In fact, none of the eventual user languages
have been fully specified, and for some, like the analysts'
editing language for COMPOSE, submitted no proposals STAT
at all.
o The PILOT Mail Operation (PMO), which was one of the
successors to the Interim SAFE, has been regarded as
successful enough to be considered as worth expanding (Bruce
Johnson, ODP), if there are any substantial delays in SAFE
implementation.
These and other topics will be explored below. Decisions.must be
made very soon about what can be reasonably expected for the
*Memo: Point Paper for SAFE Steering Committee,
(Director of CSPO), 11/27/81.
STAT
Approved For Release 2005/07/12': CIA-RDP84-00933R000500120004-2
 2/19/82 ' - OFFICIAL USE ONLY Page 3
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
performance of SAFE, and to what schedule. If there are to be
significant changes (and I am sure there will have to be), then-
the political and financial implications may be very great, and
it would be well to be sure about all the projections.
There are many lessons to be learned by the Community from SAFE,
And I shall try to discuss some of them. Many of them are far
beyond the responsibility of the current participants in SAFE,
and, indeed, some deal with the entire nature of the
relationships between Government and Industry in large and
complex purchases like this one. Sometimes it must be appropriate
to delegate the responsibility for overall and detailed design of
a large and complex system, but it was not for SAFE. We will
discuss this point further in section 4.
But the most important recommendations are those that concern the
immediate options that the government has. The need for a-SAFE-
like capability is enormous, and I believe that it would be a
devastating loss to the Community should the effort to achieve it
be abandoned. The question is how to redirect the current thrust
of the effort so that the best capabilities can be provided to
the Community as quickly as possible. That will mean a speedy
inspection of the proposed SAFE system in its current condition
to make sure that it can become the basis for those capabilities;
or else a study of how best to use the lessons learned.
One of the difficulties in making recommendations is that there
are very many factors and design decisions that have not yet been
made, or at least been agreed upon, by both the contractor and
the Government. For example, one of the fundamental capabilities
to be provided to the analyst by SAFE is COMPOSE,* which deals
with the building of a document to be produced as a finished
*There are four major capabilities that SAFE should provide: 1)
receive and distribute cable traffic; 2) create and maintain
analysts' own files; 3) search SAFE files; 4)-write or compose
reports etc.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFAL USE ONLY Page 4
Approved For Release 2005/0 / : CIA-RDP84-00933R000500120004-2
piece of intelligence -- including editing, formatting, etc.
Almost nothing about COMPOSE has been agreed upon. Indeed, the
contractor has not even made serious proposals about what it
might include, and neither has the government proposed a truly
firm set of requirements to be discussed with the contractor..
At best, SAFE as currently forecast by both the contractor and
CSPO will fall far short of what was originally envisioned by
those responsible for its initiation, at least in the current
contract. I believe that it is important to keep the vision
alive, both as a goal to be striven for, and also as a key to
keep us from accepting as a frozen design something that:will not
'let us reach it.
2.0 CSPO CONCERNS
There are a number of concerns that worry CSPO now; some have a
long history, and some have arisen only recently.
2.1 Management
CSPO has always tried to ensure that it maintained control over
communications I lit established that any contact with
had to be made through CSPO. One problem that followed
rule was that it was very difficult to observe
analysts working
so that
in their own environment on their own
STAT
from that
the STAT
problems,
of what it was that analysts were doing was
derived from the words in the specifications for SAFE -- and the
specifications are not finished yet. It is apparently also true
that id not insist either in their proposal or so far during
the contract.that such observations were essential not only to
good human factors but also to a good balance of the
functionality and the resources in SAFE with respect to present
and future usage.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/87 ~I ~T ~I.A~ USDE ONLY
pproved For Release 200 0 1 A-R P84-00933R000500120004-2
Page 5
One healthy change in the CSPO viewpoint about management was to
require emphasize modifiability and flexibility in the
hardware configurations and programming; so that when changes and
modifications and extensions become clearly desirable, there will
be but little trauma and interruption in making them. I am not
sure to what extent this requirement has actually been put into
effect, and that point should be carefully checked.
The management of large and complex systems, especially when they
involve software, is always remarkably difficult. Indeed, the
acquisition of large software systems is probably the least
successful of government contractual exercises.* CSPO early
established an evaluation technique for this large contract (ca.
$75M) called Award Fee Scoring:
o
Technical Performance
50%
o
Management
25%
o
Schedule
20%
o
Security
5%
In the Point Paper,
"Overall performance for 5th (six-month) period was
evaluated as marginal.
Problems outlined in previous evaluations persisted
with predictable results
Management was aimed primarily toward gaining technical
control -- ongoing activities were marginally managed.
*See, for example, the Final Report of the Software Acquisition
and Development Working Group, Chairman prepared for25X1
the Assistant Secretary of Defense for C -I, July" 1980.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 ' OFFICIAL U5t UNLI rave v
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Management and organizational changes late in: the
period place competent management in several key
positions; results should be evident in upcoming
periods." (ibid, p. 3)
It seems clear from that statement that CSPO believed that the
management and organizational problems n this contract STAT
were of long standing. Indeed, in the final minutes of the PDR,
(CSPO) spoke of "knowing what the problems are,"
andi hat "the level of uncertainty is down;=ad told STAT
I I that the "SAFE Project is back on track."* A tough
manager, had been named in late September as Deputy
Project Manager for Engineering, and seemed to be more effective.
Much of the reassurance that CSPO was expressing in December was
due to the PERTing had initiated less than a year
previously (chiefly as a result of pressure from CSPO), so that
1could report that the "Contractor (was) finally achieving
detailed planning necessary for control and management of
software development." (Point Paper, ibid, p. 5)
That reassurance was badly shaken when the SAFE User Language
(SUL) proposed to incorporate in the first release
(increment 2.5, now scheduled on 3/14/83) was evaluated by SAS in
1/82 as totally unacceptable, as I mentioned.above. Furthermore
the functionality underlying that language, and which was the
rationale for it, seemed dubious indeed and not responsive to the
user needs as they had been stated. The next section will discuss
those topics.
2.2 The SUL
*TRW- "The SAFE Project,": Briefing for
1/15/82, chart 3.
STAT
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 - OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
STAT
in several ways by users. The interactions of the Agencies with
Page 7
It is not overstating it to say that as far as the analysts are
concerned the most important parts of SAFE are the ways they
communicate with it; how they get it to do what they want done,
and how they receive the information and products that they want.
The SUL consists of a set of commands, in effect, which are input
through the' SAS/CSPO User Interface Working Group (UIWG), which
had believed that steady progress was being made. As late as
11/81, the SAS reported that
in the matter of user languages has been almost entirely
"(in order) to assure an orderly introduction of SAFE into
NFAC on 31 December 1982
"The User Language Specification (ULS) is to be in final
form by the end of CY81."#
In the middle of January 1982, a CSPO/SAS technical team reviewed
SAFE documentation for the SUL. There had been essentially three
languages proposed:
o The government version, basically worked out by the SAS/CSPO
UIWG; a functional set of commands. Many of them had been
delayed for later blocks as costs rose and delays occurred.
STAT o The language team had been working with the
Government, but was much more concerned with the problems of
implementation. For example, a key problem in designing SAFE
STAT is that both the CPU's and the DeltaData terminal
(7260T's) are capable of many of the functions required; how
to separate the responsibilities so as to maximize the
performance was of much more concern than to the
Government. This led to different emphases and different
solution.s to many problems.
#PUC No. 13, "The Plan for Implementing SAFE in NFAC," 11/4/81,
p. 4.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82? OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 8
o The System Development team as deriving language STAT
specifications from the functional analysis of the CPU
capabilities.
The CSPO/SAS discovered by studying the commands that the third
team was in effect the winner. had stated that the design was STAT
frozen. It seemed to SAS that the third team had had little or no
communication whatsoever with the second, the language team, and
none with the first, the Government. They felt that they had
worked for years to no effect at all. The large experience with
PILOT Mail had been totally disregarded. L evaluation was STAT
that the language and the functionality of 2.5 were "abysmal,"-
and CSPO in general has rated the proposed deliverable as
unacceptable.
Those are strong statements, but I agree with them, and I include
a couple of examples to justify them.
o In the proposed delivery,. the user deals with his files only
through DISPLAY, which is the fundamental operation. This is
a design decision that has far-reaching
ramifications -- that the fundamental operations
functionally deal with a 'screenful' of information. That
is, the user's concept of the system should be that he is
always dealing with a screenful of information. The user can
delete a file only by inputting 'delete' for every record in
the file after he has summoned it to the screen. A file, of
course, can contain a very large number of records. If a
user wants to print out a file, he has to go through the
same exercise with 'print':
"PRINT
1. A single record at a time from a display file may be
printed.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 9
2. Files, parts of records, etc., may not be printed.
Therefore, no ordering, starting at, etc.
3. No COPIES parameter. (i.e., there is no parameter that
enables a user to print a number of copies.)
4'. Pr.inting of related records which are displayed
together is. possible using the right output forms.
5. Special printer characteristics cannot be specified
(e.g., font)."*
o The Command Procedures (essentially macros that allow the
user to run sequences of commands with a_single command)
were to be implemented, but without the features that gave
them any of the powers needed -- no parameters, for example.
This is significant because the command procedure would
allow over-riding of the restrictions discussed in the
preceding item; a properly written command procedure could
delete every record in a file with a single simple command.
Inputting a .'command sentence consists of typing a command
followed by a number of parameters that tell SAFE things about
how the command is to be executed. The ways that the parameters
are expressed and ordered or punctuated are called the syntax of
the language. The syntax makes a large difference in the
convenience of the language to the user; compare the following
three versions of commands that cause a file named BERLIN to be
printed on a line printer in the usual format named FORMI:
PRINT BERLIN
PRINT BERLIN, LINEPRINTER, FORM1
Internal Memo: "Comments on Customer SUL Letters of
Direction," to Distribution, from Hagerbaumer, Breisacher, and
Almeida, 1/8/82, 82.35656.77-004.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 ' OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
PRINT FILE=BERLIN, DEVICE=LINEPRINTER, FORMAT=FORM1
Page 10
These command sentences, properly interpreted by a subprogram
that might be called the Command Interpreter, all mean the same
thing.# The shortest one, the first, assumes that the user has
established the lineprinter as the DEFAULT device, and the format
specification FORM! as the default format. In fact,
proposal for the initial SUL syntax seems to look very much like
the third version.
Syntax is a complicated subject, but it is obviously crucial to
the analysts' usage of a system. The third version takes more
than four times as many keystrokes as the first. The use of
default values for parameters is a key issue. Sometimes
parameters have to be entered in a predetermined order, and that
too can be burdensome to the user analyst.
There are other restrictions for the user to bear:
Gov't question:)
"Is the version number incremented when a "file" action
takes place or when a user calls up an item to be
edited.(si_c) The record is 'copied' when routed, will this
be a new version? Can a user request to see "latest"
versions or must he specifically name the version? When the
user sees the status of a compose file will he see both item
name and version number? Does the user have the option of
replacing a record in lieu of creating a new version?
0
response:)
"Version number is incremented when an item is edited.
The record is copied when routed.
Version number is associated with the routed record.
#Of course, in the.proposed deliverable no such commands would be
usable at all, since they call for a whole file to be printed in
a single command.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 ' OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 11
"The user must specifically name the version.
The user does not see both item name and version number (no
version mark). The user does not have the option of
replacing a record in lieu of creating a new version."*
That means that. the user analyst must remember a version number
and type it in for every file transaction; this would be a
trivial programming task to get the computer to do. It would to
my mind be insufferably burdensome to the user.
The number of restrictions of usage placed on the analyst is far
too large to be acceptable. Furthermore, of course, they should
be checked in the context of reasonable actual usage -- the PMO
is the ideal place to make sure that the projections are
reasonable.
My judgment is that CSPO is correct in deeming that the extra
burden and restrictions placed on the user analysts
proposal are unacceptable in a delivered system. The fact that
these restrictions derive from a functionality means that they
are not merely cosmetic, and hence fixable, but instead reflect
real trouble with the design.
I go on at some length about these topics because they show the
extent of the communications disconnect between the Government _
needs and the contractor's interpretation of them. The 2.5
release as proposed at the PDR is not acceptable to the
STAT Government, has agreed (as of 2/19/82) to improve its
- STAT
first release. In fact, it is my understanding that STAT
recently accepted the entire Government position on the SUL. The
underlying issues of design in support of it are far less clear,
however.
STAT Response to Gov't. Comments on Data Base Definition
Document, 1/25/82, at the PDR.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 - OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
STAT
STAT
2.3 System Design
Page 12
One of the most distressing aspects of the current status of SAFE
is encompassed by the following comments made at the end of the
PDR:
"The system design is not complete" and there are still
"open areas." I PDR, 1/28/82)
o "The design must be subject to validation." (ibid)
o There are real concerns about the "delivery of 2.5 and its
viability in the users' hands." (ibid)
o There are still "substantial system issues ...
Applications Software, I (response was:
o The system design is "not yet quite right." (ibid)
It is now less than 14 months from the scheduled delivery of the
IOC. The design uncertainties exist in many fundamental functions
that SAFE must perform, not just in the SUL, discussed above:
o In response to Government comments about procedures for
"We plan to update this level of information as we move into
detailed design activities. As the detail (sic) design is
completed, it will be reflected (in the) documentation."*
STAT 0
STAT
PDR, 1/28/82)
'The micro sequence array (of MICROSEQ) contains
execution.steps and parameters associated with the execution
I J Responses to Gov't comments on Product Specifications for
App cations Software, PDR, 1/26/82, p. B--023.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 - OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 13
steps. The details of the structure and contents are still
being defined and should be resolved. shortly'."#
TAT
TAT
TAT
"As the units and procedures (in the UIS) are comp
through detailed design, the detailed work messages and
subroutine calls will be reflected ..."k
0
"The detailed design of micro generation is not yet
complete ..." (ibid, p. C-022)
eted
The DeltaData 7260T is still under development; the function
keys and the documentation are both unclear.*
"Major SAFE C Design Issues:
TAT
TAT
TAT
Performance ...
User Language
Failover(sic)/Recovery
Operating Modes
External Systems ..."#
The only well-established text editor now available in SAFE
for the proposed delivery is the one provided by the
DeltaData 7260T terminal; "this is not acceptable as the
sole text editor for SAFE,"
CSPO, at the PDR).STAT
o It is well known in large user-oriented time-sharing systems
that for most of the time the computer is engaged in
interpeting and carrying out commands that essentially have
to do with organizing and executing editing. As we remarked
above, there is no editing language well enough specified
PDR, 1/26/82, p. C-010.
*Discussi.ons at PDR, 1/28/82.
ibid, p. B-031.
Responses to Gov't. Comments on UIS Product Specific.ation,STAT
he SAFE Project," Briefing for
1/15/82, chart 25.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 14
for inclusion in SAFE. Note that ODP, whose current widely
used E-language is X-EDIT, has for some time been
deliberating about the upgrading of that to one more
sensitive to users' needs and to the increased capabilities
of modern systems. ODP_ has now actively embarked on
developing a moderncree-baed E-language.* At the PDR,
there was no evidence'-- h=asuch deliberations were being
received and considered
It should be remarked that these are not the modifications and
"tuning" that devolve from a system's being put into practical
operation and then adapted to actual as opposed to projected use.
No large piece of the SAFE hardware and software has actually
been tested. If there are therefore many fundamentally unfinished
design issues in SAFE, how realistic is it to propose an IOC date
of 3/14/83? I do not think it is realistic at all, and I have not
found any experienced system analyst who thinks it is, and CSPO
seems to agree. The point is that SAFE must work reliably -- far
more reliably than just well enough to correct bugs and to adjust
some software and rewrite others.
This skepticism is separate from the problems arising from an
unacceptable SUL. I shall return to it later in this section.
2.4 Failure and Recovery
Failure (in the u documentation it is inexplicably referred to
as 'failover') must be a key concern. Every computer system can
find new and thrilling ways to fail, but SAFE must protect its
contents and its.users with an unmatched reliability. Since SAFE
is large and complex, there are many things that can go wrong.
itself has recognized that how to handle failures has still
*Personal communication, Bruce Johnson, ODP.
STAT
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 . OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
rage i
to be resolved. It is a'"Major SAFE C Design Issue."* Failures
can result from caprices of hardware or software, and since
neither has been fully designed, it is not easy to
kinds of failures that will occur. History assures
that they will. Not only have methods for handling
including, of course, recovery from them, not been
the design requirements have not even been written
most general terms.
2.5 Performance
project the
us, however,
failures,
specified, but
except in the
The performance of SAFE has been projected with Performance
Analysis. "Performance Analysis is the primary vehicle ... to
verify the software architecture." F PDR, 1/28/82) This STAT
requires the "appropriate models," and "all tests and benchmarks"
are performed on the models. The models are built -- indeed they
have to be built -- using scenarios of projected usage.
Now that is curious. First, SAS tried unsuccessfully for 18
months to persuade lo produce in the form of scenarios their
notions about what SAFE would actually do for an analyst. Second,
the scenarios used for the (computer-based) models are not
current, but are based on data three to four years old. Third,
the scenarios, and the other assumptions underlying the models,
take no advantage at all of the experience the Government has had
with PMO.
The computer-based projections do not seem unreasonable, given
the vaguely communicated notions of usage, and supposing that
there will be no.major changes in architecture. They show that
the first release will in several ways not satisfy the
requirements without certain modifications and improvements,
STAT
*The SAFE Project, A briefing for
1/15/82, Chart 25.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For F_elease 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 16
which are ow''being planned for (of course, they cost extra).
the mos -favorable projections, with the improvements, the
margin become small and mostly positive. We discuss this point
further in section 3.4.
To a certain extent, that is all to be expected; at the end of
the PDR, aid that "the performance problem is no
surprise." I't must still be remembered, however, that the
projections come from simulations.
2.6 The Brighter Side
The CSPO spokesmen profess to be encouraged by what they learned
at the PDR:
OT the program ... an ambitious project plan."
"The Government does understand the state
"The level of uncertainty is down," and "we
need to wrap up the findings, to finalize the design."
.said he was delighted.*
the senior DIA representative on CSPO,
These optimistic reactions are perhaps de rigeur at the end of a
PDR. Of course, they are understandable after four days of
grueling reviews and active, sometimes acrimonious, discussions;
and for many of the Government personnel, I know, the work
extended far into the night.
*The PDR was chiefly directed at the C system, for CIA. The D
system for DIA is different in many ways, but also similar in
many; the exact commonalities have not yet been fully determined.
In any case, the D system will be delivered a year or so later
than the C.
Approved For Release 2005/07/12 CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
2.7 BUT
Page 17
The question to me is whether any optimism is justified at this
time. If I_"management was aimed primarily toward gaining
technical control," (p.5 above), what had they been doing for the
previous 24 months of the contract? Should one believe that they
could now do better than they had earlier? What had CSPO been
doing for those 24 months?
I will put it strongly:
We are less than 14 months from the scheduled delivery of.a
system that is rated unacceptable. As of 2/19, there is
apparently agreement that that delivery will not be made; no
other delivery date is firm.
o The design is incomplete.
o There are many serious issues to be resolved.
o No substantial piece of programming or coding has been done',
let along been debugged, tested and used. (Note that this is
not somebody's fault -- it was specified in the contract.)
o The user languages are undefined, so that documentation and
training are up in the air.
o . The known experience with' PMO, the only extant system with
similar requirements and actual performance in the real
operational, environment has been ignored by the contractor,
or not been made available to him.
The situation is even worse than that. In some environments, like
a University or an R&D establishment, users can tolerate
inefficiencies and failures -- some of them might even enjoy the
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 ' OFFICIAk USE ONLY
Page 18
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
interactions that can help develop a viable and productive
system. I do not believe that that is true of the CIA or the DIA.
The DDI analysts (and the other ones too) that I have met are
interested in finding the best ways to help them do their jobs,
wickedly demanding at best, and they are not at all interested in
helping a project that for many is alien to their experiences and
their tastes. That means at least:
SAFE must visibly and immediately work: it must do things
for analysts that they want done; it must change how it does
those things to satisfy different analysts' different needs,
even frivolous ones; it must incorporate and exhibit decent
human factors; it must tolerate human errors gently and
supportively; it must provide ways to recover from its own
failures, with no loss (not minimum loss)-of data, and with
no loss (not minimum loss) of security.
o SAFE must work all the time: that means 24 hours a day,
every day in the year; it must be tolerant and adjusting for example, if a CPU fails, another one should take its
place automatically; if a terminal fails, the user has to be
able to move to another and start over where he left off; if
a user-'cannot remember how to perform a function, it should
assist him on-line.
o To its users SAFE must emanate feelings of adjustability and
of the possibility of unbounded improvements in response to
them. If users don't like something, or if they reasonably
demand another feature, they should feel that SAFE will
react positively to them; SAFE should also be generating new
capabilities for testing and applying in the marketplace of
intelligence analysis and production.
The 2.5 increment planned for the first delivery is clea
that SAFE, currently negotiated by the Government
y not
STAT
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2 / p &ved For Release 2005/07/'CI F(2IAII P$k1 60$3 i 00500120004-2 Page 19
3.0 OTHER ISSUES
Nearly two years ago, STAP discussed the SAFE project in memos to
the DCI, expressing certain concerns and making certain
recommendations. Some of those concerns have been alleviated, but
some are still with us; some of the recommendations have been
followed, some not. In this section I shall discuss the topics we
discussed then and their relevance to the current situation in
SAFE. The key document is "Options for SAFE," STIC 80-002, from
STAP.
3.1 PILOT SAFE
STAP made a strong recommendation to upgrade Interim SAFE toward
a system that would perform as much as possible like what was
wanted from SAFE.* To a very significant extent that
recommendation was followed, and the result has been the
development of the PILOT Mail Operation, PMO.
PMO has already provided some interesting and relevant lessons.
According tol head of SAS, whose responsibilities also
include running PMO, "the usage, number of terms used (in SEARCH
profiles), the number of operations and commands used by
analysts, are all far higher than we anticipated... and we
learned some things that we can't do anything about for the early
SAFE releases."# Here are some other comments:
"Two months ago we installed the SAFE PMO in (an OSWR Branch
which) has already obtained significant benefits (including
*The relevant portions of the STAP Options paper are attached as
an appendix.
#Discussion at CSPO, 12/4/81.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 20
an) ability to quickly and efficiently create branch files
which are consistent, non-redundant, and accessible to
all...The branch is quickly getting messages on the day they
were sent rather than the several days (2-3)... via the hard
copy/registry route"*
"The A-Specification states that each SAFE mail file will
receive an average of 17 electrical documents each day. Our
experience in PMO...suggests that a more reasonable daily
average of mail received is 105 documents...
"The A-Specification states that a mail profile will contain
50 terms on the average, PMO experience suggests ... that the
average number of terms per profile is 111."#
imbedded, leading, and trailing "don't cares" are all
used in the PMO profiles. We see the continued use of the
"don't care" capability in SAFE at the same high rate as in
the PMO.
50% of the terms are unique within an
office...especially...within the five ... regional offices.
In nonregionally oriented offices, the incidence of unique
terms is as high as 90%.11*
Here are some questions# that ought to be answered by Pilot SAFE:
1. What are the usage patterns of naive users?
2. What are the usage patterns of experienced users?
3. What needs for modifications of SAFE performance become
*Memo from Director, NFAC, to Comptroller, CIA, 6/19/81.
#Memo from Chief, SAS, to Director, CSPO, 11/6/81.
*Memo to Director, CSPO, from Chief, SAS, 1/7/82.
#Fromn STAP Options for SAFE, pp. 6-7.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 21
manifest from the transition of naive users to experienced
ones.
4. What are the user documentation and training requirements?
5. What new requirements emerge from the experienced usage?
6. How should the services provided by SAFE be modified to take
advantage of new technology?
7. What facets of the system can be safely frozen in design
concept? What parts must be carefully engineered to retain
flexibility of function and performance?
It is still true that PMO does not undertake at present to
explore the other capabilities that SAFE ought to have -- the
full range of the other three functions, mentioned in the
footnote on page 3. It must be expected that the projected usage
of those functions will change with experience too.
STAP recommended a broad human factors study of what it is
analysts do:
"Further, ... we reiterate the urgency for a very general human
factors study of intelligence analysis and the behavior of
intelligence analysts."#
Indirectly, the PMO is providing valuable data itself. But that
information, of course, valuable as it is, goes but a short way
towards the goals we were talking of. Our points then are as
#Options for SAFE, p. 11. Note that the footnote in the quote
'(not cited) itself noted that the point had been made many times
before. The emphasis was in the original.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 22
valid now, I believe. A study initiated two years ago would by
now be contributing to many of the design decisions being
currently negotiated by the Government just as the POTAT
study now can.
There are still many human factors issues in SAFE that do not
reflect the state of the art, and that ought to be examined for
application in Pilot'SAFE. Some of them were mentioned a year and
half ago in a memo.to CSPO from STAP, 8/6/80, Subject: SAFE
Project User Language Specifications. I intend to review that
document with the Government people again soon.
3.3 How Can We be Sure About SAFE?
The SUL and the functionality of SAFE were the issues on which
CSPO and the SAS spent disproportionately much time and effort.
It is clear that those efforts will have been of little avail
without a lot more work and pressure Many other parts 0TAT
the SAFE project have in effect been taken on faith. Examples are
o The Data Base Management System (DBMS)
o The InterComputer Communication
.0 Structured File Processing
o CPU Capacities
Note that -I am not alleging that there are any deficiencies in
the I Iwork on those topics. It is just that the Government
ought to be in a position to be sure by its own examination of
the actual machinery and the data on its use. It is possible,
indeed likely, that the expertise does not exist within CSPO for
some topics, like DBMS. Given the most crucial nature of the DBMS
-- it will freeze many future choices and options for a long time
-- CSPO ought to consider putting together an ad hoc panel of'
experts,* and STAP would be delighted to suggest members.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 ' OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
3.4 The Capabilities of SAFE Hardware
Page 23
Another worrying aspect of SAFE is the capacity of the CPU
hardware under reasonable and anticipated operating conditions.
STAT Discussions that had with) I STAT
personnel suggested that a single mainframe can handle at most 35
users, and 15-20 would be a more reasonable figure for active
STAT users. The current CSPO]Iplans call for 100 users per
mainframe. It should be pointed out that the design is not far
enough along that that figure of 100 should be regarded as
frozen; furthermore, a certain amount of work for 'the user can be
performed by the DeltaData terminal. As before, this is such a.
crucial factor that it would be worth an independent check. It
should also be pointed out that thel ardware is totally STAT
incompatible with all the other mainframes used by the Agencies.#
The application of. an independent panel of experts to evaluate
this problem should be considered.
3.5 The Intelligence Community and SAFE
There are some aspects of SAFE-like support that ought to be
borne in mind, even though they are deliberately not planned now
to be a part of the SAFE capabilities. Although the present
concern of Community management must be the short-run decisions
about the optimal course for SAFE, STAP believes that there are
some long-run interests that should affect design and utilization
decisions.
SAFE is not now scheduled to be a Community-wide resource, even
though it will supply services to two Agencies. The D system will
have ports into COINS# (a very few, just two, I believe); but --
*Such ad hoc panels were recommended in the STAP Options paper.
#Personal communication, Bruce Johnson, ODP.
#COINS, Community Online INtelligence System, is a very large
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 ? / 19 / 8 2Approved For Release 2OO'MT/ 1 ACIA-FDP84 0'0'933ROO0500120004-2
Page 24
those will be enough to gauge their value. Adding more of
something that is known to work and work helpfully is always far
easier than to try to add a completely new feature. STAP believes
strongly that a SAFE-like capability will form almost the
cornerstone of this country's intelligence processing in not too
many decades; and that, furthermore, that capability must enable
the analysts to take advantage of all the data bases, collection
techniques, processing powers, and so on, to which the Community
has access. Actually, CIA has had two COINS terminals in the
library, I believe, for more than ten years, but users find them
awkward to use, little training is provided, and there is minimal
encouragement from management.
STAP concludes that it is vital to start now to find out-how to
give SAFE these greater capabilities. We suggest again that a
reasonable way to begin is to provide Pilot SAFE with ports into
COINS. It will be necessary:
o To explore and decide on the proper security rules, including
terminal access, file access, audit trails, and
communications security. Note that COINS already has its own
solutions to most of these problems.
o To experiment with some form of gateway to control access and
communications. Note that COINS has had much experience with
gateways, and for their purposes the wrinkles have been
ironed out.
To begin to study accessing data bases with different access
languages and protocols.
network of intelligence and C31 computers with its central node
and control at NSA. It was started about 15 years ago, and
originally had only 3 or 4 elements including the CIA and the
DIA. The Director has been and is -It carries data STAT
at all levels of security.
Approved For Release 2005/07/12 : CIA-RDP84-00933ROO0500120004-2
 2 /'' /''Approved For Release 2095/OMI ~GIA'lR6PP-6J933R000500120004-2
Page 25
The current plans for SAFE propose that there be no online access
for SAFE users for data bases outside SAFE, and no access at all
for outside users into SAFE data bases. We hope that these plans
will be modified. We suggest, and in fact recommend, that the
Community commit itself to providing analysts throughout the
Community with carefully controlled access to.all the data bases
it owns. Obviously some need very circumspect access indeed; but
we should be learning how to do that right, instead of trying to
avoid the issue. 'There have been no studies that suggest that
that task is somehow inherently impossible. COINS experience
suggests that it can be done.
4.0 WHAT HAS AFFECTED SAFE
The SAFE project is in a bad way. I do not believe?that the
fundamental SAFE capabilities -- represented, say, by Block 1 --
can be provided without a very substantial delay beyond the
current schedule. Even if Block 1 capabilities are provided, how
can we be sure that we can keep the other blocks moving ahead,
and keep on,,providing capabilities for the future requirements?
It would be too simple to think that once the surface
disturbances have been smoothed over, the SAFE Project can be
considered rescued and "back on track." The difficulties with the
language, the SUL, are upsetting enough in themselves, but they
represent a fundamental failure of the design process, and that
goes deep indeed. We should not be tempted to believe that once
that is fixed everything will be 'all right.
4.1 The Underlying Causes: History
The SAFE project has a long history, and many people have
contributed to the decisions from which the present has arrived.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/ 19 / 8 2 Approved For Release 2t0l7t11_CI0RDF00933R000500120004-2 Page 26
Some of the decisions seem to have been made almost casually: as
part of the contract, provide a "Phased Implementation
Study," to recommend providing a capabilities in steps instead
of all at once. Therein may be read:
"2.1.6 Development Methodology Changes
Schedule compression resulting from (delays) causes several
modifications to the development methodology;
6) To help relieve schedule compression, user language
prototyping will not be performed."*
That speaks for itself. Other items slipped in-similar ways. For
example, the ability for an experienced user to 'slave' his
terminal to another's is invaluable in enabling one user to-help
another with his problems online. Yet an initial requirement for
such a capability vanished along the way, and was scheduled to be
implemented in segment 6, some unspecified time after the IOC.#
Similarly, performance tools, simulation and optimization tools
are delayed indefinitely.
In some real sense, we should talk about the causes for this
situation-only in order to try to correct the inadequacies that
have led to the present situation. In late 1976, a memo was sent
to the Chief, Special Projects Staff, ODP, containing comments on
a paper entitled "SAFE Design Concepts." It stressed the dangers
of assuming that one could write then the functional requirements
for a system to be used far in the future; it noted that analysts
don't know what to ask for if what they are offered is outside
their experience, and that the system itself, if successful,
would inherently change patterns of usage.
I"Phased Implementation Study Final Report,"
D. 20.
25X1
ee i id, p. 51, fig. 3-15, under "teleconferencing."
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 27
The difficulties of SAFE arose through the nature of the needs
and the processes by which the project was initiated, the
contractor selected, and the contents of the contract specified.
A project like SAFE is 1) very large, both in hardware and in
software, 2) very complex in capabilities, structure and
function, and 3) interactive with many users. In fact, nearly all
large software systems acquired for the Department of Defense
suffered from cost and time over-runs, often extreme.* Especially
because we did not know exactly what the users would be wanting
to do, it would seem to be advisable not to try to purchase a
single system to 'do the job.'
Rather, not too much new should be attempted in each increment-of
capability or function; and each increment should be evaluated in
operational conditions and integrated with other components. The
users -- the analysts.-- must be active and contributing
participants. These cconsiderations should be explored carefully,
and they suggest a form of acquisition of capabilities that is
very different from the current SAFE contract.
4.2 The Underlying Causes: People
The people who work in the Intelligence Community are by and
large diligent, competent and dedicated. But'there 1 arse few among
them who represent excel l efice (but there are some)\.,i,n' the
forefront of computer technd-logy; that is exactly olne of the
resources needed in managing and controlling a project like SAFE.
Indeed, many of us can remember when the Intelligence Community
in fact supported the state of the art -- Project Lightning, for
example, moved computer technology ahead five years in one
stroke.
Manifestly, the need for real state of the art experts in the
*The SADWG report, V.E.Jones, ed., ibid.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2 , 19/82 Approved For Release O0o 0 / 17 ~'1 :D~A-RDP84ONLY-009338000500120004-2 Page '28
STAT 4.3 The Role of
STAT
technologies that are so much a part of intelligence these.days
's not unique to SAFE; but there it applies especially, because
the
tasks that SAFE is dedicated to are not 'glamorous' in the same
way that space reconnaissance or molecular beams are. One result
for SAFE is that questions about implementation or design tend to
be answered by the contractor alone, and then imposed as it were
on the Intelligence Community. Even if there are experts inhouse,
the bureaucratic environment tends to deaden their ambition,
be ause apparently so little can be done:-
It is puzzling why seems to have ignored the real needs and
directions of SAFE. SAFE is not that small a contract, the
Intelligence Community is not that unimportant even in the
context of the military industrial complex has some very
bright people working for it, some even brilliant. Why were none
.of them assigned to help SAFE?
Even today, other questions remain: if a tough new manager
STAT I uis assigned to be Deputy Project Manager for Engineering
STAT unde why did she accept a PDR date with so many
undecided items? In fact, why was the proposed user language
unacceptable? -- both language groups were working for her. If
the COMPOSE facility is essentially totally undefined as
STAT
Ihac four months to find that out. Of course,
had thirty.
of now,
The point of these comments is primarily to raise the question of
the suitability not only of the management structure, but also of
the particular personnel. If we are going to expect improvements
STAT from the way they handle SAFE, we must inquire why we
should expect the people now to do better than they_have done
already.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2 / 19 / 8 2Approved For Release 2005/f()1IQ 2A1CIA&tbPt4-196933R000500120004-2
Page 29
Competent managers in computer technology have a hard row to hoe.
They must be aware not only of the people and tools they deal
with, but also of the bounding technology itself and its peculiar
demands and constraints. The managers should have said.-- and
they should be saying now -- that they and their engineers and
systems analysts need to observe actual intelligence analysts
working in their tasks, that they need to try operational
experiments to tune their system components before putting them
together to form the bigger system.
These comments stress thati (itself must exert more leadership
in the direction of SAFE than it has been doing. At the PDR I
heard several times in response to questions about SAFE
capabilities that as going to meet the requirements. It is
part of0 responsibilities to see that.they also provide a
system that does the job, even if a contract cannot be written to
say that in contractese.
4.4 The Role of CSPO
CSPO also has a tough job. Somehow the technical strengths to
gauge and evaluate a proposed system have to be produced and
applied where they will count. As I remarked above, there is
little reason to be satisfied that the uninspected pieces of
design for SAFE are in fact acceptable, since so many of the
people do not seem to understand the operational context that
SAFE must work in.
In fact, the contract was initiated with strong restrictions
imposed on CSPO's freedom of action. It was determined elsewhere
that DIA and CIA were both to be customers on the same contract,
though their needs are very different. Other requirements were
laid on CSPO: contractor communications were to be funneled
through them, and controlled by them; and the details of the
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
STAT
 2 1 19 / 8 2 Approved For Release 200407912': ClL DF I! J0933R000500120004-2 Page 30
contract were by and large imposed on them with little recourse.
In retrospect, these restrictions werera significant handicap on
CSPO's ability to manage and control.
But CSPO must also be assuming more leadership in where SAFE is
going as a whole. Leadership is a hard thing to define: it does
not have to be concentrated in one body, though it often is; it
does not have to know every detail, though it often knows many;
it must have the ability to inspire and excite, to raise levels
of ambition, to bring large pictures to workers at the level of
detail. CSPO does not have that kind of leadership. now, and it: -
needs it.
5.0 CONCLUSIONS AND RECOMMENDATIONS
5.1 Discussion
The problems discussed in this memo are not easy to solve. It
would be perhaps tempting to regard the money and the effort
spent so far as an investment that has to be protected with more
money and more effort. Indeed, at DIA asserts that STAT
because things have not been going well we should be prepared to
do just that:
"Several major design and technical issues remain... The STAT
false start and delay ... has (sic) cost at least $5 to $6
million in effort and a slip of at least 9 months...
-currently (and, we think, optimistically) projecting an IOC
for the-DIA Block 3 in September 1984... The costs to
complete ... are estimated i n project plan to increaseSTAT
$12 million ...
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
now recognize
extended from
personnel are
Page 31
on behalf of senior management ... does STAT
that costs would be increased and the schedule
the original plan... most new technical
just completing the learning curve...
js working with) Ito modify the operating STAT
system software ... both companies also met ... to assess
the ... yet unannounced new
fardware ...
STAT
"Conclusions and Recommendations: The difficulties
encountered ... are not extremely unusual. 1 has gone- STAT
through a difficult learning curve .. but now desperately
needs continuity... We strongly believe the present project
management ... should be supported to maintain project
continuity.
The argument can be carried logically to: the worse a contractor
does, the more he needs extra support, understanding, and
continuity. The memo emphasizes that the Government does not
really know where it stands, and that there is real risk for the
Government whatever it does:
"Even under this schedule there is considerable concurrent
software development... it increases the complexity of the
project and the uncertainty that the whole system will work
when all software is integrated. The risk here is schedule
and cost more than performance."#
But there are obviously limits to the most tolerant contract
management. Supposing it turns out that most of the elements of
the system design are badly flawed: are we really prepared to
keep injecting more and more resources? As we saw above, the
hardware is not settled, and there are major design issues. How
STAT
*Memo fo
DIA, 1/12/82, U-01/RS.
#ibid, p. 3.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
can the Government know what the performance can be?
Page 32
That is to say, there is some level of assessment of the current
state of SAFE which would lead to a recommendation of
cancellation of the contract. The consequences would be deeply
disturbing and traumatic. to the Intelligence. Community. The CIA
has been counting on SAFE strongly enough, but there are other
viable efforts that can provide some avenues for support for the
analysts. But DIA lacks those other avenues almost entirely, and
interruption in progress toward SAFE-like capabilities would be
catastrophic. SAFE has been ten years in its planning and
organizing; one would not like to have to wait another ten.
Since a profound evaluation of the actual current state of SAFE
seems to be essential to rational decisions, we recommend that it
be done. It would seem appropriate to have that evaluation done
by experts who have not participated in the operation or
management of the project so far, even if merely to add weight to
their credibility.
We can see a range of possibilities:
1. The evaluations are strongly negative; in spite of the
negative consequences, the contract should be cancelled, the
pieces picked up as well as can be done, and the efforts
restarted. It must be remembered that the Community's needs
are still there. It is possible that commitments have been
made that require salvage of the work contracted for; the
labor needed for that might be large, but it should not be
allowed to impede the recommended enhancements of the
ongoing PMO and other expressions of the drive toward the
capabilities that the analysts need.
2. The evaluations are strongly positive; once the SUL problems
have been solved, there is every reason to expect that the
design will lead to an excellent and satisfactory system,
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 33
after the Blocks are completed. The extra delays and costs
will still have to be allowed for, of course.
3. There are enough faults and uncertainties uncovered in the
evaluations that the Government cannot justify paying for
yet another learning curve, and cannot trust the current
procedures to produce what is needed. What we have to do
then is more in the nature of getting what we can in the
nature of a system good enough to experiment with,. to plan
with, and to use as a sort of laboratory facility. In that
event, the initiation of parallel efforts, like a Pilot SAFE
built around PMO, could be vastly aided by the experimen-
tation that would be made possible. -
It seems to me much the most likely that the t"hird possibility
will obtain. We might well hope for something better, but the
Government ought not to count on it. The conclusions and-
recommendations below are based'on the supposition that the third
one is indeed valid.
5.2 Conclusions
1. The SAFE project is in very deep trouble indeed. The DCI
needs to know what the status of SAFE is now, and what can
reasonably be expected of it given any of the possible
decisions that may be made.
2. . The big obvious areas of concern,) fare:
The SAFE User Language
Many Aspects of System Design
Simulation, Benchmarking, Performance Analysis.
It is important that those areas be checked by impartial
experts. For example, the failure of SUL represents
fundamental problems in the underlying system design.
STAT
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 Approved For Release 2v0'SF07i'IZ LCIR-'RDP84L-ID933R000500120004-2
3. Other areas of concern are:
Data Base Management System
InterComputer Communication
Structured File Processing
CPU Capacities.
Page 34
There is no reason to trust the proposed design as being
suitable for the applications. These areas especially need
inspection and evaluation by experts who work for neither
CSPO
4. The contractor has never really understood the problems of
intelligence processing and analysis, as represented by the
proposed solutions. to the implementation of SAFE. It is
extremely important that the contractor have that kind of
understanding, and that the work reflect it.
5. Increment 2.5 is not acceptable as a deliverable. The least
that can be accepted is the full Block 1.
6. The earliest reasonable date for the delivery of Block I is
late in, calendar 84; that presumes that much of the current
underlying design turns out to be acceptable and sound. It
should be expected that the extra time needed for the proper
delivery of Block 1 will need proportionate funding.
7. If the current underlying design for SAFE needs drastic
revision, then it must be accepted that much of the
investment thus far in SAFE has been lost.
8. The design of SAFE must be undertaken with continuing
contact between the engineers and the analysts, so that
there is'a real appreciation of the operational contexts
which SAFE must work.
in
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 : OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
5.3 Recommendations
Page 35
STAP recommends that if the SAFE contract is to be continued,
then:
1. The operation of Pilot Mail be continued and extended; the
inclusion of some of the other functions of SAFE should be
considered very strongly. This operation should be regarded
as an integral part of the SAFE development process. Systems
analysts and engineers responsible for design decisions
should be responsible for being aware of experience with
pilot SAFE.
2. An ad hoc panel be set up to evaluate the underlying design
of SAFE as it currently is. It is important that this be
done quickly, and that the panel provide an assessment by
this spring. The panel should be composed of outside
experts. STAP will be happy to lend assistance.
3. If the panel's assessment is favorable enough, the contractor
be tasked to provide an integrated design for Block 1; this
design should allow for further blocks and features perhaps
even beyond those specified in the original requirements.
4. If the panel's 'assessment is favorable enough, a number of
Government/contractor teams be established to review the
existing set of user requirements to ensure that the
.contractor does indeed have a thorough understanding of each
requirement--what it is s meeting it, how the STAT
intelligence analysts would use it, and so on. Establishing
these teams should be spelled out in the contract.
5. Both the contractor and CSPO be made responsible for
providing a greater degree of technical expertise and
leadership in the development of SAFE.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2 / 19 / 8 2Approved For Release 20t/6-kfi AbIAU!415 &4 Y933R000500120004-2
Alternatively, the SAFE contract should be terminated.
Page 36
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 2/19/82 OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
GLOSSARY OF ACRONYMS
COINS Community Online INtelligence System
COTR Contract Office's Technical Representative
CPU Central Processing Unit (of a.compu:ter)
CSPO Consolidated SAFE Project Office
Page 37
DBMS Data Base Management System
DDI Deputy Director for Intelligence used as shorthand
for Intelligence Directorate; used to be-NFAC, National
Foreign Assessment Center
IC Intelligence Community, as in IC Staff
IOC Initial Operating Capability
OCR Office of Central Reference
ODP Office of Data Processing
PDR Preliminary Design Review
PMO
SAFE
SAS
STAP.
SUL
UIWG
ULS
Pilot Mail Operation
Support for the Analysts' File Environment
Systems Analysis Staff
Science & Technology Advisory Panel
SAFE User Language
User Interface Working Group (of CSPO)
User Language Specification
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 STAT Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 OFFICIAL USE ONLY Page 39
Approved For Release 2005/07/12 : CIA-RDP84-00933ROOD500120004-2
STAP OPTIONS FOR SAFE
1. Introduction
On 18 March 1980 we forwarded to you eight questions
regarding the future evolution of the SAFE system and the
relation of CIA SAFE to other community systems (See
Appendix I). In this report we propose various actions
which, if implemented, could yield a more valuable community
asset in the long run. Because of the size of the system and
its complexity, a delay (6 months to two years) in IOC can
be anticipated. Productive use of this delay time can be
made, as we discuss in subsequent sections.
. In our examination of SAFE, we were impressed with the.
need. for a community manager of the ADP-communication
systems. None now exists and SAFE is not being integrated
into an overall community architecture. As a result the
incremental value of SAFE will be less than it could be.
Even without a community manager, the future capabilities of
SAFE could be strengthened and possible steps in that
direction are described in Section 2. The longer term
.questions of technical direction of the overall Intelligence
Community ADP-communication systems will be the subject of
additional STAP analysis and should be considered a separate
issue from that of SAFE.
The evolutionary development of SAFE will require
analysis of how the community uses SAFE. Sections 3 and 4
describe means by which such analysis could proceed. As SAFE
comes on line, it will be essential for future planning to
evaluate its usefulness. A possible means for evaluation is
described in Section 5.-
Finally, we believe that a rich body of experience in
systems similar to SAFE exists and could be beneficially
applied to enhancing SAFE'.s capabilities. To this end, we
OFFICIAL USE ONLY
Approved For Release 2005/07/12: CIA-RDP84-00933R000500120004-2
 ? OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
Page 4u
propose in Section 6 establishment of an Advisory Council on
Technology for SAFE.
This report primarily examines CIA SAFE and its use by
the Intelligence Community. Our emphasis on CIA SAFE derives
from the fact that NFAC stands to . benefit greatly from a
truly operational SAFE. DIA needs center on the accuracy,,
maintenance' capability and general utility of their large
encyclopedic files. These files require restructuring and
improved maintenance capability as well as a high level of
concurrency in use. Our analysis focuses on the analyst
support function of SAFE; this function is of secondary
importance to DIA. DIA's requirement can more easily be met
than CIA's. Thus while we propose a "true pilot SAFE" for
CIA, such an- activity should not impede the development of
the DIA SAFE. Indeed, the lessons learned in the "true pilot
SAFE" will be of use in the evolutionary development. of the
DIA system.
In outlining the options for the future, we are fully
aware of and appreciate the concerns of the SAFE management
office and the Office of Central Reference. .-The steps
outlined below will delay the scheduled delivery of an
operational system, but we believe that the present schedule
cannot be met since such critical items as command language
and central hardware have not yet been decided upon. The
delay we anticipate can be put to use to obtain operational
experience on a "true pilot SAFE." The delays whether
anticipated or not will cause problems with OMB and Congress
and these should be recognized now.
2. Strengthening the SAFE Management
Strengthening of the community management of SAFE is
essential if it is to become effective in satisfying its
prescribed functions, and be capable of expanding flexibly
and responsibly to aid the entire community.
OFFICIAL USE ONLY
2
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 OFFICIAL USE 'ONLY rage 41
Approved For Release 2005/07/12 : CIA-RDP84-00933ROO0500120004-2
provide full-time support with undivided loyalties. They
must, of course, have the rank and authority to deal with
their contractor counterparts.
It is also likely that some outside expert advice would
be helpful; and for 3) above, a standing Advisory Council on
Technology (ACT) would seem most fitting. It is not clear
whether the council should restrict its considerations to
SAFE, or should in fact ultimately deal with'a broader range
of technological problems. These questions are amplified
below-in Section 6.
3. A True Pilot SAFE
Interim SAFE was initiated in 1974 as a set of.
capabilities on a 370/158 run by ODP. Four main capabilities
were sought for the original SAFE project, and they are
still valid today:
1) A mail/message/distribution system
2) Private files available on-line for analysts
3) Public files available on-line for analysts
4) On--line facilities for read, edit, write, and
document production
These four have to be somewhat extended and modified in
detail to match either today's purported goals or the real
needs that underlie the requirements for the system.
The chief uses made of interim SAFE were:
1) To provide limited experience for certain
analysts in order to survey their expressed needs.
OFFICIAL USE ONLY
5
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
2) To demonstrate the capabilities to management
echelons, in order to help with the budget and funding
processes.
3) To derive certain specifications that might
serve as guides for the actual` SAFE system
specifications.
4) To illustrate.-the -capabilities in - the
intelligence environment to possible proposed -SAFE
contractors.
it is important to observe that Interim SAFE was never
used as a pilot system as that term is used in
? ineering -that is, to provide experience with a small
system whose performance is operationally projected.. to be
what the final system ought to be. In practice, of.-course,
the ? pilot system serves in engineering to modify'
requirements and specifications in both usage and
engineering.
In Interim SAFE, we were informed that An general
statistics about usage were not gathered because they would
be "not representative."
The questions that ought to be answered by a true pilot
SAFE include:
1) What are the usage patterns of naive users?
2) What are the usage patterns of experienced
users?
3) 14hat * needs for modifications of SAFE
performance become manifest from the transition of
naive users to experienced ones?
4) What are the user documentation and training
requ i remen ts?
5) What new requirements emerge from the
experienced usage?
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
6) How should the services provided by SAFE be
modified to take advantage of new technology?*
7) What facets of the system can be safely frozen
in design concept? What parts must be carefully
engineered to retain flexibility of function and
performance?
Only a few.of these questions can be dealt with through
the current Interim SAFE. But there has been a great deal of
experience elsewhere in systems similar in nature and size
to SAFE; we note the unique aspects of much of the
intelligence environment, which is why a true pilot SAFE is
needed. But . if , advantage is taken of the continuing
experience of these other systems, it is likely that the
process of initiating a pilot SAFE and interacting with it
to.guide final SAFE development can be speeded up.
The questions above that seem at first glance to be
uniquely answerable by pilot SAFE are 1, 2, 3, and 5. These
have to do with analyst usage of the tools and capabilities
provided to them. We are therefore suggesting that a first
step is to start collecting systematic longitudinal data on
analyst usage of the current Interim SAFE, continuing during
a conversion to a true pilot SAFE.
There is little doubt that the general capabilities
sought can be provided on a small scale by any of a large
number of current installations outside the community, as
well as inside. At least four members of STAP have had wide
experience with such systems. Such systems also provide some
of the extra capabilities that are not now planned as part
of SAFE, / but, that are considered highly desirable,
including:
1) collaborative capabilities, whereby several
analysts can simultaneously and remotely collaborate on
the same task.
*For example, good split-screen graphics terminals can
take advantage of more powerful editing capabilities than
previous terminals.
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 OFFICIAL USE ONLY Page 44
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
2) general communications access to remote files
over community or common carrier., lines; this is
relevant to making SAFE truly a community-wide
resource, as is being strongly urged by certain members
of the IC staff.
3) new. evolutionary, user languages, including the
capability for user on-line control of multiple jobs
simultaneously.
The actual implementation of pilot SAFE can therefore
be handled in several ways:
1) Use the interim SAFE now running. in ODP.
Upgrade its capabilities, installing - the above desired
new capabilities, aiming at an integrated single system
for continuing development. .
2) Use. an existing system from outside, but
installed at the CIA, using outside contractor support
as well as in house personnel.
3) Use an existing system at an outside
contractor's installation, sending analysts on TDY to
provide the usage, etc. The difficulties involved in
this option are obvious, and it is included only for
the sake of completeness.
The advantages of the first option are that interim
SAFE is now running here, that both users and system
personnel are familiar with it, and that it performs some of
the needed capabilities already in the desired intelligence
environment. The disadvantages are that some , system
reprogramming will. be' necessary to bring it to state of the
art, and that this will likely need system architecture and
programming resources beyond what is available here.
The advantage of the second option is that such a pilot
system is nearly an off-the-shelf item, and could be running
in the CIA fairly quickly. On the other hand, the special
requirements of the community environment would enormously
delay the user population as well as the systems and
programming personnel at the CIA.
OFFICIAL USE ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
 Ur'Y'iCiAL U5t ONLY
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
In summary, we strongly suggest option 1, i.e., use the
current interim SAFE. A conscientious effort toward the kind
of pilot SAFE being discussed here entails:.
1) Increased technological resources available in
personnel., plus somel contractor personnel, to
keep the former honest, as it were. A total of six
people for. the first eight months would be.needed, and
then perhaps dropping to four on a continuing basis.
2) Enlarging the user population and the user
studies. This is so important, and has so many
ramifications, that is the subject of a separate
Section 4.
3) Establishing a direct COINS link that. will
-enable study and experimentation by analysts in
community-wide access and retrieval.
4) Initiating data gathering and analysis,of usage
pattern and changes in usage patterns by analyst users
(see also Section 5)
5) Testing prepared user languages and other tools
as soon as possible.
4. The Users of SAFE
There are two main reasons why the population of users
of Pilot SAFE must be enlarged and modified from that of
users of Intermim SAFE:
1) We need to find out answers to what users do,
how they do it, and how they change.
2) We need users to find. out what SAFE can do
for them, and how it can be responsive to
their developing needs.
The first dictates the initiation of the continuing study of
user patterns of usage that we have already mentioned. An
IBM study of a somewhat different user community illustrates
the approaches and the attacks that might be considered; it
OFFICIAL USE ONLY
9
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2
25X1