SUBJECTIVE EVALUATION FACTORS, A PROPOSAL

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP83M00171R000600060008-8
• Release Decision: RIPPUB
• Original Classification: T
• Document Page Count: 4
• Document Creation Date: December 15, 2016
• Document Release Date: November 12, 2003
• Sequence Number: 8
• Publication Date: April 15, 1980
• Content Type: MFR

Approved For ease 2004/01/20: CIA-RDP83M00171op 0600060008-8
T 0 P S E C R E T
15 April 1980
SUBJECT: Subjective Evaluation Factors, A Proposal
FROM.
What subjective evaluation factors (i.e. qualitative judgements)
should DASITT utilize to compare or contrast the various alternative
imagery AUP-T system support network? %
In recent weeks, the DASITT had identified an alternative, around
which a growing consensus is developing, which is likely to be more
costly that PM plans, even after these are adjusted upward in resource
requirements. Likewise, there are four other alternatives which DASITT
had described/costed and for which DASITT believes satisfies at least
the minimum user requirements for ADP-T support functions to complete,
accurate and timely imagery processing activities. If DASITT does not
believe an alternative meets the above test, we must eliminate it from
cost and consideration.
We have already used subjectivity and sometimes difficult to
quantify parameters in our costing model. But, we are postulating that
sufficiently accurate data (model parameters and transaction numbers)
has been collected to determine the approximate cost of any alternative
support op.-scan. Or we will modify such parameters until we are
"satisfied' that our :lodel has fidelity and is rational. 111 thus seems
that we are in the arena of cost-effectiveness considerations. I mean,
we believe that some alternatives have features which produce a i~etru:rk
which is more effective than other alternatives. In other words, each
alternative has a different cost-effectiveness ratio (efficiency). What
then are some subjective evaluation factors which might be used to
develop reasons why some one alternative is superior to some other
Cony - -of 8
Approved For Release 2004/01/20 : CIA-RDP83MOOl71ROOg 600060008-8
25X1
25X1
25X1
 Approved For ease 2004/01/20: TP CIIA--RDPR8 EMT0171 0600060008-8
Any DAS1TT alternative can be made as effective as any other
alternative if cost and time resources are unconstrained. But cost and
time are real and legitimate constraints. Therefore, one alternative is
more or less effective than another when cost and time are constrained.
Much of work has resulted in alternative costing. It
has been quan i o the degree possible within DASITT constraints of
resources (time and money). But it alone is not enough.
Find subjective factors to evaluate the effectiveness of our
alternatives irKiependant of cost. Lets call these factors, Qualitative
Effectiveness Factor (QLF's). I propose the following:
1. Maximize the Network's Adaptibilit to change. This will be
responsive to new imagery collection and exploitation
requirements or modifications to organizational responsibility.
2. Maximize the network's compatibility/similarity with program
manager and ADP system managers plans. This will reduce
dissatisfaction with system performance and requirements
satisfaction.
3. Minimize the Network implementation time. This will reduce
development risk, improve schedule adherance and milestone
accomplishment.
4. Minimize the number of different procedures a user must execute
to accomplish a needed function. This will avoid confusion,
reduce user dissatisfaction and minimize training.
5. Minimize the quantity of non-planned duplicative and redundant
information stored. This not only saves money but increases the
opportunity for improved standardization, completeness and
accuracy of information (i.e. ias not scattered all over the
place, out of Cato or inconsistant).
6. Minimize the quantity and duplication of new software
development required. This is conserving of`expensive S/W
resource, focuses primary responsibility where it should be and
improves compatibility and inter-operatibility.
7. Minimize the quantity of duplicative imagery substantive
intelligence reports produced. While this seems primarily a PI
resource and management issue, ADP-T systems which do not
discourage this activity will be more costly and less responsive
to user needs, too much information, too many places, may
discourage analyst use.
II
T 0 P S E C R E T
Approved For Release 2004/01/20: CIA-RDP83M00171R0004
25X1
 Approved For ease 2Q040dI3D : CSIAERDPR81M?0170600060008-8
8. Minimize the number of system locations within the network with
24 hour/day, 7 day/week availability requirements. These are
expensive, high-risk systems to produce, require extensive
resources to operate and maintain and many such systems will not
.benefit from the "economies of scale".
Visiting QEF's:
I propose that DASITT rate each alternative against these eight
QEF's. I would use four degrees, like poor, fair, good, best with a
numeric equivalent of .3, .5, .75, 1.0, respectively. I would like to
calculate the alternative QEF "value" on the basis that:
* all QEF"s are of equal weight
* Individual QEF weight is the inverse of rank position
* Individual QEF weight is 1/2 the inverse of rank position.
Summary :
What I've attempted to do above is quantify subjectively to some
degree. If this is not desired, an alternative is to simply discuss
these factors qualitatively. A final and different approach appears
below in a list of considerations DASITT may wish to work against some
other way in Section 6 of the report. These are:
a) Ease of Use
b) System Survivability
4-c) System Development Risk
d) Modularity and Expandability
e) Failure in performance standard
f) Impact of single system (node) failure
g) New telecornmunicaitons reliance
h) Access to needed Data provided, how?
i) Completeness of Information, where?
~- j) Timeliness of update, when?
k) Soft-copy ramifications
1) Ease of new technology transfer and use
m) Avoidance of non-planned duplication
n) Availability requirments (Reliability requirements)
o) Distributed data base features
p) Distributed function processing focus
q) Impact (cost/problems) of capacity underestimation
r) Consequence of failure
-s) Adaptibility to new/changing requirements
III
T 0 P S E C R E T
Approved For Release 2004/01/20: CIA-RDP83M00171R00
25X1
 Approved For ease 2004/01/20: CIA-RDP83M0017D0600060008-8
T O P S E C R E T
I hope this memo is useful. I would suggest that at any given cost,
the alternative with the highest QEF score will be more effective than
any other, and this should be the DASITT recommendation. I further
suggest'that additional resources ($'s and time) expended on the
alternative with the highest QEF score will result in an even more
effective ADP-T network than would the same amount of resources ($'s and
time) spent on an alternative with a lower QEF score. The bottom line
is then to assure the network development (select the alternative) which
has the highest QEF that our dollar resources between now and 85 can
afford. I believe this may well be Alternative 5, but it would be
interesting to use the QEF's as I've suggested and find out.
25X1
Distribution:
Original:
Copy 2:
3:
4:
5:
6;
7:
8:
25X1
IV
T 0 P S E C R E T
Approved For Release 2004/01/20 CIA-RDP83MOOl71R00061060008-8