APPENDIX 2 THE NARRATIVE REPORT

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP00-01458R000100130011-6
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 21
• Document Creation Date: December 12, 2016
• Document Release Date: December 11, 2001
• Sequence Number: 11
• Content Type: REPORT

Approved For Release 2002/01/25 :CIA-RDP00-014588000100130011-6
Rppenc3i x 2
The Narratirre Report
Approved For ~e~~~1~1~~~~~-A ~t~~Q1~>3~00~~~~3001.1-6
 ax.s~~~.z~.lst ~ ,~?QQi'~Q'~~y~'~~EA~EJ~O~A4458R000100130011-6
actual test scares but only their interpretation of them.
If the request for a write-up is general, that is,
na specific component or job within the Agency is ident--
.. ~-
The Narrative Report
When a person outside of the Washington area completes
a formal application for an Agency position, he or she is
assigned to one ~of the field test settings to take PATB I
which consists of five cognit ive tests and the Strong-
Campbeil Interest Inventory. The answer sheets for these
candidates are mailed to Washington and delivered- to the
office of PSS. Candidates from the Washington area are
given PATB T and PATB II and their answer sheets are also
sent to PSS. Answer sheets for both groups of candidates
are scored but nothing mare is done with them and no one
sees them unless a unit of the Agency requests a write-up,
i.e., a report of performance on the tests. This report. is
prepared by psychologists in OMS/PSS who will not give
fied, then the write-up tends to be non-specific. If a
component of the Agency or a specific job is identified,
then the write-up is supposed to be focused on specific job
r
profiles for that componen. We encountered the terms
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
~~7^ ~ r: r~ r f1 i Y ; f
Q 'p ]t.J Cr~:+w~ Mi: tf' R F t.: eta ~ ~ '4"r r t~.-~4'ru ,4 4:4f ~ -A~
 4 n a 9 iii'"'?~ ',~"`i t7='r"~3Z9 n 1i.s~ i5 R7 !-
Apprcj ;~~o'~?R~1~~~~C~2b1~ ?~R~10-~~18~2000100130011-6
l~~1 : I'ti r r ~y> $ 1 ~: {~ 5 ti 6: C' t, 'i a .~ ,+ u a.
"jQb profiles" and "job-group profiles" a number of times
i'n memoranda reporting interviews with the Chief of PSS
or in memoranda written by the Chief of PSS. Ta us,
these terms meant that a systematic detailed analysis of
prafessional jabs had been done to determine the knowledges,
skill s and competencies needed to perform the professional
jobs satisfactorily. Further- investigation an our part.
proved this interpretation to be incorrect. Na systematic
job analyses have been done. Ta the psycholagicaT staff the
terms mean test profiles that have been generated for a
ntamber of jab groups in the Agency.
Although we tried to find out how these test profiles
were generated,, we were unable to do so to our satisfac-
ton. In a memorandum written by the Chief of PSS to the
DDA (25 July 1979), the Chief states that test profiles
for a number of jobs in the Agency were generated as part of
the initial development of the PATE. However, when we asked
him questions about this and other aspects of the initial
development of the battery, he stated that there was no
material available on the early history bf the development
of the test except that contained in Test Data Book, Na. 75,
dated 1 July 1958. No test profiles are included in this
~ l t ~~ ~ L b ~~ 6) .. l~: J ~q 3..
1 y ~? ~ .r v? r
4 ~? 1 ~' ) ~~. '
.i:.7 SS: ~^Fi F{:y+tl Li..i 4V 11`~ ~?r
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
a
 F~'a'~~ ~ff021ffi'1?/~~~ ETA=F~[3PC~(~4100130011-6
source for any Agency jabs.
In a memorandum reporting
briefing for pDA done by 'the C/PSS and
?~ ~ ~ members,. we noticed that he had stated each psychologist 'had
a test data book to assist him or her in evaluating test
We asked to see the books, but the C/PSS told us
---~ that no such boaks existed. According to him,. each new
psychologist is trained by an experienced psychologist, wh
has all of these data in his head.' cm
We were permitted to see, but not to examine closely ~,, ~
,.~ ~~
because of security reasons, a sample computer print-out of ___.
the test profile for one applicant. As the G/PSS ex- /
~J
plained the. print-out to us, the test profiles and job ?
~~j -ate-v~%t
prafi les appeared to be ,generated from the studies. that had u.?ti~
~
,~~/
~!
been. done on PATB Over 1 tS ?0 vP~rs of itca _ Wes haves r _ .1,r~/~~ ," M
basis of that review that there is no consistent or convinc-
ing evidence for the .iob-related validity of PATB. We also
~e ~~~- pointed aut i n that Appendi x that al l o_ f ,the studies used
..
.~-~smal l' samples composed 1 argel y of white males and onl y two
of the studies had been cross-validated. In neither of the
Sa'ut'~i~,. cross-validation studies were the findings of the first
~ ~~.
,~~~
,.
po,~'
.may results.
~~
~
study verified, which indicates that the jab-related validity
Approve I~"~~d~~L~`2'0'd1/~1/25 : CIA-RDP00-014588000100130011-6
 Approv
~~gJ~e~~Q~O~~~~,? 1C~4:,FtDP(~Q~~]1?~~~~0100130011-6
~4'Ci3 u'lL J it Li. d ~i 45.4.. LL ~ i ~.,L 41~ 4L arL~LLyI'~., ~~t?~"L. Y
of PATB still needs to be demonstrated.. Any test profiles
for specific jobs in the ~Igency that were .generated- from
these sources would be unreliable because of the small
samples used, of extremely doubtful. validity, and probably
~~
biased against minorities and women because thes
_.~.._
underrepresented in the samples used in the studies.
Since we are nat absolutely sure that the test prafiles.
.and job profiles have been generated from the studies that
have been done on PATB over the past.20 years, let's assume
that. they were .generated at the time of the initial con-
struction of PATB in the 1950'5 by testing personnel in
professional jobs at that time. Would such test profiles
constitute evidence for the validity of PATB? The answer is
no. The fact. that a group of current employees had a
particular test profile is merely description. It does not
provide the evidence needed to determine whether applicants
for the- same positions must have the same test profile to
perform satisfactorily on the job. As a matter of fact,
since the test prafiles for the original group represent the
average scare for a number of individuals, mast of the
individuals in the original group would not have had that
profile. If the test profiles were based on an early
T`~ ~~c~ ~;. {,~, may. ~?r. ~~
c .fir I
{ ~' ,tso, ij9! ~{1~ ~ ~ 4~~ a y~ try JrS ~ r S 4~ "?~ :... ~~ d ~! s,
~~~~iTw K~A u~li'~~N~YW L~.~U ~'1 4L L L? ~ ... ..
Approved For Release 2002/01/25: CIA-RDP00-014588000100.130011-6
 ~iS.~i~~i~8~~u~G~CkC Y~1'.. ~liiI Cart kE'~i~ ~4;.~~~, ~,.td4~.
A~~prbvgd For Release 2002/01/25: CIA-RDP00-0145880001001
},. ...
.would be unfair' to minorities, particularly, because they
group of employees? their use with the applicant group
were underrc~nr~sP~,n,ted in the 1950's employed group.
Predictive Aspects of the Narrative Report
The section of the narrative report that is most
directly affected by using an invalid and unreliable data
base is the last one, Comments and/ar Recommendations. !Je
examined 21 "sanitized" narrative reports, 13 of which had
this section completed. In 11 of the 13, the narrative
report recommends the applicant for a specific jab or to a
specific .unit of the Agency. We rechecked all of the validity ~r~,~;~j
scares on the other scales when there is no evidence that
_ _ ~.
these people could not pe~farm satisfactorily on the job.
This practice violates EEOC guidelines onifairness as
1 rk
d
test results. Such recommendations tend to Tead to the
exclusion from consideration for employment those individuals
rrtho score low on the cognitive tests or who have "unfavorable"
without adequate validity data promote unfair use of the a
~(.
Recommendations for specific types of employment made
a--
suppart any of these recommendations. This troubles us.
data that we had and could find no evidence that would''?
in
icated in the quotation below.
6 ~,+'l ~?u~J Y~IY~~~~iJ 1+d4 Jr Yvr wj~ _~~~~~. I, C~`-f
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
 Approve ~~~~~~~~~~~~ ~f~1i~P~1~8'$~~0100130011-6
lJhen .members of one race, sex or ethnic group
c:haracteristicalTy obtain lower scores on a
~. se.~ection procedure than members of another
group, and the differences in scores are not
reflected in differences in a measure of job
performance, use of the selection procedure
may unfairly deny opportunities to members of the
1/
groups that obtain the lower scores.
Before ending the discussions of the Comments and/or
Recommendation se.ctian of the narrative report, we think
that we should make a few additional comments concerning
some statements frequently made: by the staff of PSS that are
related to this section, In a number of reports of inter-
views with the Chief and staff of PSS, the C/PSS is reported
as stating that no cutoff scores are used far PATE, that
test results are never used in a pass/fail context, and
that PSS has no role in hiring. decisions. Although it is
true that no single cutoff score for each test is used to
screen out applicants and that the pass/fail designation
1/ Equal Employment Opportunity Commission. Uniform
guidelines on employee selection procedures (1978).
Federal Register, August Z5, 1978, 43, (l66), p. 38301.
?L~~~f T~nF'~~~ ~~Tp,,~" e!~N~~F``~1~~G` ~r`~'r.-- C.,n!`Y,'~rl~'
7? t~~Ir ~I ~ : i ~ t f ` ~y ` r 5 ~ ~y ~~~~.i irri. ~~.t .w '.Jd it ew {
Approvec~'Fo~ release ~0~71rb1/~15 : CIA-RDP00-014588000100130011-6
 A' ? l ~ ~ry~~~ a~~cy ~~ Y~ +ti S:: a'J gan`r: f ~: an .''r
Appro~~~ ~; ~Q~Z10~'t,l~~'=~hF~`DTr[~Q-0{"Y4~8'F~000100130011-6
T ~~~
w~^~~`-
,~{, or.
1S ~ ~"' 4~s e
~,5,-~--~~.
is not directly used, indirectly -both are used when the
un'validated equations and profiles are used to make a
recommendation to hire or not to hire an applicant.
To say that PSS has no role in hiring decisions is
dissemb1ence of the highest order. PSS, through its narra-
to dire ar not to hire are greatly influenced by the narr-
ative report, particularly the recommendations made by PSS.
From these interview reports, one would conclude that the
failure of PSS to recommend an applicant is equivalent to a
"kiss-of-death" for that applicant in some of the units,
This makes the recommendation section of the narrative
five reports, plays a significant role in ;some hiring
decisions. From reports of interviews with people in
different units of the Agency who apparently have the
responsibility far making the final selection decisions, it
is quite clear that a significant proportion of the decisions
report even more troublesome because PSS makes its recommend-~ ~4 ~^~
atianS with a level of r_nnfi dpnrP anti fi n~1 itv thafi i c not A i~'~
supported by the validity and refiabipity of the data.
We have recommended in Appendix l that operational
use of the multiple regression and discriminant analysis
equations be discontinued until the ~4~~'~1Q~ls have been
~ y is ~Sa~ '~`'~yt1 ~ ~ ? ~ ? r i n k1 +'.,?
e3 ~3i'tx.~~#~1 C~~'~ ~ ~. ~,Y _iti ~~ C .. c:.a ti~:t .s"~. 4+794.
{{ N? r(r, Fi :y ~~
?ii~,~41~6~:LJ~~.~N~.~i a.~~.l V A JN~--
Approved For Release 2002/01/25 :CIA-RDP00-014588000100130011-6
 t ~~ `~E~ ~
Appr+bver~`F~ir~Y~~a'~"20(t2i17'1~T~5 : CIA-RDP00-014588000100130011-6
~ ~~'~
?~`~r~o~ ~ cross-validated. Since the recormendation section of the
`~ narrative re ort re resents the o erational use of -these
vy/ P P p
equations, we think-that their use far this section of the
.report should also be discontinued. The best use of this
section of -the narrative report would be far summarizing.
descriptively the strengths -and weaknesses of the aplicant.
narrative report is to provide a clear, accurate and meaning-
ful description of an applicant's characteristics as revealed
one, and probably the mast important, function of the.
f?escri five Aspects of the Narrative Repast
tk-en use this description together with other sources. of
PATB. Persons in the various units of the Agency can
~wk ~ in?Formation about the candidate such as the Personal History
intended to be descriptive. Two sections describe perfor-
mance on the intellectual tests of PATB and one section is
devoted to each of the following: (1) measured vocational
Six of the seven sections of the narrative report are
or all candidates. -
Statement, transcripts from educational. institutions, and
letters of recommendation to arrive at employment decisions.
_.
Sy using a variety of sources of information, persons in
the units should be able to make employment decisions
that are beneficial to the Agency and fair and equitable
~?h~i9~~' is `I ~'.
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
 d'~~~
` ~
f
~
~
~
~~~
-~Z ~~
r1
~~
~R:~
..c
aq
~ q
'p"'~"nom'
~
ti
~ ^7- ~r
~.
A
Ff
~ y/
~
}}~y
`
~
'
3j yp~~
3
--
~
~~A
j
fir ~
~
7
7
)
i
.~=
R_~
'1_ ~y~~
A~.i>
l { f 7 3'K I it ,/ y CM
F~
"4
. G
wA
Y
.,
Tl i 1. d
a+:
~ 1i ~ 7777:Y~~~.
~ Q4r'.~f
t13 M i~f Y
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
interests, (2) foreign language, (3) writing ability, and
(4) attitudinal and personality factors. The value of
these descriptive portions depends upon two major factors,
the validity and reliability of the individual tests and how
well written the descriptions are.
In Appendix I, we indicated that the content and
construct validity of the individual tests comprising PATE
has not been determined. Although this limits the value of
the description, it does not make it completely useless.
The content of the Vocabulary, Reading, Contemporary Affairs
Test and Numerical Operations tests clearly indicates ghat
they are appraising what their titles suggest they are
appraising. The Essay test is a writing sample and directly
,~~,
appraises one type of writing ability. The Strong-Campbell.
Interest Inventory is a standaraized instrument which
provides validity data in its manual to establish what it is
appraising. However, we cannot infer from the content of
the other tests and scales what they are measuring or what
w~4
We suspect that the Figure htatrices test measures ~;,c~,
c~ ~~-~~
abstract reasoning because tests of this type usually do; r ~~~ ~
~ ~~
however, one cannot establish the validity of a test just by ~w~`
determining its superficial similarity with other tests.
-There are no data to indicate what abilities the Language
Approved Fork
u:L
 App . ~ ',~ ~~-00?Q~~~R000100130011-6
Aptitude, Interpretive Reasoning, and Considerations tests
are appraising or what is being appraised by the work
attitude and temperament scales. These types of tests and
scales need to have their construct. validity established.
factor analytic studies would have been extremely useful. in
determining what these tests are measuring but, unfortcxn-
ately, no such studies are available. Without construct
val idity data one cannot say anything about what a score
means. The psychologists who write the. narrative repast
have tried to avoid this issue, particularly in reporting
performance on the cognitive tests, by just listing the test
by name and giving an adjective such as average or excellent
or poor to describe the performance. As a result these
descriptions are atornistic and. fragmentary which makes
ir~possible -for the reader to get a clear,- comprehensive
picture of the cognitive competencies of an applicant.
We have also i nd icated i n Appendi x I that the rel i a-
bilities for many of the tests of PATE are distressingly
law and, as a result, the standard errors of measurement far
these tests are relatively large. We found that there were
no reliability data for minorities on any of the tests and
no data far women on the work attitudes scales. In the
absence of such data, one should be extremely tentative ire
10
Approved For%E.~~~~~~~2~: CAA=F42DP0'+~j011~0~1`'b~ 30011-6
 a.~r _99. r.~''?h pr-?yy!jt}j} nt~"~-~,t -k Tire 'f %. 6~~:?X
,1~{. :1 li kG'.: t~ CM ~i ry t G-1n ~ w v ... is 4~ ~?.~~. (1'.~~
A rov d For RRelease 2002/01/25: CIA-RDP00-014588000100130011-6_
? int~~pre~~ng~ their performances on the tests. The nar-
native reports -that we examined did not take this into
account. They described test performance with the same level
of confidence far all applicants and for all tests. This is
troublesome because it leads the reader, who is usually
naive in testing, to .ascribe a-level of accuracy and finality
to the performance that i s not merited by the rel i obi l iti es
of the tests.
Two parts of the descriptive sections of the narrative
reports, measured vocational interests and writing ability,
caused us considerable concern. At the present time voca-
tional interests are appraised with the Strong-Campbe]1
Interest Inventory. There are no Agency norms for this test
and no validity studies have been done to determine whether
scores on this instrument are related to job performance.
4Ie suspect that managers in the units are not aware of this.
In addition, in 11 out of 21 reports that we read, the
scares on this test were misinterpreted. In reporting t-hese
-----~_ J
scores, the psychologists used phrases such as goad verbal - ~,
persuasive skills; a rugged, practica l-individual; outgoing;,
and strong organizational and supervisory skills. The
Strong-Campbell Interest Inventory appraises none of these
1/
characteristics, and the manual specifically warns users
against these types of interpretations.
1~ Campbell, David P. Manual for the Strang-Campbell
Interest Inventor Stanford, California. Stanford Uni-
versity Press 7 pp 17, 21, 87? i~14~ pp~~:~- Lug `
`
,~xz t1'..i Arta"?.`~~1 ~N:.,''.~~~~a~ ~'i''~i~t". ~rs~ ~.ro l~~ ~
Approved For F~elease 2002{01/5/: CIA~~~-01~4,~8~001~1~3~Q.OA~' ~r~ ,,2rc-7,,yc~.,,~
~7.~ nL n 1 .A'~r?.~i~'~1 A
/!/1a'" /
 Approved For Release 2002/0~/~.5e,~.:GJ~A,R[~PO(}=6'h4~R~OQp100130011-6
~Ma ~' ~py ra h. ~ t~Lxl' ~~ ti.. 4x~':v-.a.. ~x!EV`w
~p#+shl1&Q ieb6s~" 49i C6 ~ d w. .d .f ~
The Strong-Campbell Interest Inventory is an extremely
c~amplex instrument that yields 158 scores - scores an 6
General Occupational Themes, scores. on 23 Basic Interest
Scales, scares on 124 Occupational Scales, an academic
orientation score, an introversion-extroversion score and
3 administrative index scores. It is impossible to tell
from the narrative description which scare or scores are
being interpreted. However, it is quite clear that a number \\?
_~
of the psychologists who are writing this part of the--
d-e~Ya mw,w~'a~ry. w~..o( M.,~,,,o.u~a fior~i~,
narrative description do not unvderstand the instrument. ~or
example, on one narrative report the following description
was given: "Measured vocation (sic) interests are very
bread, encompassing,- virtually every occupational field..
This type of profile suggests a highly-motivated, versatile
indiViduai, eager to enter the world of work." An individual
who has a ]arge number of high scores on the interest
inventory has marked ?'like" to an exceptionally Targe nu eir
of the items 'on the inventory.- This type of person is
discussed on page $5 of the manual as follows: "There is no
single characteristic descriptive of all persons with high
LP's, (note: percentage of like responses), but same
combination- of the adjectives "enthusiastic," "curious,"
"shallow," "unfocused," "energetic," "manic" will fit many
12
Approved Fo ~~~~
Y, ~wuli.:,~.i
~~{~~'~,;~~~A`~~1~6 O'~s45$R'~Q" 130011-6
 Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
t wo Li ~. d
of them." As one can se.e, the interpretation given in
the manual is at variance with the PSS psychologist's inter-
pretation.
ale think that it is impossible to compress into Td ar
fewer typed lines a meaningful interpretation of a complex
instrument like the Strang-Campbell. Interest Inventory. To
try to do so tends to misinform rather than inform the
reader. Far this reason and also because no validity data
or norms -are available for the use of the instrument for
selecting personnel in the Agency, we recommend that no
report be made to the units of these scores.
ability
the wide variation in describing the candidate's writing ~ ~ _ ~ ,~/~j
ar judging the writing sample has been determined.. Third,
bility of the writ?ing sample nor the reliability of scoring u ~
w
sample has not been validated. Second, neither the relic- ~~
The part of the narrative report that describes the
writing ability also caused us concern. First, the writing'77
~., . ,.... ~ ~ .~~.., ~ v~ J1.41 ~ ~~y v~ J u~y ~ uy ~~~c rr~ r ~ ~ ny ~cxu~~ ~ c .
Fourth, for some unexplained reason, the report of writing
indicates
~,J~ 1 _. ?ij~ 1
~.,..lw/V 1 hL~"""
ability includes the candidate's
~ "a'y~~cl~oul_ not appear to serve any useful
,~ ~# ,,,,,,~,.el~
fio.~urs ~~ a ~~s f i.i,.
{~ Sew ~ . __.d W~'~c ~ _ P
e1u,,. cn~.i-~ ,d.ouv~
ovr~. ~.'..` wctNS w`"
d~
,~tvt
n.
b~ ust
there...are no established ~ `R"
claimed ability which does
 Appro~~~''4~F~~~~{~21b~/~~a.~~AiC~~~
~0~~000100130011-6
The actual. description of writing ability is done, on
the average, with 15 words and the aspects of writing
ability that are commented upon vary considerably from one
narrative report to another. We suspect that much of th~'
variation in the reports of writing ability are due-more to
the idiosyncrasies of the psychologists writing the report
than to differences in writing abilities of the applicants. -
This is bothersome because one purpose of -the narrative
report should be to supply the managers' in the units with
comparable data on all candidates. The descriptions being
presented are not comparable; they use ambiguous terms and
leave too many blanks that the managers must fill- in far
themselves. For example, dues the phrase, not badly written,
mean the same as demonstrates well-developed writing skills?
If no comments are made about errors in spelling, grammar
or syntax, does it mean that the candidate made. no such
errors or does it mean that the particular psychologist who
wrote the description did not think -that the errors made
were worth mentioning?
We have three major concerns about the section of the
narrative report that presents the description of attitudinal
and personality factors. First, the reports assume that
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
 Approved 5~df'R~r~a~~
~'~ the validities of the-work attitude scales and the tempera-
~'~'?rr'~ ~'
ment scales have been established when, in fact, they have
~~ ~,~
~`'';a-~ not been. The inferences in the reports that the scales
measure such attributes as gregariousness, introversion,
cautiousness or introspectiveness are completely unjustified.
Sec and, the descriptions do not reflect the low reliabilities
of these scales for white males, and the absence of any
reliability data for the work attitude scales for females
and far minorities. Third, occasionally the psychologists
appear to forget that they are describing self-reports of
the applicants and describe, instead, actual behavior. For
example, one report states '"He is an outgoing type,. who
eagerly takes part in planning social activities and informal
gatherings." This statement describes actual behavior and
the psychologist had no data on the actual behavior.
Fortunately, these kinds of misstatements do not occur very ':
~;
frequently.
In describing the performance of an applicant on the
intellectual tests, the psychologists-..use adjectives to
describe the performance and different labels to identify
the tests. The Test Data Book No. 15, 1 July 1958, gives
the following adjectives that were to be used to report
intellectual test scores: superior, top 5%; very high, next
highest 15%; high average, next 20%; average, next 20%; low
average, next 20y; poor, next 15%; and very poor, lowest 5%.
In the 21 narrative reports that we examined the following
[~
Approved r~Q~~~~t"
_._C : G : ti ~ 1: L .J% 6 O lul Y. ~1
~~/~~; CIA-R?~0~y~5 00130011-6
 A rove ; Dodd @ ~1
ad~,~ectives were used: very .superior, superior, excellent,
F~:i~g~ average, above average, average, fair, weak, poor, very
door. We .were not able to find a set of guidelines for
translating. the scores on the intel]ectual tests to this set
of adjectives. However, it is possible to match the adjec-
tives to the coded scores used for the tests as shown by
the following: 9=very superior; 8=superior; 7=excellent;
6=high average; 5=above average; 4=average; 3=fair; 2=weak;
l=poor; and 0=very poor. If this is indeed what is being
done, then the psychologists are making finer discrimina-
tiatis in the test scores than are justified by the reli-
abilities of the tests. Na explanation of -the meaning of
the adjectives is provided on the- narrative report and it is
highly probable that the user of the report will misinterpret
what the adjectives are supposed to represent.
The label used to identify the Figure Matrices test
varies in different narrative reports;.sametimes it is
identified as abstract reasoning, some imes as non-verbal
reasoning, and sometimes as the ability to deal with pic-
torial symbols. Our experience in testing indicates that a
person who is naive about tests will place a higher value on
th-e -scare from this test if it is labelled as abstract
reasoning. than he will if it is labelled non-verbal reasoning.
Approved'~~'~'F~~I~~~~/2~k~G~1iA`~~0~"(~ 500130011-6
 Appro~4.~~~~1~~ ~2~Q`1 ~~5~*~#~~R~O-~ ;~~8~2000100130011-6
It would be very desirable to require everyone who writes
the narrative reports to u'se the .same labels for the test.
in this instance, since there ~s na validity evidence that
demonstrates that. the Figure Matrices test i.s indeed measur-
ing abstract ar non-verbal reasoning, it would be better to
identify it simply by its title, Figure Matrices.
We noticed in the section reporting the performance on
Part i intellectual tests a statement about the applicant's
,claim as to the percentage of the class where his or her
c:c~-~ - -
col1ege grades fell. We question the usefulness of this ~-~ ~
/
piece of information. fur experi~ce has shown that ~~,n,~ '~'
students are quite accurate in report ing their grade point '~ ~ ~B'
a
averages, but that they are much less accurate. in
identifying ..~,~~r~L~-~`~
in what percentage of the class they fall. We also question
the inclusion of this information because its meaning is not
clear unless one knows the selectivity of the institution
attended, the distribution of grades given in that insti-
tution and other factors such as whether the student
worked full time while attending college. Since applicants
are supposed to supply college transcripts with their
Personal Nistary Statement, managers in the units should
have the actual transcript and do not need this self-report
of grades.
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
 In the 21 narrative reports that we exansined, we noted
that the psychologists writing the reports missed many
opportunities to write an interpretation that would lead to
constructive and fair use of the test results.
1/
Standards state the test user should consider alternative
interpretations of a given score. Since the psychol-
ogists are the ones who are writing the narrative reports,
they are the ones to whom this standard is addressed, A
goad example of the failure of the psychalogists to follow
this standard is the report given for an applicant identi-
f-i~d as coming from a bilingual home. The report states
that Abstract Reasoning is high average; Arithmetic Reasaning
is average; Reading. Comprehension is weak; and Vocabulary is
very poor. The psychologist failed to paint vut that this
pattern is typical for bilinguals. A true bilingual person.
processes verbal information much mare slowly than does the
monolingual person and tends to be penalized on verbal tests
that are, timed. The Vocabulary test i s highly speeded and
the other verbal tests are somewhat Speeded even for mono-
linguals. If the applicant is truly bilingual, then his
1/ ~ Standards for Educational and Psychological Tests.
l~ashingtan, D.C.: American Psyc o ogica ssaciation, ,
P 72.
18
i~P,i :fly i; ~y~'t.':.-. r
~{ ~G-~,~~t.y.~~ r. 1 ~.
?~1SkJYri~~dr.~~~ i:L ~u
Approved For Release 2002/01/25: CIA-RDP00-014588000100130011-6
 Approve ~~~n9r,~JF,@~~~a?~~~~%~1f~2~.:~1}Y?`FZ~P~p+0~4~8~FOt0100130011-6
~Rd*OP'i 6G C!~n~ 0 9L~~~' C : is ~. Gi: ~ L ... i~'.. ~7..,. ~~.:~' .. .. ..~ .
scores on the verbal tests most probably seriously under-
estimate his true abi]ity. The psychologist sh~uTd -have
pointed this out and should have advised the reader of the
report to assign greater importance to other sources. of
information about the person's abilities than to the test.
scores. If the psychologists are going to do nathing mare
than write a somewhat stereotyped description of test
performance, and this appears to be what they are doing in
the sample of reports that we read, it would be better to
R~~1- s` enerate the test results b com uters which can do the same
~jls ~k '`~'~~~. g y p
Ske ,~~, ~,~ fob -much mare efficiently and economically.
Approved For Relr~ ~6~/Q1-/~~~:~G~A-RDR0001~,4.58 0001Q.Q'~~f4411-6
~ vLa 1
 Approved For Rele~,~f~2f~~..C;YA=R~PUti~?~,~58~:E~01~?~'3~16
~~.w~aP~r~~~E'.:~6ti.`'kGG,.:.r~ut0w :&~6ii~~iy. 4Sct`~. tvd~i{
References
C~mp.t~ell, David P. .Manual for the Strong-Campbell Interest
1n.v.entor~r ' x'325 (P~erged Form}. Stanford, California.
Stanfard University Press 1974.
Equal Ernployment Oppoutunity Commission Uniform Guidelines
on employee selection procedures (1978). Federal Register.
August 25, 1978, (166), 38290-38315.
Standards for educational and psychological tests.
Washington, D.C.: American Psychological Association, 1974.
2Q
Approved For~~~`~Q~~d~~L,~~`~.~~~P~~1~01~~0~130011-6
~~~..e~jb ~~"iBH'c~'~