FUZZY SET APPLICATIONS IN REMOTE VIEWING ANALYSIS

• Document Type: CREST
• Collection: STARGATE
• Document Number (FOIA) /ESDN (CREST): CIA-RDP96-00789R002200140001-2
• Release Decision: RIFPUB
• Original Classification: K
• Document Page Count: 52
• Document Creation Date: November 4, 2016
• Document Release Date: October 14, 1998
• Sequence Number: 1
• Content Type: REPORT

Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Final Report- -Objective A, Task 3
FUZZY SET APPLICATIONS IN
REMOTE VIEWING ANALYSIS
By: BEVERLY S. HUMPHREY THANE J. FRIVOLD
EDWIN C. MAY WANDA W. LUKE
JESSICA M. UTTS VIRGINIA V. TRASK
Prepared for:
PETER J. McNELIS, DSW
CONTRACTING OFFICER'S TECHNICAL REPRESENTATIVE
333 Ravenswood Avenue
Menlo Park, California 94025 U.S.A.
[phone]
Cable: SRI INTL MPK
ved For 1 Sea%1@-209 8/08 : CIA-RDP96-00789R002200140001-2
 0 prov d For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Final Report- -Objective A, Task 3 December 1987
Covering the Period 1 October 1986 to 30 September 1987
FUZZY SET APPLICATIONS IN
REMOTE VIEWING ANALYSIS
By: BEVERLY S. HUMPHREY THANE J. FRIVOLD
EDWIN C. MAY WANDA W. LUKE
JESSICA M. UTTS VIRGINIA V. TRASK
PETER J. McNELIS, DSW
CONTRACTING OFFICER'S TECHNICAL REPRESENTATIVE
0
MURRAY J. BARON, Director
Geoscience and Engineering Center
333 Ravenswood Avenue ? Menlo Park, California 94025 ? U.S.A.
ved For RtgIidaV?-2D?0/( d08 :S6DIXRDPfi=0UM9ROD2O9440001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
ABSTRACT
In FY 1987, fuzzy set mathematical techniques were applied to the problem of remote
viewing (RV) analysis. Two analytical methods were developed: the first was designed to be
sensitive to the verbal content of the RV response; the second was designed to account for the
visual/spatial arrangements of response elements. A definition of "ground truth," against which
these new analytical techniques could be tested, was also devised.
The verbal method is predicated on the application of fuzzy set mathematics to the figure
of merit (FM) technology. * It also features a new descriptor list, which was introduced to
provide a richer vocabulary for analysis. The list's hierarchical structuring in levels, ranging from
very abstract to very concrete, affords considerable flexibility for analytical manipulation of
descriptor elements. A pilot application of the verbal analysis was shown to correlate highly with
ground truth.
The combination of fuzzy set technology and the new descriptor list also proved effective
for the visual/spatial approach. The implementation of these techniques, in conjunction with a
third technique known as "cluster analysis," has resulted in an algorithm for the production of
orthogonal target sets. This has resulted in a significantly more effective rank-order analysis
procedure.
The visual and verbal analyses were each determined to have certain strengths and
weaknesses. The verbal analysis can be statistically more powerful with good RV responses and
provides a more comprehensive breakdown of the verbal information in an RV response. It is
quite labor intensive to apply, however, and it appears to be relatively insensitive to noisy RV
data. "Noisy," in this context, defined as a preponderance of incorrectly identified response
elements. The visual analysis system is inherently much less powerful statistically and is less
capable of providing systematic objectification of the true RV signal content. It can be rapidly
applied however, and is sensitive to the primary manifestation of true RV signal in noisy
data--namely, to the visual arrangement of RV response elements, regardless of their verbal
labels. Potential applications of these techniques in their current states have been suggested;
areas of future research for their refinement have been identified.
The FM analysis has continued to undergo refinement since its inception in FY 1984.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
TABLE OF CONTENTS
ABSTRACT .............................................................. ii
LIST OF TABLES ......................................................... iv
LIST OF FIGURES ........................................................ iv
I INTRODUCTION ............................................... 1
II METHOD OF APPROACH ....................................... 4
A. Target Pool Preparation ..................................... 4
B. Fuzzy Set Theory .......................................... 5
C. Descriptor List Design ....................................... 6
D. Verbal Analysis: the Fuzzy Set Figure of Merit .................. 13
E. Visual Analysis: An Application of Cluster Technology ............ 15
F. Development of Ground Truth ................................ 19
III RESULTS ....................................................... 22
IV RECOMMENDATIONS AND CONCLUSIONS ....................... 24
A. Possible Research Areas for Refinement of the Current Systems .... 24
B. Possible Applications of the Current System ..................... 25
REFERENCES ............................................................ 26
APPENDIX A - INSTRUCTIONS FOR ANALYSTS ............................. A-1
APPENDIX B - TARGET POOL CLUSTER ANALYSIS ......................... B-1
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
1. Names of the 19 Clusters ............................................... 17
LIST OF FIGURES
1. The Fuzzy Set "Very Young? ............................................. 6
2. Descriptor List ......................................................... 8
3. Detailed Cluster Analysis of the Island Cluster ............................... 17
4. Sample Ground Truth Form ..............................................20
5. Normalized Mean for Each Target/Response Pair ............................. 21
6. Ground Truth Compared to Verbal FM ..................................... 22
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
I INTRODUCTION
Since the publication of the initial remote viewing (RV) effort at SRI International,*' two
basic questions have remained in evaluating remote viewing data:
? What is the definition of the target?
? What is the definition of the RV response?
In the development of meaningful evaluation procedures, we must address these two
questions.
In the older, IEEE-style, outbound experiment, definitions of target and response were
particularly difficult to achieve. The protocol for such an experiment dictated that an
experimenter travel to some randomly chosen location at a prearranged time; a viewer's task was
to describe that location. In trying to assess the quality of the RV descriptions (in a series of
trials, for example), an analyst visited each of the sites and attempted to match responses to
them. While standing at a site, the analyst had to determine not only the bounds of the site, but
also the site details that were to be included in the analysis. To cite a specific example using this
protocol: if the analyst were to stand in the middle of the Golden Gate Bridge, he/she would
have to determine whether the buildings of downtown San Francisco, which are clearly and.
prominently visible, were to be considered part of the Golden Gate Bridge target. The RV
response to the Golden Gate Bridge target could be equally troublesome, because responses of
this sort were typically 15 pages of dream-like free associations. A reasonable description of the
bridge might be contained in the response; it might be obfuscated, however, by a large amount of
unrelated material. How was an analyst to approach this problem of response definition?
The first attempt at quantitatively defining an RV response involved reducing the raw
transcript to a series of declarative statements called concepts.2 Initially, it was determined that a
coherent concept should not be reduced to its component parts. For example, a small red VW
car would be considered a single concept rather than four separate concepts, small, red, VW,
and car. Once a transcript had been "conceptualized," the list of concepts constituted, by
definition, the RV response. The analyst rated the concept lists against the sites. Although this
represented a major advance over previous methods, no attempt was made to define the target
References are listed at the end of this report.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
site. It was also extremely labor intensive and did not readily allow for rapid processing of RV
data.
During an FY 1982 program, a procedure was developed to define both the target and
response material.3 It became evident that before a site can be quantified, the overall remote
viewing goal must be clearly defined. If the goal is simply to demonstrate the existence of the RV
phenomena, then anything that is perceived at the site is important. But if the goal is to gain
specific information about the RV process, then possibly specific items at the site are important
while others remain insignificant. For example, let us assume that an office is a hypothetical
target and that a single computer in that office is of specific interest. Let us also assume that a
viewer gives an accurate description of the shape of the office, provides the serial number of the
typewriter, and gives a complete description of the owner of the office. Although this kind of a
response might provide excellent evidence for remote viewing, the target of interest (the
computer) is completely missed. This response, therefore, does not yield information about the
role of complex target elements at the site.
In FY 1984, work began on a computerized evaluation procedure,4 which underwent
significant expansion and refinement during FY 1986.5 The mathematical formalism underlying
this procedure is known as the "figure of merit" (FM) analysis. This method is predicated on
descriptor list technology, which represented a significant improvement over earlier "conceptual
analysis" techniques, both in terms of "objectifying" the analysis of free response data and in
increasing the speed and efficiency with which evaluation can be accomplished.
It became increasingly evident, however, that this particular application of descriptor lists
was inadequate in providing discriminators that were "fine" enough to describe a complex target
accurately; it was also unable to exploit fully the more subtle or abstract information content of
the RV response. To decrease the granularity of the RV evaluation system, therefore, it was
determined that the technology would have to evolve in the direction of allowing the analyst a
gradation of judgment about target and response features, rather than the hard-edged, (and
rather imprecise) all-or-nothing binary determinations. A preliminary survey of various
disciplines and their evaluation methods (spanning such diverse fields as artificial intelligence,
linguistics, and environmental psychology) revealed a branch of mathematics, known as "fuzzy
set theory," which provides a mathematical framework for modeling situations that are inherently
imprecise.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
This report describes the FY 1987 analysis program, * which focused on the application of
fuzzy set mathematics to descriptor list technology. In addition, an effort was made to further
reduce granularity by redesigning the descriptor list itself, and to introduce an altogether new
adjunct technology for capturing visual/spatial RV information. A separate but parallel research
line investigated the application of expert systems technology to the problem of RV analysis. t
This report constitutes Objective A, Task 3, "Fuzzy Set Applications in Remote Viewing
Analysis."
t The results of the expert systems approach to RV analysis can be found in the FY 1987
Final Technical Report.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
II METHOD OF APPROACH
The method of approach was aimed at accounting for both the verbal and visual aspects of
RV data. In this section we present the techniques that were developed to address these
objectives.
A. Target Pool Preparation
A few preliminary guidelines governing target pool composition were distilled from two
sources: (1) the opinions of RV monitors about the appropriateness of various kinds of RV
targets for novice viewers, and (2) RV analysts' assessments about the difficulties encountered in
using descriptor lists to score National Geographic Magazine target materials.
As a general rule, the current subjective consensus is that targets are inappropriate for
experimental purposes if they exhibit any of the following qualities:
? They are contrary to the viewer's expectations. (For example, if the target
pool is basic gestalt scenes, then there will not be a living room scene.)
? They are imbued with negative emotional impact. *
? They violate the "spirit" of the descriptor list's intended use.
? They exhibit descriptors that are ambiguously depicted.
Every effort was made in FY 1987 to create a target pool of 200 National Geographic
Magazine photographs that did not exhibit any of the deficiencies enumerated above. Particular
attention was paid to ensuring that all descriptors were unequivocally depicted: a given target
would be rejected a priori, for example, if its particular photographic angle exhibited what was,
in fact, a lake (a completely-bounded body of water) as a bay (a partially-bounded body of
water). This approach was taken in order to minimize the potential for disputes concerning the
visual content of any given photograph: in this way, it was anticipated that the feedback
experience for the viewer would be unencumbered, as much as possible, by visual versus
conceptual ambiguities.
* Laboratory anecdotal evidence suggests that targets having negative emotional impact often
result in psi-missing responses.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
B. Fuzzy Set Theory
Fuzzy set theory was chosen as the focal point of the RV analytical techniques, because it
provides a mathematical framework for modeling situations that are inherently imprecise.
Because it is such an important component of both the verbal and visual analyses, a brief tutorial
from an earlier reports will be recapitulated to highlight its major concepts.
In traditional set theory, an element is either a member of a set or it isn't--e.g., the
number 2 is a member of the set of even numbers; the number 3 is not. Fuzzy set theory is a
variant of traditional set theory, in that it introduces the concept of degree of membership: herein
lies the essence of its applicability to the modeling of imprecise systems. For example, if we take
the concept of age (known as a linguistic variable in fuzzy set parlance), we might ascribe to it
certain subcategories (i.e., fuzzy sets) such as very young, young, middle-aged, old, etc. Looking
at very young only as a fuzzy set example, we must define what we mean by this concept vis-a-vis
the linguistic variable age. * .If we examine the chronological ages from 1 to 30, we might
subjectively assert that we consider the ages 1 through 4 to represent rather well a spectrum of
the concept very young, whereas the age of 30 probably does not accurately represent very young
at all. As depicted in Figure 1, fuzzy set theory allows us to assign a numerical value between 0
and 1 that represents our best subjective estimate as to how much each of the ages 1 though 30
embodies the concept very young.
Clearly, a different set of numerical values would be assigned to the ages 1 through 30
for the fuzzy sets young, middle-aged, and old--e.g., the age of 6 might receive a value of 0.5
for very young, but a value of 1.0 for young, depending on context, consensus, and the particular
application of the system. In this way we are able to provide manipulatable numerical values for
imprecise natural language expressions; in addition, we are no longer forced into making
inaccurate binary decisions such as, "Is the age of 7 very young--yes or no?"
* It is important to note that the design of the fuzzy application occurs in accordance
with the subjectivity of the system designer. Fortunately, the fuzzy set technology is
rich enough that it allows for a virtually unrestricted range of expression. Technically
speaking, young is the fuzzy set and very is the modifier, but it is beyond the scope of
this paper to present terminology in depth.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
2. Summary of Definitions
In this example, the set of ages ranging from 1 through 30 is a crisp set and can be
defined as the universe of elements. By crisp, we mean, for example, that the age "30 years and
1 day" would fall outside the universe of elements by definition. Very young, young,
middle-aged, and old are fuzzy sets drawn from this universe of elements. They are called fuzzy
because they contain members from the universe of elements that exhibit varying degrees of
membership, ranging from partial to complete. These terms are important for understanding the
design of the current descriptor list and the mathematical formalism underlying its
implementation.
C. Descriptor List Design
The primary impetus in redesigning the descriptor list was to reduce its granularity by
affording a richer vocabulary of terms for response and target interpretation. The following
sections outline (1) the derivation of the list's universe of elements, and (2) the design
philosophy that resulted in their hierarchical arrangement.
The universe of elements--i.e., the entire list of descriptors shown in Figure
2--represents the union of the set of elements in the National Geographic Magazine target pool
of 200 and the set of elements obtained from the FY 1987 RVs for which this pool supplied the
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
targets. This set of elements was obtained in a blind fashion--i.e., in the absence of knowledge
as to which responses correctly matched which targets. In the case of the RV transcript-derived
elements, an effort was made to preserve the vocabulary used by the viewers. Some of the
descriptors, therefore, are either response-dependent or target-dependent or both, while others
(particularly at the more abstract levels) appear to be more potentially universal across descriptor
lists.
2. Descriptor List Structure
The universe of elements is structured in levels, ranging from the relatively abstract
(information poor) to the relatively complex (information rich). The current system has the
capability of using seven primary and three secondary levels of descriptors: the main intent of this
structure is to serve as a heuristic device for guiding the analyst into making judicious concrete
descriptor assignments based on rather abstract commentary. A secondary goal is to make an
attempt at objectifying the analyst's unconscious decision-making process. The use of levels is
advantageous in that each descriptor level can be weighted separately and used or not, as the
case may be, in a computerized data base. This enables various combinations of levels to be
deployed, in order to identify the optimal mix of concrete versus abstract descriptors. Optimal
here might be defined as the highest figures of merit, or perhaps by some other measure, such as
a consensus rating of the experimental series in question.
The determination as to which descriptors belonged on which level was made after
consideration of two primary factors: (1) the apparent ability of the viewers to be able to resolve
certain features, coupled with (2) the amount of pure information thought to be contained in any
given descriptor. In the first case, for example, it appears that viewers have greater difficulty on
average in perceiving waterfalls as opposed to rivers. Some of these "factor one" determinations
were based on the combined observational subjective lore, obtained by both analysts and
monitors in the course of either analyzing or conducting numerous RV experiments; some were
determined empirically from post hoc analyses of viewers' abilities to perceive various descriptor
elements in previous experiments.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
N~ a
w
(~
~ ?~ o
U
Y
6
ti
v~ v
lr ..
O Sr
y
-
y U q

In
~
O
ti
G
i
.a
~+ cd p c~ O
C
?F. ~V
r U w w
d
N
..
c
H
O
N P1 V' fD I~. W m
O
C\j
h-y
0
d
ACd
~
d w
y
4
O~
00
Approved For Release 2000/08/08 : C~4-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
U
C chi
i y bq
O
'~ ,^3 ~ U
w y
q 'p
'~~' N G N U N
v G 6 i. Ri
U z
,
' 7 c~i h as c'[i
'~. b . rte, ~n
z
W F
w
N N N N N
N f7 c7 (h ch
CD
Q)
k s.
w H
d o
a
cy
v
p"')
N
a NV
fD
M~1
COO
z
ao
off
xN
b
w
w
F
Q
T
w
z~y
b A,
m ?
~a^'o vro?r
u
0
cd
0 O y y'0 0 A O
v
[w,w
ay ~a
F
da
cj *c;
} cts
d
3 0o u
c
v
v aki G Q.
i. C)
~z
a
.,
NN
CO C) r00)
LI)) to
T.
U) LO
N N
W C
A
~ ~~
aai N a q v
q y o .- -
o v
> b
o v Uyb
y
n
w
~A
c7
v v a v
U)) Ia L() UUi
a~
Q
~
U
fA
'
~"' d
f7
Vim' d' dam'
h
1Q
to
Approved For Release 2000/08/08 : C14-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
b b
W U N N N^
v q ~ b ~
vb
.~~ _ o Op ?A 3
~v
(D d)
cri
w
z
\-?~ q
z
p .C w N N w O O d
q o.. y
00 o > '4
0 b
W
w
y
~
~
b h
~
s. o is
a 10
o 0
w
W F Ra Cj
CIO "a
v
W
o ~ o
~w-~ 'ti o
Uf" R'Q
vo ?~v
F-,
z
v
.
o
00
N a
.-, v
a
o
y b
a
E-4
o 0
a
j
C
F7 y
-
~
10 C, 00
00
a~
Cd
y
b N
0-4
00 00 00
id
b
r,
a
?yn0
UV
??o ?
/y
93
G7 i"'\ 03 "'
~T4
an dd w
H
??
z
w
a o
w
~
?n
r.i
W a
r
.
b0 Vi w
?0 0 0 0
.14
o N
m e}
00 00 00 00 00
^
Cd
.
1Nryy
z
O
'
U a `~ b0 i-.
z W "~ bq
O
v~
`~ N ai
0f
O
79
0 ,. bT
~
4
w
b a
i
dA " ~'
:6 -5
a
w
0
F14
O~
10 00
m
O
a,
h h [
om h h h h h h
b0
>. p w A b9 P.v
O v A ~r
H
a
o
O
.
00
U
a
b b b b b b b b h
a
w
~
M
Approved For Release 2000/08/08 : Qb4-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
0
ao
C/]
x9
u fr,
0
OD
a
p a ^
OS+U
IC
00
ci
000
00 01
ate) - O R -
U
0
Approved For Release 2000/08/08 :1CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The "factor two" determinations were made primarily by arranging the descriptors
such that a descriptor at any given level represents the sum of constituent descriptors at lower
levels. The world is not a very crisp place and not all of its elements are amenable to hierarchical
structuring: certain violations of the "factor two" rule appear, therefore, throughout the
proposed levels. It should be noted, however, that some of the more glaring violations were
largely driven by the "factor one" determinations (i.e., the viewers' abilities to discern certain
elements) enumerated above.
3. Target and Response Definitions
The descriptor list shown in Figure 2 was filled out for each of the 200 targets,
according to a consensus (see Section III) reached by three analysts on each of the descriptors.
This approach was used to mitigate the potential influence of any single coder's biases and
idiosyncrasies. A numerical assignment (?, 0 < ? ~ 1) was made for each descriptor in
response to the following question:
How visually important is this descriptor to this target?
Each target's coded list served as its formal definition for the purposes of both verbal
and visual analysis. It should be noted that the descriptor set was devised to define targets in
terms of visual importance. If the analysis were to be re-framed along a different dimension of
interest--e.g., conceptual, functional, allegorical, etc.--the descriptor set in Figure 2 would have
to be revised to address the new mission.*
For responses in which verbal analysis was used, g assignments for each descriptor
were made in response to a different question:
To what degree am I convinced that this descriptor represents this response
element?
Responses were coded according to this different criterion (but still using the list in
Figure 2), because RV response composites are often not detailed enough to determine relative
visual importance for any given element. For the purposes of verbal analysis,t the coded
descriptor lists represented the formal definition of each response.
* Details of these analyses can be found in Sections D. and E., below.
t An altogether different response definition was used for the purposes of visual analysis.
This is discussed in Section E.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The assigned ?'s, themselves, were one-digit fuzzy numbers between 0 and 1 (e.g.,
.1, .2, .3, etc.). In some rare cases, two-digit assignments (e.g., .15, .25, .35, etc.) were made;
any finer assignments, however, were deemed to be meaningless.
D. Verbal Analysis: the Fuzzy Set Figure of Merit
The current RV verbal analysis is predicated on the application of fuzzy set mathematics to
the figure of merit technology.
1. Fuzzy Set Definition of Figure of Merit
Once the fuzzy sets that define the target and the response have been specified, the
comparison between them to provide an FM is rather straightforward. In previous work,5 we
have defined Accuracy as the percent of the target material that was described correctly by a
response. Likewise, we have defined Reliability (of the viewer) as the percent of the response
that was correct. The FM is the product of the two; to obtain a high FM, a viewer has to
describe a large portion of the target material correctly in as parsimonious a way as possible.
These quantities for the jth target/response pair are as follows:
Wk (Ri n Ti)k
n
k
cur
U Wk Tj,k
k
G+ Wk (Ri n Ti)k
Reliability, = ri = k
2 Wk Ri, k
k
Figure of Merit i = Mi = ai x ri ,
where the sum over k is called the sigma count in fuzzy set terminology, and is defined as the
sum of the membership values, ?, for the elements of the response, target, or their
intersection--intersecting Ri, Ti, and (Rif1Ti), respectively. A fuzzy intersection is defined as the
minimum of the two membership values. In this version of the figure of merit definition, we have
allowed for the possibility of weighting the membership values, Wk, in order to examine various
linguistic contributions to the FM.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
For the above calculation to be meaningful, the ?'s for the targets must be similar in
kind to the ?'s for the responses. As we noted above, this is not the case. The target ?'s
represent the visual importance of the element relative to the scene, and the response ?'s
represent the degree to which the analyst is convinced that the element is represented in the
response.
With advanced viewers it might be possible to change the definition of the response
?'s to match the definition of the target ?'s. In this case, the viewer must not only recognize that
an element is present in the target, but must also provide information as to how visually important
it is. This ability is beyond the skill of most novice viewers. Alternatively, we have opted to
modify the target ? definition, by employing the fuzzy set technique of a-cuts. An a-cut is a
way to set a threshold for visual importance, so that all target elements possessing that threshold
value or higher are considered to be full members of the target set. In fuzzy set parlance, an
a-cut converts a fuzzy set to a crisp set. The result is that the target set is now devoid of visual
information: a potential target element is either present or absent in the target set, regardless of
its actual visual importance. Even with this conceptual change in the target definition, the FM
formalism described above remains applicable, because a crisp set can be considered as a fuzzy
set with all membership values equal to 1. It should be noted that the a-cut conversion does not
occur in the data base proper, but rather in the analysis itself. In this way the integrity of the
original visual importance information can be maintained, and the a-cut can be exercised as a
variable.
2. P-Values for a Given Figure of Merit
It is difficult to arrive at a general assessment as to how well a given response matches
a specified target. The ideal situation is to obtain some absolute measure of goodness of match.
While the FM is an approximation to this measure, it provides no information as to the likelihood
of a particular FM value. The answer to that question is nearly impossible, because it requires
knowledge of the viewer's specific response bias for the session. It is possible to determine
general response biases,4 but that knowledge is only useful on the average. For example, a
viewer may love rock climbing and may spend most of her/his free time involved in that activity.
Thus, the general response bias would probably entail aspects of mountains, rocks, ropes, etc.
Suppose, however, that the viewer spent the evening previous to a given RV session on a
romantic moonlight sail on San Francisco Bay. For this specific RV session, the response bias
might be apt to include romantic images of the moonlit water, lights of the city, and bridges.
The current solution to the problem is to provide a relative assessment of FM
likelihood (i.e., a differential measure). A relative assessment addresses the following question:
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
"How good is the response matched against its intended target, when compared to all possible
targets that could have been chosen for the session?" Unfortunately, the answer depends upon
the nature of the remaining targets in the pool. An example of the worst-case scenario illustrates
the problem. Suppose that the target pool consisted of 100 photographs of waterfalls, and the
viewer gave a near-perfect description of a waterfall. Suppose further that this viewer had been
well calibrated so that we can assume his/her description was not fortuitous. An absolute
assessment of the resulting FM should be good, whereas a relative assessment will be low.
The worst-case scenario can be avoided, to a large degree, by carefully selecting the
target pool. In section E.1 we describe an application of fuzzy set theory that addresses target
similarities vis-a-vis the problem of target pool optimization.
To provide a relative assessment of the likelihood of a given FM, we define the
p-value for the session as the fraction of targets out of 200 that have an FM equal to or higher
than the given FM.
Consecutive RV responses by the same viewer are not altogether statistically
independent. The statistically independent random element in the session is the target.
Therefore using the p-value defined above maintains the trial-by-trial independence. The mean
chance expectation for the p-values is a unit uniform distribution with a mean of 0.5.
We are currently exploring various methods for combining the session p-values to
provide an overall assessment of a series of viewings.
E. Visual Analysis: An Application of Cluster Technology
The verbal analysis appears to afford a reasonably comprehensive estimate of RV response
information along a certain dimension: namely, identifying what the response elements are,
ranging from abstract to concrete descriptions. This analysis is quite insensitive, however, in
accounting for the spatial arrangement of the response elements. It has long been the subjective
opinion of analysts that it is the visual arrangement of response elements--irrespective of what
these elements are actually called by the novice viewer--that comprise the major part of the true
RV signal line. This hypothesis was the major impetus for the development of an analysis system
that would account for this visual/spatial type of RV information.
1. Target Similarities
Target definition for the purposes of this mode of analysis is exactly the same as the
one that was used for the verbal analysis--i.e., a given target is defined by its descriptor list,
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
which has been coded to reflect the visual importance of each target element. Since the average
number of elements that have been assigned a non-zero value is approximately 37, the fuzzy set
representation of the target pool is rich in visual information. We used this information to
determine the degree to which the target set contains visually similar targets.
It is beyond the scope of this report to describe the extensive work in the literature
seeking to find algorithmic techniques that mimic human assessments of visual similarity. One
recent article describes techniques similar to the one we used.6
We begin by defining the similarity between target j and target k (SJ,k) to be a
normalized fuzzy set intersection between the two target sets:
SJ.k =
(E Wi (TJnTk)i) 2
i
F Wi TJ,i F Wi Tk,i
i i
For N targets there are N(N-1)/2 unique values (19,900 for N=200) of Sj,k. The value j and k
that correspond to the largest value of SJ,k represent the two targets that "look" most similar.
Suppose another target (m) is chosen and Sm,J and Sm,k are computed. If both of these values
are larger than Sm,n (for all n not equal to j or k) then target m is assessed to be most similar to
the pair j,k. The process of grouping targets based on these similarities is called cluster analysis.
Table 1 provides an overview of the 19 clusters found from the total analysis of the
200 targets. Figure 3 shows the graphic output of a single cluster in detail: a much more
complex--and visually difficult--graph is generated for the full cluster analysis (see Appendix
B) ; this smaller subset, therefore, has been chosen to be illustrative of the whole analysis. * All
targets in this particular sample cluster are islands; the island in each photograph is visible in its
entirety. Except for one outlier (i.e., a hexagonal building covering an island), the islands fall
into two main groups, i.e., with and without manmade elements. The natural islands include
three similar mountain islands, two sandbars, and two flat verdant islands.
* In order to make the graphic output more meaningful, we actually did the analysis
with 1 - Sj,k .
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
NAMES OF THE 19 CLUSTERS
No.
Name
No.
Name
1
Flat Towns
11
Cities with Prominent Geometries
2
Waterfalls
12
Snowy Mountains
3
Mountain Towns
13
Valleys with Rivers
4
Cities with Prominent Structure
14
Meandering Rivers
5
Cities on Water
15
Alpine Scenes
6
Desert/Water Interfaces
16
Outposts in Snowy Mountains
7
Deserts
17
Islands
8
Dry Ruins
18
Verdant Ruins
9
Towns on Water
19
Agricultural Scenes
10
Outposts on Water
Linear Geometries
(e.g., Runways)
Many Structures
(e.g., Town)
Ruins
Flat And Verdant
Sand Bars
1198
11 IR
1128
1186
1179
1177
1185
1139
1140
1088
1161
1049
1038
Hexagonal Building Covering Island---- > 1003
0.0 0.2 0.4 0.6
1 - sJ,k
Target lxxx
Number
FIGURE 3 DETAILED CLUSTER ANALYSIS OF THE ISLAND CLUSTER
17
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
2. RV Analysis Using Orthogonal Targets
The visual analysis of RV data has been the primary method used in simple
rank-order judging. In the past, the technique involved a ranking of the actual targets in a series
against a given response. There are two possible problems with this approach:
(1) The targets are not optimized for judging discrimination.
(2) Long series are difficult to assess.
Using the results of the cluster analysis, it is possible to select a number of decoy
targets that are all different from each other and from the intended target, as well. The
technique we have chosen involves a random selection of 6 out of 18 possible decoy clusters; the
intended target cluster is not used. One target is randomly selected from each of these six decoy
clusters. The intended target is added to the group of six decoys, and they are all numerically
ordered on target number; these are then presented with the single response to the analyst for
assessment.
The analyst's task is to rank order the targets from the best to the worst match based
on a subjective assessment of the visual correspondence. Analysts are essentially instructed to
pay primary attention to the drawings in a response (particularly, in terms of perspective) and
secondary attention to the attendant verbal commentary, i.e., the written material is not to be
ignored completely, but its contribution is small. The rationale for this is evident in the overall
intent of the visual/spatial analysis. Analysts are also instructed to regard incongruent or
surprising response elements as more salient, and to be slightly biased in favor of elements
occurring at the beginning of a response. The reasons for these instructions are less evident and
are largely attributable to the observations and subjective impressions of experienced and
successful analysts. It was also determined that a consensus approach is useful for objectifying
-the decision-making process: for the FY 1987 data, all rankings were determined by a
consensus of two analysts. For a series of many sessions, the sum of ranks for the actual matches
is computed and a standard sum-of-ranks distribution is used to compute the p-value for the
series.
The primary advantages to this approach are the following: (1) an analyst never has
to provide an assessment for more than seven items at a time, and (2) the analysis can be rapidly
applied for very long experimental series. The most important advantage, however, is that the six
decoys and the intended target look as differently as possible, given the nature of the target pool
at large and the fact that decoys must be selected randomly. The application of fuzzy set
technology has provided an algorithm for the production of orthogonal target sets: this is
perceived as the primary contribution to the success of this analysis technique.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
F. Development of Ground Truth
To determine whether any of the new analytical approaches were effective, a standard had
to be developed against which they could be measured. It was determined that this
standard--known as "ground truth"--should consist of a "real world" normalized consensus as
to the degree of correspondence between RV responses and their intended targets.
To achieve this objective, each person from a sample population of 37 was presented, on
an individual basis, with six remote viewing responses and their associated targets. This set
comprised an experimental series for Viewer 177 in an FY 1986 photomultiplier tube (PMT)
experiment.? Each response was matched to its intended target; this created six such sets,
numbered sequentially from 9001 through 9006. The responses were fairly typical of novice
viewer output: they consisted of two to five pages of rudimentary drawings with some associated
descriptive words. The targets consisted of six photographs of outdoor scenes selected from an
FY 1986 National Geographic Magazine target pool.
Each person was asked individually for his/her subjective judgment as to the degree of
correspondence between the remote viewing responses and their respective targets. This was to
be accomplished without the benefit of a given definition of "degree of correspondence;"
respondents were free to formulate their own criteria. The only information provided was that
responses typically begin (on the first page or pages) with small bits of information and eventually
culminate in a composite drawing at the end.
After this brief introduction to the task, each individual was instructed to examine all of
the responses and their intended targets. Then, on a session-by-session basis, the subject was
asked: (1) to assess the degree of correspondence between the remote viewing response and its
intended target, and (2) to register this correspondence assessment by making a vertical hash
mark across a 10 cm scale ranging from "none" to "complete." Figure 4 provides a sample of
the form that the respondents used to indicate their six assessments.
After completing the assessment task, respondents were then asked to furnish some minor
autobiographical information (i.e., age, occupation, male/female, handedness, belief in ESP).
Degree of belief in ESP was also registered on a 10 cm scale: each individual simply made a
vertical hash mark at the appropriate point on a continuous line between "no belief" and
"complete belief."
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
ANALYSTS' INSTRUCTIONS FOR REMOTE VIEWING SERIES 900X
Thank you for helping us perform a post hoc assessment of a series of remote viewings.
The targets were actually 35 mm slides that were attached to a photomultiplier, a device to
measure small amounts of light. We were searching for possible physical correlates to remote
viewing.
You will find in your packet six remote viewing responses labeled 9001-9006, respectively.
Also shown is the target number of the intended photograph. We have supplied the original,
rather than the 35 mm slide.
We would like you to make a subjective judgment as to the degree of correspondence
between the remote viewing response and its associated target. Familiarize yourself with the task
by first looking at all the responses and their intended targets. Then, on a session-by-session
basis, rate your assessments. You are completely free to define what is mean by "Degree of
Correspondence." Indicate your judgment by marking one line across the appropriate
continuous scale shown below. A vertical line near the "None" end of the scale will indicate that
you feel there is very little correspondence between that response-target pair. Likewise a vertical
line near the "Complete" end of the scale will indicate that you feel that there is a significant
degree of correspondence.
Many of the responses begin with a little information and build toward a composite drawing
at the end. Please assess the response in its entirety as best you can. Thank you again.
SESSION
DEGREE OF CORRESPONDENCE
No
ne
Complet
e
9001
1034
9002
1042
9003
1065
9004
1094
9005
1005
9006
1024
Age:
Occupation:
Male/Female:
Handedness (R/L)
None
Complete
20
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
To perform the ground truth analysis, distance measurements were taken from the left end
point of each scale to the vertical slash mark for each assessment. These measurements were
then entered into a computerized data base. The first step in analyzing this aggregate of data was
to compute the mean and standard deviation for each individual's set of assessments. In order to
nullify response bias, each analyst's set was normalized by a z-score transformation. For a given
target/response pair, all analysts' data could then be combined. The mean and standard
deviation were then computed over all analysts' assessments for each target/response pair.
Figure 5 shows these normalized means for each target/response pair: these constitute the basis
for the ground truth against which other techniques were measured. See Section III, Results.
The solid bars illustrate the degree of positive correspondence between the remote viewing
responses and their intended targets, while the shaded bars represent the degree of negative
correspondence (i.e., worse than average for this set of six). Note that these are relative to the
others in the set. All six correspondences in the set could be good.
FIGURE 5 NORMALIZED MEAN FOR EACH TARGET/RESPONSE PAIR
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
III RESULTS
To effect a meaningful comparison between ground truth and the verbal analysis, the same
PMT series that served as the ground truth set was also analyzed by the fuzzy set figure of merit
method. The verbal fuzzy sets (?'s) for the six targets and six responses were consensus-coded
by five analysts ranging from expert to novice. A typical spread of ? assignments was ?0.1 with
an occasional outlier. Some of the elements were vigorously debated until a consensus was
reached. Figures of merit were calculated for each target/response pair, and all possible unique
differences (n(n-1)/2 = 15; n = 6) were computed. These differences were then correlated
against similar differences computed from ground truth (described above).
FIGURE 6 GROUND TRUTH COMPARED TO VERBAL FM
Figure 6 shows the results of this calculation where RV quality difference for verbal is
shown as open circles and ground truth quality difference as triangles. The visual qualitative
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
agreement is supported by the linear correlation coefficient between the two sets of points of
0.720.
This implies that the combination of (1) the structure of the universe of elements (i.e., the
verbal hierarchical structure), (2) the fuzzy set mathematics, and (3) a consensus approach to
assessing the fuzzy sets themselves, provided an accurate representation of the subjective scoring
of the same data by a large number of randomly selected individuals. In short, the FY 1987
verbal analysis system provided output that closely correlated with ground truth.
To summarize, the visual and verbal analyses each have their own respective strengths and
weaknesses. The verbal analysis is statistically more powerful and provides a more
comprehensive breakdown of the verbal information in an RV response. It is quite labor intensive
to apply, however, and it appears to be relatively insensitive to noisy RV data. * "Noisy," in this
context, is defined as a preponderance of incorrectly identified response elements. The visual
analysis system is statistically much less powerful and is less capable of providing systematic
objectification of the true RV signal content. It can be rapidly applied, however, and is sensitive
to the primary manifestation of true RV signal in noisy data--namely, to the visual arrangement
of RV response elements, regardless of their verbal labels. It was the consensus of experimental
monitors that the FY 1987 data were relatively noisy. Therefore, although the visual analysis
system has yet to be formally compared to ground truth, it was chosen as the primary analytical
method for the FY 1987 RV experiments.
* The PMT experimental series chosen for the pilot verbal analysis application turned
out to be considerably more robust than much of the data obtained from the FY 1987
experiments.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
IV RECOMMENDATIONS AND CONCLUSIONS
A. Possible Research Areas for Refinement of the Current Systems
Several future research areas are suggested for the refinement of the current analytical
systems. One area that needs to be examined systematically is the use of both inter- and
intra-level weighting factors. For the purposes of the FY 1987 analyses, all levels and descriptors
were accorded equal weight. The ideal goal would be to determine the optimal weighted mix of
abstract versus concrete elements, as a means to achieving the following objectives:
(1) refinement of the cluster analysis for targets, in an effort to simulate, as closely
as possible, what is meant by "visual similarities" between targets, and
(2) refinement of the verbal analysis of responses, in an effort to achieve even
greater correlations between the fuzzy set figure of merit analyses and various
forms of ground truth.
Another area that requires examination in some detail is the set of descriptors (i.e., the
universe of elements), and the hierarchical nature of its structure. It is probable that some
descriptors are more appropriate than others; furthermore, they might be more effectively
structured in a semantic network as opposed to a true hierarchy. If a hierarchical structure is
retained, then some attention must be paid to the formulation of logical consistency rules that
govern descriptor use. This would include numerical relationships governing the values (?'s) of
higher-order descriptors (e.g., port) vis-a-vis the combined value of their constituent parts
(e.g., city, river, boats, jetties, commercial, etc.).
One perceived inadequacy of the verbal system is that it atomizes conceptual "units." For
example, if the response element is red box, information is lost in separating red from box.
Current research in fuzzy set theory indicates that fuzzy aggregates of fuzzy elements--"fuzzy
sets of fuzzy sets"--are mathematically complex but possible. Some effort should be made to
determine whether this technology could be implemented, as a means to capturing the
information content of the RV response with greater accuracy.
For the visual analysis, research into visual similarities between pictures of natural scenes
may serve as a potential refinement tool. The aim here would be to enhance the visual
orthogonality of the decoy targets as much as possible. Experiments in normal perception of
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
similarities would assist in determining whether scenes are perceived as similar because of their
low-level geometries, concrete elements, or some combination of factors. Should the lower,
abstract levels prove important to similarities perceptions, then Fourier decomposition of target
materials might assist in providing more accurate target ?'s at these levels. In any case, the
ultimate aim would be to refine the target cluster analysis such that it closely simulates ground
truth perceptions of orthogonality.
B. Possible Applications of the Current Systems
The verbal and visual analytical methods exhibit different strengths and weaknesses and
appear to account for different aspects of the RV signal line. Even without further refinement or
modification, the systems appear to be quite sensitive and accurate in their respective domains:
the verbal system, for example, correlated highly with collected ground truth.
This suggests that they might be used effectively in their current state for certain
applications. One possibility is that the two methods might be used in combination to afford
operative definitions of "advanced" versus "novice" RV functioning. One might predict, for
instance, that verbal scores for novices would be relatively low, while their visual scores would be
somewhat higher. An advanced viewer, on the other hand, might be one who consistently
produces scores in both areas above certain predetermined thresholds. It is also possible that this
approach might eventually provide a formal definition for RV "learning." As the two analytical
systems are deployed and refined, other applications may be identified for systematic
investigation in the future.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
REFERENCES
1. Puthoff, H. E., and Targ, R., "A Perceptual Channel for Information Transfer Over
Kilometer Distances: Historical Perspective and Recent Research," Proceedings of the
IEEE, Vol. 64, No. 3 (March 1976).
2. Targ, R., Puthoff, H. E., and May, E. C., 1977 Proceedings of the International
Conference of Cybernetics and Society, pp. 519-529 (1977).
3. May, E. C., "A Remote Viewing Evaluation Protocol," Final Report (revised), SRI Project
4028, SRI International, Menlo Park, California (July 1983).
4. May, E. C., Humphrey, B. S., and Mathews, C., "A Figure of Merit Analysis for
Free-Response Material," Proceedings of the 28th Annual Convention of the
Parapsychological Association, pp. 343-354, Tufts University, Medford, Massachusetts
(August 1985).
5. Humphrey, B. S., May, E. C., Trask, V. V., and Thomson, M. J., "Remote Viewing
Evaluation Techniques," Final Report, SRI Project 1291, SRI International, Menlo Park,
California (December 1986).
6. Zick, R., Carlstein, E., and Budescu, D. V., "Measures of Similarity Among Fuzzy
Concepts: A Comparative Analysis," International Journal of Approximate Reasoning,
Vol. 1, No. 2, pp. 221-242 (April 1987).
7. Hubbard, G. S., May, E. C., and Frivold, T. J., "Possible Photon Production During a
Remote Viewing Task: A Replication Experiment," Final Report, SRI Project 1291, SRI
International, Menlo Park, California (December 1987).
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
INSTRUCTIONS FOR ANALYSTS
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
ANALYSTS' GUIDE TO THE FY 1987 NATIONAL GEOGRAPHIC
This document is intended to assist analysts in the assessment of novice level remote
viewings (RVs) of National Geographic Magazine targets. As we are all aware, novice transcript
data can be quite sparse or abstract or both: these characteristics have often resulted in
considerable loss of information within the framework of the previous twenty-bit descriptor list.
In FY 1987, we have attempted to reduce the granularity of the analysis system by incorporating
the use of fuzzy set mathematics and by introducing the ability to interpret responses and targets
in more abstract terms.
The degree of abstraction and atomization that transcripts and targets can usefully tolerate
are open questions at present. One method of exploring these parameters involves the use of
levels of descriptors, ranging from the relatively abstract (information poor) to the relatively
complex (information rich). The proposed new evaluation system has the capability of using
seven primary* and three secondary levels of descriptors, through which we hope to parse and to
objectify the chain of logic that analysts employ for making decisions about concrete descriptors
based on rather abstract commentary. The intent here, of course, is to make an attempt at
bringing the analyst's unconscious decision-making process out of the murky realm of
superstition and magic and into the hard, cold, scientific light of day! The use of levels is
advantageous in that each descriptor level can be weighted separately and used or not, as the
case may be, in a computerized data base. This will enable various combinations of levels to be
deployed, in order to identify the optimal mix of concrete versus abstract descriptors. Optimal
here might be defined as the highest figures of merit or perhaps by some other measure, such as a
consensus rating of the experimental series in question.
It is anticipated that most of the potential problems will occur on the abstract descriptor
end of the spectrum: how abstract can a descriptor be and still be useful? To illustrate the
potential problems posed by increasingly abstract descriptors, consider the fact that all responses
Consider the elegant symmetry here between the descent into hell, Dante-style, and the
appropriateness of having seven primary levels of analysis.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
and targets contain some admixture of the two highly abstract elements horizontal lines and
vertical lines; to perform fuzzy assessments along these dimensions, therefore, may not enhance
the distinguishability of targets and responses. On the other hand, a concept such as rectangular
forms, while still quite abstract, is only selectively present in transcripts and targets: it may prove
useful, therefore, for capturing data outright that have heretofore been acknowledged only tacitly
via the process of interpretation. It may be, therefore, that the ideal system would encapsulate a
degree of greater abstraction, without adding levels of descriptors that are so abstract as to be
universally represented in all responses and targets. In any case, this is what the new system
proposes to explore systematically. In the next section an overview of the descriptor levels will be
provided.
B. An Overview of Descriptor Level Theory
The entire list of descriptors represents the union of the set of elements in the National
Geographic Magazine target pool of 200, and the set of elements obtained from the FY 1987
RVs for which this pool supplied the targets. This set of elements was obtained in a blind
fashion--i.e., in the absence of knowledge as to which responses correctly matched which
targets. In the case of the RV transcript-derived elements, an effort was made to preserve the
vocabulary used by the viewers. Some of the descriptors, therefore, are either
response-dependent or target-dependent or both, while others (particularly at the more abstract
levels) appear to be more potentially universal across descriptor lists. The determination as to
which descriptors belonged on which level was made after consideration of two primary factors:
(1) the apparent ability of the viewers to be able to resolve certain features, coupled with (2) the
amount of pure information thought to be contained in any given descriptor. In the first case, for
example, it appears that viewers have greater difficulty on average in perceiving waterfalls as
opposed to rivers. Some of these factor one determinations are based on the combined
observational subjective lore, obtained by both analysts and monitors in the course of either
analyzing or conducting numerous RV experiments; some have been determined empirically
from post hoc analyses of viewers' abilities to perceive various descriptor elements in previous
experiments.
The factor two determinations were made primarily by arranging the descriptors such that
a descriptor at any given level represents the sum of constituent descriptors at lower levels.
Obviously, the world is not a very crisp place, and so there appear to be violations of this rule
throughout the proposed levels. Keep in mind, however, that certain of the more glaring
violations are largely driven by the factor one determinations (i.e., the viewers' abilities to
discern certain elements) enumerated above.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
For the purposes of our experiments, the higher the level in number, the higher the
weighting factor (W)--which is to say that more concrete information merits more credit.
Fortunately, the analyst need not concern himself with the Ws, because these consist of
predetermined multipliers built into the mathematical structure of the system. The Ws consist of
a two-dimensional weighting system that reflects mission-dependency: in this case, the overall
mission is to accord greater weight to more specific kinds of information. The secondary mission
is to provide a mechanism for assessing the viewer's (or viewers') ability to resolve a given
descriptor within a given level.
The numerical values that the analyst will have to concern himself with are the values
given to the individual descriptors themselves (g's). For targets, the ?'s will be assessed on a
descriptor-by-descriptor basis according to the presence or absence of each descriptor in terms
of visual importance. For responses, the it's will be assessed according to the analyst's best
estimate as to whether each descriptor is present or absent at all. The concept of visual
importance is not relevant for novice-level responses, because the data are often not detailed
enough to form meaningful composite pictures.*
C. The Descriptor Levels and Their Use
With this basic theory in mind, it is appropriate at this juncture to provide an overview of
what each level typifies and how the descriptors in these levels might be used by the analyst. All
subsequent commentary, therefore, is in reference to the actual descriptor sheets themselves,
which will be used for encoding the FY 1987 targets and responses. As a general comment on
some of the mechanics of the sheets, descriptors in parenthesis, are simply alternate words that
may be encountered in lieu of the main descriptor--e.g., rise (slope); descriptors in brackets,
will probably be encountered only rarely--e.g. [oasis].
Perhaps the simplest way to obtain an overview of the seven descriptor levels is to proceed
from the bottom up--i.e., from the simplest or most abstract, to the most complex or concrete.
Levels 1 and 2 are provided to account for the abstract geometry present primarily in
responses and secondarily in targets. Level 1 contains one-dimensional geometric elements and
is aimed at the most primitive RV ideograms. For example, the following situation might occur:
* More complete information on the mechanics of making the .t assignments is provided in
Section D.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The viewer draws Labels it And The target is
this: this: this:
"Stepped"
and
"City"
It would seem reasonable to assume here, that a mesa might be adequately described as a
stepped feature and that the viewer's initial ideogram represented some amount of "true" signal.
In the previous twenty-descriptor system, there was no mechanism for giving the viewer credit for
the validity of his raw ideogram, despite the fact that he misinterpreted its meaning. In the new
system, the viewer can receive Level 1 credit straightaway for the perception of stepped. The
Level 1 weighting factor (which the analyst need not worry about) will probably be small, because
the Level 1 elements are primitive, ubiquitous, and information poor. As a result, the viewer
won't receive very much credit for a primitive perception like stepped, but this would appear to
be reasonable: logic would dictate that a higher order perception such as mesa should in fact be
worth more credit.
Level 2 consists of combined 2-D and 3-D geometries. The rationale for combining
the two geometries resides in the observation that viewers' drawing skills are primitive: a box, for
example, usually has no more representational information than a rectangle.
The principal problem for the analyst in using levels 1 and 2 will be in attempting to
decompose concrete target elements into abstract geometries, such that some meaningful overlap
between targets and responses can be achieved. Fortunately, fuzzy set theory holds promise for
ameliorating some of this difficulty, in that it will enable the analyst to make small numerical
assignments for "low confidence" descriptors. For example, if the target were a series of
mountain peaks, an analyst might be willing to make small numerical assignments for the highly
stylized descriptors of inverted V-shape, repeat motif, triangle, and perhaps, cone (if the scene
looked particularly volcanic). In the optimal situation for target coding, there would be
descriptor lists constructed for each viewer, such that the list used for any given viewer would be
specifically tailored to the vocabulary and ideogramic idiosyncrasies of that viewer alone. In this
way, targets could be abstractly decomposed in a given viewer's own terms. Unfortunately, we do
not have the luxury of doing this at present, because one list must suffice for a group of viewers.
The governing rules in making assignments at levels 1 and 2, therefore, would appear to be the
following:
(1) Targets should be as liberally but defendably interpreted (albeit with
probably small fuzzy values) across as many geometric descriptors as
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
seem reasonable, in order to be inclusive of all the viewers' potential
drawings and debriefs, while
(2) Response geometries should simply be chronicled in the descriptor list as
they are literally observed. Level 2 descriptors such as rectangle are not
to be parsed into the constituent parts of horizontal lines and vertical
lines. To assign ?'s for "constituent parts" bits, response elements have
to be clearly and discretely identifiable as these things in and of
themselves.
In most cases, there are probably both literal and stylized descriptors for any given
target element that require higher and lower values, respectively. In the mountains example
above, for instance, mountain ranges usually qualify most literally as irregular forms, whereas the
more stylized interpretations involve inverted V-shapes, etc. In any case, it should be noted that
any assigned target ? will have to be defendable on a visual basis.
It is anticipated that when a truly significant geometric form is encountered in a
target, such as in the Transamerica Pyramid, it will be accorded a higher triangle fuzzy value
than would be accorded for mountains, because the Transamerica Pyramid is more
unequivocally a triangular/pyramidal form. In this way, if a viewer registers a triangle ideogram
for the Transamerica target, he will receive substantially more credit than if he uses a triangle
ideogram for a mountain target.
There is little doubt that the process of decomposing the concrete target elements into
clusters of somewhat "imagined" viewer-derived abstract elements is going to be a
noise-producing exercise. We're assuming here, however, that (1) not every concrete feature
can be decomposed into every abstract feature (e.g., the analyst will not typically be able to
justify decomposing mountains into a cross-hatch (grid) geometry, but in all likelihood, he'll feel
comfortable in doing so for city), and (2) there is enough true viewer signal to effect significant
overlap between targets and responses. Even at the abstract levels, one hopes the true signal line
will rise above this very noisy background. The true advantage of this system, however, is that
levels 1 or 2 or both can be turned off in the data base and the analyses rerun, if it appears that
these initial hypotheses are incorrect.
2. Abstract Descriptor Levels I: Levels 3 and 4
Levels 3 and 4 are aimed primarily at viewers' archetypal and visceral/qualitative
impressions, respectively. While levels 3 and 4 are more information-rich than the primitive
ideogramic geometries, they are still not sufficient in and of themselves to produce a coherent,
concrete picture. Generally speaking, if an RV session is proceeding in an "on-line" fashion, a
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
geometric ideogram (levels 1 and 2) will be debriefed by descriptors from the level 3 and 4
Because levels 3 and 4, like levels 1 and 2, are generally response-derived rather
than target-derived, the same rules for application pertain:
(1)
(2) Response elements should simply be chronicled in the descriptor list as
they are literally observed.
Targets should be as liberally but defendably interpreted (albeit with
probably small fuzzy values) across as many descriptors as seem
reasonable, in order to include all the viewers' potential drawings and
debriefs, while
Again, the principle foreseeable problem will occur in trying to encode the target
materials. The analyst will need to show some flexibility in understanding that a lush abundance
of vegetation, for example, might be viscerally perceived by the viewer as the texture fuzzy, or the
hard edge of a modern skyscraper might be experienced initially as smooth and shiny/reflective.
Perhaps the guiding principle here is for the analyst to be imaginative, but not excessive, in
determining which descriptors merit fuzzy values. Any assigned target .t will have to be
defendable on a visual basis. Again, as in the case with levels 1 and 2, some descriptors are
more stylized than others and should probably be assigned lower values accordingly.
It might be noticed that levels 3 and 4 contain some pairings of descriptors that are
opposite in meaning--e.g., rise versus flat, ordered versus disordered, complex versus simple,
etc. This will enable the analyst to make assignments along opposing dimensions when they are
present in targets or responses. For example, a mesa in the middle of an otherwise flat plain
would presumably merit high values for both the rising and flat dimensions. If you recall, we had
considered at one point having a single descriptor, elevation, for which a "0" would have
indicated total flat or horizontality, and a " 1" would have indicated total rise or verticality. In
the mesa target example just cited, elevation would probably merit a 0.5, to indicate 50% flat and
50% vertical rise; a 0.5 value along this single dimension would make this target indistinguishable
from a target containing a single 45 ? slope mountain peak with no background except sky. This
would appear to be conceptually incorrect because of the resultant loss of information across
both dimensions in the case of the mesa. This kind of problem is the primary rationale for the
inclusion of opposites. Clearly, it was not appropriate to supply an opposite for every descriptor,
because most negatives are usually not asserted by viewers--e.g., if there is no perception of
flowing, the viewer isn't apt to say not flowing.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
3. Concrete Descriptor Levels II: Levels 5, 6, and 7
At level 5 and above, the descriptors are primarily target-derived rather than
response-derived. Levels 5, 6, and 7 contain the bulk of the kinds of descriptors that we were
accustomed to using in the previous twenty-bit list. Using the concrete descriptor levels poses a
different set of problems for the analyst, because of the shift in the interpretive burden from
target encoding to response encoding. The primary guidelines, therefore, for making assignments
at all concrete levels are as follows:
(1) Target elements should simply be chronicled in the descriptor list as they
are literally observed, while
(2) Response elements should be liberally but judiciously interpreted, with
the actual numerical assignments made by taking into account, to
whatever extent possible, the given viewer's response biases and
idiosyncrasies.
This second point is a difficult one and requires further clarification. In an optimal
situation, we would have complete and systematic retrospective knowledge--a track record, as it
were--for each viewer, in which we knew from post hoc analyses the probability for any given
abstract response ideogram correctly matching any given concrete descriptor. Unfortunately this
is quite a labor intensive investigation, and we haven't had the opportunity to perform it as yet.
Clearly this is a critical area of research to move into, if we are ever to have any ultimate success
at objectifying the analytical process.
What we do have at present are experienced analysts, who have looked at many
years' worth of data that have been obtained from the viewing population in question. Although
we haven't systematically codified our observations, we have some sense that when certain
viewers say or draw certain abstract things, they are more apt to mean one thing as opposed to
another. In general, the analyst should be fairly liberal and inclusive in his choices of which
concrete descriptors are candidates for fuzzy values; where the awareness of response
idiosyncrasies should come into play is in the assignment of fuzzy values for these choices. For
example, if viewers 105 and 177 used the quality descriptor fuzzy in their responses, it would
probably be reasonable in both cases to select vegetation as a possible interpretive descriptor for
this impression. An experienced analyst, however, might be justified in determining that this
interpretation is more valid for 177 than it is for 105, and would accord a higher numerical value
for vegetation in the case of the former. Of course, if an experienced analyst has very high
confidence that a certain combination of abstract elements simply cannot be interpreted as a
certain candidate concrete descriptor for a given viewer, then that concrete descriptor should be
eliminated from the candidate list altogether. Until the response biases of each viewer are
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
somehow systematized and objectified, it is certainly in the analyst's best interest to have a sound
grasp of the response styles of his viewing population insofar as it is possible.
4. Concrete Descriptor Levels I: Levels 8, 9, and 10
It is a matter for discussion as to whether levels 8, 9, and 10 are appropriate for
inclusion with our core group of novice viewers at all, given that we infrequently see these levels
of response detail. At present, therefore, they have been labeled as secondary rather than
primary levels. Another matter for discussion is whether these sub-elements, as they are
currently configured, belong at higher informational levels than 6 and 7. In some sense, in order
to "build" the perception of a level 7 port, a viewer must have either consciously or
unconsciously registered the impression that there are boats and perhaps piers (currently level 9)
in the target. On the other hand, if a viewer says mosque, and is correct, this is much more
useful from an information perspective than if he says religious. In any case, the current
configuration of levels appears to require some form of readjustment.
D. Overall Guidelines For Making the Numerical Assignments
For targets, a numerical assignment ? is made for each descriptor in response to the
following question:
How visually important is this descriptor to this target?
It should be noted that the descriptor set has been devised to define targets in terms of
visual importance. If the analysis were to be re-framed along a different dimension of
interest--e.g., conceptual, functional, allegorical, etc.--the descriptor set would have to be
revised to address the new mission.
For responses, ? assignments for each descriptor are made in response to a different
question:
To what degree am I convinced that this descriptor represents this response
element?
Responses are coded according to this different criterion, because RV response composites
are often not detailed enough to determine relative visual importance for any given element.
The ?'s themselves will typically consist of one-digit fuzzy numbers between 0 and 1
(e.g., .1, .2, .3, etc.). In some cases, it may be appropriate to make two-digit assignments (e.g.,
.15, .25, .35, etc.); any finer assignments, however, would probably be meaningless.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The sequence for using the descriptor hierarchy is different for targets and responses. For
targets, it is easiest to start with coding the concrete elements at the top of the hierarchy and
work down to the abstract interpretations. For responses, the following sequence appears to be
the most efficacious:
(1)
(3)
Go through the response and assign a " 1" to every element explicitly stated by
the viewer.
(2) Go through the response again and assign ?'s for elements that appear to be
depicted rather unequivocally in the response (like a rectangle, for instance),
but which is not labeled rectangular form or structure as such. These kinds of
cases don't merit " ls" per se (because they require some inference on the part
of the analyst), but the ?'s would be pretty high. Other situations that would
probably fall into this category are instances when a viewer explicitly names
something like hill but equivocates by adding qualifiers like could be or might
Once you are comfortable with (1) your explicit " 1" assignments throughout
the levels and (2) the ? assignments you have made at the lower levels
(primarily 1-4), you should now be ready to make your "interpretive leaps" to
the concrete descriptors in level 5 and beyond. Confidence will probably be
fairly low in synthesizing these higher order composites, and the ?'s should
reflect this accordingly. Of course, if the weight of all your evidence really
leads you to believe that the viewer has constructed a certain kind of
scene--i.e., if your confidence is high--then give it a high A. The important
idea here is that the judicious interpretive leap should be made whenever
possible. The inclusion of the abstract levels does not relieve us of the
responsibility of undertaking reasonable interpretation--rather, they are
included at least in part to serve as heuristic devices for synthesizing higher
order composites. There may be a few cases in which the response data are
simply so sparse that they cannot lead to reasonable concrete interpretation so
do not interpret, as long as it is defendable. These cases are anticipated to be
exceptions.
After the coding process has been finished for any given target or response, it is a useful
exercise to take a Gestalt-oriented look at the coding. The important question to ask is
following: does the descriptor portrait you have painted of this response or target strike you as
reasonable? If not, try to evaluate where the portrait goes awry, and adjust the ?'s accordingly.
E. Specific Guidelines For Descriptor Use
In the course of pilot attempts to use the descriptor list, certain preliminary guidelines have
emerged for the use of specific descriptors. These are summarized by level and by descriptor
below.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
1. Level 1: 1-D Geometry
This descriptor should include curved as well as straight parallel lines. If an
assignment is made for parallel lines, an attendant descriptive assignment for arc, horizontal
lines, or vertical lines is also in order.
b. Vertical Lines, Horizontal Lines, and Diagonal Lines
For targets, these configurations of lines need to be interpreted as they appear
in the 3-D projection of the photograph. Vertical lines are to be interpreted as perpendicular to
the plane of the earth; horizontal lines are lines that are parallel to the plane of the earth;
diagonal lines are lines that are projecting diagonally out of the plane of the earth.
One might justifiably argue that in certain instances, this interpretation for
targets (particularly with respect to diagonal lines) will result in the loss of some information. For
example, if the target is a picture of an agricultural field that contains a vivid diagonal dividing
line between two varieties of crops, we will not be allowed to make an assignment for diagonal
lines because the line is in the plane of the earth. On first examination, this. would appear to be a
potentially catastrophic loss of information. We must remember, however, that we have the
visual correspondence analysis that will enable us to recapture much of this kind of photographic
perspective information. In some sense, this division of the interpretive burden may be ideal,
because we will be able to account for the true linear geometric information contained in the
target site itself, as well as account for the given target photograph's perspective view of that site.
For responses, the intent of the viewer is the central consideration for the
analyst in making an interpretation. For example, the following situation might occur:
The viewer draws And Labels it
this: this:
In this case, the analyst would make no assignment for diagonal lines, because the viewer's intent
was to draw the edge of a plane in perspective.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The wave form descriptor is intended to encompass both actual and stylized
manifestations of wave forms. It will most usually receive a value for a viewer's primitive and
stylized depiction of water:
It should also receive a numerical value for any target manifestation of waves
or ripples--i.e., the boat wake in the Gatun Lake target (1024) is a case in point.
It is probably inappropriate to make wave form assignments based on a strict
physics interpretation. For example, it would most likely not be a viewer's intent to have the
following possible drawings interpreted as wave forms:
a. Irregular Forms (Irregular Features)
Because irregular forms can be visualized in almost every target, it is the
current consensus that this descriptor should be used parsimoniously.
3. Level 3: Archetypes
a. Building(s) (Structure(s))
The building descriptor for targets should receive numerical assignments
according to the prominence (i.e., visual presence or absence) of the buildings for the given
target. In the Havana target (1005), for example, the buildings are quite prominent; in the
Florence, Italy target (1056), on the other hand, one knows in the aggregate that the target is a
city, but the buildings, themselves, are not individually prominent or resolvable. The building
descriptor, therefore, while given a numerical value in both cases, would probably be higher in
the former than in the latter. For responses, the following situation might occur:
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The viewer draws And Labels it
this: this:
"City"
In this case, the analyst would be correct in making a numerical assignment for building(s)
through inference.
For targets, an assignment for the structure descriptor should occur only when
a justification can be made for the misinterpretation of a natural feature as manmade feature. In
a target photograph of a mesa, for example, the structure descriptor would merit some value,
because of the knowledge that mesa geometries are often misinterpreted by viewers as building
profiles. Response codings are usually less ambiguous, because a viewer will typically respond
overtly with "structure," in which case the numerical value would be "L"
The boundaries descriptor is to be used in the sense of a distinct or discrete
dividing line between different elements in a target or response. It may serve to demarcate one
kind of land form from another, land from water, etc. It can also serve in the more traditional
sense of a fence, a wall, etc.
4. Level 4: Qualities
The weathered descriptor is one that should be used parsimoniously; most of
nature (and the targets used) should be regarded typically as falling somewhere in the range
between 0 and 0.1. The intent of this descriptor is to address the strikingly weathered quality of
such targets as the Grand Canyon or any number of ruins sites.
b. Implied Texture
All of the Implied Texture descriptors are to be used according to what the
target element looks like, as opposed to what the analyst imagines the target element would feel
like.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Under the heading of Other Visual, the fuzzy descriptor (a synonym for
obscured) is intended to denote atmospheric obscurity. Under the heading of Implied Texture,
however, the fuzzy descriptor is intended to address textures only. Obviously this is a descriptor
that must be carefully examined in context. If a viewer writes "fuzzy" and clearly associates it
with a response object, then the analyst should be inclined to assign a value to the Implied
Texture fuzzy descriptor; if, however, a viewer writes "fuzzy" and clearly associates it with a
horizon line or the word "sky," then the analyst would be apt to assign a value for the obscured
descriptor.
The striated descriptor is intended to suggest the presence of grooves, furrows,
channels or narrow bands or lines.
Under the Implied Temperature heading, the two most potentially
problematical descriptors are hot and humid. Both descriptors should only be used for targets if
the visual indicators are overwhelming. An analyst might be justified in using hot, for example, if
the target is a photograph of a desert scene with visible heat waves radiating off of the dunes.
The humid descriptor might be used appropriately if the target were a photograph of a dense rain
forest with water clearly dripping off of dense foliage. Clearly, the general rule is to be
parsimonious in making assignments for these descriptors. (See the discussion for weathered,
above.)
This descriptor is to be used only in the literal sense as stated; for plains or
deltas. It is not the intent that it should be used in the more general sense of flat land.
This descriptor is to be used only in the sense of a typical ocean coastline, not
as the coastline of a lake, or other inland body of water.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
The wilderness descriptor is perhaps most aptly thought of in the more general
sense of untouched by the hand of man.
One very persistent conceptual problem appears to be the one-to-many aspect of certain
descriptors in the list, a situation which the list has no clear way of handling. For example, the
descriptor repeat motif is one that could be applied to more than one element in a target or
response. In a hypothetical target for instance, one would like to be able to say that there is a
repeat motif of structures as a city, a repeat motif of irregular forms, invertedlor V-shapes, as
mountains. Given the out-of-third-normal-form nature (to use data base parlance) of the
current list, we are only able to make a single overall assessment for repeat motif; information
seems to be lost (or distorted) in the breaking of the link, therefore, between repeat motif and its
intended element. The same argument would probably be germane for the descriptors
manmade and natural.
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
APPENDIX B
TARGET POOL CLUSTER ANALYSIS
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Agricultural
Scenes
Verdant
Ruins
1093
1037 1194
1194
1072
1043
1010
1101 losa
1021
1052
1019
1029
1004
1187
1107
1050 -~
1159
1141
1103
0.4 0.6 0.8 1.0
1-si,k
FIGURE 1-B CLUSTERS 18 AND 19
Cities With
Prominent Geometries
Cities With
Prominent Structure
1070
}}}~
} llss
Cities On Water
}}~j 1108
}848g~}~~C
}815 1163
f10 1149
}}38 1157
}838 }8~$
0.0 0.2 0.4 0.6
1 - sj,k
FIGURE 2-B CLUSTERS 4, 5, AND 11
gg 1681
}874
}b
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
0.8 1.0
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
1193
1068
Towns on Water
1092
1060
1190
1184
1171
1085
1120
1102
Outposts On Water
loo?s
1147
1034
115,
Ills
1195
1077
Mountain Towns
1009
1073
1007
1006
1079
1031
1026
Flat Towns
1183
1002
1158
1123
1189
1160
1012
0.0 0.2
0.4 0.6 0.8 1.0
1 - Sj,k
FIGURE 3-B CLUSTERS 1, 3, 9, AND 10
Desert/Water
Interfaces
1129
1082
1131
1126
1146--1
1046
1045
Dry Ruins
1048 --J
1087
1033
1051
1042
0.0 0.2 0.4 0.6 0.8 1.0
1 - Sj,k
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Snowy
Mountains
1066
1022
1136
1119
1091
1076
1061
1075 =
1058 1143
1181
156
ZUS~
1057
1180
1090 1086 ----~_
1062 - 1014
1176
1135
I-P
1125
1111
1 - s ] k
FIGURE 5-B CLUSTERS 2, 7, AND 12
1197
1013
1095
Valleys And Rivers
1024
1132
1040
1156
1116
1096
Meandering Rivers
1191
1106
1071
1145
1122
Alpine Scenes
1173
1016
1152
1017
1098
Outposts In
1000
Snowy Mountains
1144
1023
1064
1035
0.0
0.2 0.4
0.6 0.8 1.0
1 - Sj k
FIGURE 6-B CLUSTERS 13-16
8 8
1
111188
1115
110444
1 1 48
1097
1041
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
1198
1138
1186
1179 7
1177 -
1185
11397
1140
1088
0.0 0.2 0.4 0.6 0.8 1.0
1 - Sj,k
Figure 1-B
Figure 2-B
Figure 3-B -'
Figure 4-B -
Figure 5-B
Figure 6-B
Figure 7-B -
0.0 0.2 0.4 0.6 0.8 1.0
1 - S,
J ,k
FIGURE 8-B CLUSTER OF CLUSTERS 1-19
1161
1049
1038
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2
Cr,- 1
-"VG" S/ rw((a-2
A,(w w
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200140001-2