Document Type: 
Document Number (FOIA) /ESDN (CREST): 
Release Decision: 
Original Classification: 
Document Page Count: 
Document Creation Date: 
November 4, 2016
Document Release Date: 
May 18, 2000
Sequence Number: 
Case Number: 
Publication Date: 
February 22, 1989
Content Type: 
PDF icon CIA-RDP96-00789R002200450001-8.pdf1.48 MB
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Report of a Workshop on Experimental Parapsychology International Security and Commerce Program Office of Technology Assessment United States Congress February 22, 1989 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release @P/q /qPcF DANA7 9R002200450001-8 ? International Security and Commerce Program Workshop September 30, 1988 Professor James Alcock Glendon College Toronto, Canada Daryl Bem Professor of Psychology Cornell University Ithaca, NY Charles Honorton Director Psychophysical Research Laboratories Princeton, NJ Ray Hyman Professor of Psychology University of Oregon Eugene, OR Rapporteur Courtland Lewis Arlington, VA Robert G. Jahn Professor of Aerospace Studies Princeton University Princeton, NJ John Palmer Institute of Parapsychology College Station Durham, NC Theodore Rockwell Chevy Chase, MD Marcello Truzzi Department of Sociology Eastern Michigan University Ann Arbor, MI Jessica Utts Professor University of California, Davis Davis, CA Chairman Alan Shaw Office of Technology Assessment Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 CONTENTS I. INTRODUCTION ........................................................... 3 Historical Background ................................................. 3 Nature of Research .................................................... 3 Recent Controversy .................................................... 4 Purpose of the OTA Workshop ........................................... 6 II. DESIGN AND CONDUCT OF EXPERIMENTS ...................................... 7 Methodology ............................................................ 7 Replicability ......................................................... 9 Criteria for Acceptable Experiments .................................. 11 III. ANALYSIS OF DATA ..................................................... 12 IV. THE MEANING OF EXPERIMENTAL RESULTS .................................. 16 Assessments of Meaning and Meaningfulness ............................ 16 What Would Constitute Proof of Psi? .................................. 18 Conflicting Orientations ............................................. 19 Need for a Theory of Psi ............................................. 20 V. RELATION OF PARAPSYCHOLOGY TO THE BROADER WORLD OF SCIENCE ............ 20 Is Parapsychology a Science? ......................................... 20 Resistance by "Establishment" Science ................................ 21 Choices for the Future ............................................... 22 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Aaaroved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789ROO2200450001-8 HISTORICAL BACKGROUND The modern field of study known as parapsychology grew directly out of 19th-century "psychical research" into reported psychic phenomena such as telepathy and clairvoyance. The field had its earliest roots in various investigations into a wide range of supernatural, occult, and mystical topics reaching far back into human cultural history. However, the field did not become formalized as an arena of organized scholarly research until 1882, with the establishment in London of the Society for Psychical Research (SPR). A U.S. counterpart, the American Society for Psychical Research (ASPR), was founded in 1885. Both societies still exist today.1 From the beginning, the field represented by these organizations has been highly controversial. Although it has occasionally attracted and engaged well-known scientists, it has generally encountered substantial resistance and criticism from the scientific "establishment" on the basis of its aims and methods. To its detriment, parapsychology has had difficulty in freeing itself of association, in the minds of those outside the field, with extreme and unsubstantiated claims, commercial ventures of questionable validity, and a certain amount of quasi-scientific "research" carried out in its general area of inquiry. NATURE OF RESEARCH Throughout this century a considerable body of parapsychology research has been conducted in a manner that attempts deliberately to follow scientific methodology. The research approaches most often derive from and resemble research in psychology, although a few programs based on engineering and applied science have recently appeared. A seminal example of academic research in the field was the work of J.B. Rhine and Louisa Rhine at Duke University, beginning in the 1920s. The Rhines established many of the basic concepts and protocols of modern parapsychology. In 1937, they founded The Journal of Paraps cholo2y, which is perhaps the leading journal in the field today. The general focus of parapsychological research is on "psychic phenomena," or "psi," which is defined as "processes of information and/or energy exchange which involve animate consciousness in a manner not currently explicable in terms of known science."2 The field can be divided into several major categories and subcategories, as shown in Table 1. Most (but not all) of the parapsychological research in laboratory settings involves categories I and II. 1 For a detailed examination of the development of parapsychology, see: Jahn, R.G. & B.J. Dunne. Margins of Reality: The Role of Consciousness in the Physical World. New York: Harcourt, Brace, Jovanovich, 1987. 2 Jahn, R.G. The persistent paradox of psychic phenomena: An engineering perspective. Proceedings of the IEEE, 1982, 70(2): 136-170. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO2200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 In 1957 a professional organization in the field, the Parapsychological Association (PA), was formed. (Unlike the SPR, membership in the PA requires formal recognition of professional status by the Association's Council.) It was accepted as an affiliate by the American Association for the Advancement Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 I. Extrasensory Perception (ESP) A. Telepathy B. Clairvoyance C. Precognition/Retrocognition D. Animal ESP II. Psychokinesis (PK) A. Physical Systems (equipment, etc.) B. Biological Systems A. Reincarnation B. Apparitions C. Mediumship IV. Out-of-Body Experiences (OBE) SOURCE: Jahn, R.G. The persistent paradox of psychic phenomena: An engineering perspective. Proceedings of the IEEE, 1982, 7Q(2): 136-170. Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 of Science (AAAS) in 1969, signaling a tentative new status and recognition of the field. In 1976 a group of philosophers, magicians, science writers, scientists, and others concerned about a widespread increase in popular interest in the occult formed a Committee for the Scientific Investigation of Claims of the Paranormal (CSICOP).3 CSICOP and a number of affiliated local groups have since become a primary source of criticism and skepticism directed primarily at highly publicized claims of the paranormal, but often-extending more generally to the field of parapsychology. A highly charged and long-running debate has emerged, featuring charges and countercharges between members of CSICOP and the scientific community, on the one hand, and members of the parapsychological community, on the other. One recent chapter of that debate appeared in the March 1985 issue of Me, Journal of Paravs chologyy, in which a leading parapsychology researcher (Charles Honorton) and a leading psychologist and critic (Ray Hyman) presented their views on the nature of one category of parapsychology experiments.4 In the December 1986 issue of the same journal, the two published a "joint communique" in which they explored in a constructive manner areas of agreement and disagreement regarding experiment design, documentation, and data analysis.5 (Both individuals participated in the OTA workshop.) Another landmark publication appeared in 1987: a substantial portion of one issue of Behavioral and Brain Sciences was devoted to articles and letters on the subject.6 The most recent episode in the continuing controversy over the field appeared in the form of a December 1987 report by the National Research Council (NRC), which evaluated parapsychology along with other possible mechanisms for enhancing the performance of U.S. Army personnel.7 The NRC's evaluation of the potential of parapsychology was generally negative, prompting strenuous and detailed claims of bias and unfairness from the parapsychological community.8 It was in this context that the OTA workshop took place. OTA examined parapsychology at the request of its oversight body, the Technology Assessment Board, which expressed interest in the human potential aspect of 3 It should be noted that few scientists were involved in the creation of CSICOP. Although today many of its Fellows are scientists, the organization has always been led and operated by non-scientists. 4 Hyman, R. The ganzfeld psi experiment: A critical appraisal. and Honorton, C. Meta-analysis of psi ganzfeld research: A response to Hyman. The Journal of Parapsychology, 1985, 42(1) : 3-91. 5 Hyman, R. & C. Honorton. A joint communique: The psi ganzfeld controversy. The Journal of Parapsychology, 1986, .Q(4): 351-364. 6 Behavioral and Brain Sciences, 1987, 1Q(4). 7 National Research Council. Enhancing Human Performance: Issues. Theories. and Techniques. Report of the Committee on Techniques for the Enhancement of Human Performance, Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press, 1988. 8 See, for example: Palmer, J.A., C. Honorton, & J. Utts. "Reply to the National Research Council Study on Parapsychology." Special report prepared for the Board of Directors of the Parapsychological Association, Inc., Research Triangle Park, N.C., 1988. Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 7 parapsychological research. This workshop and report are OTA's response to that interest. OTA's intention in holding the workshop was not to continue the debate described above, but to illuminate it by identifying the main points of contention and the reasons for them. For that reason, background materials were researched to provide a fuller elucidation of some issues not adequately aired in the course of the one-day meeting. Discussion was limited to the area of laboratory parapsychology. Participants represented three groups: researchers in parapsychology, constructive critics of the field, and knowledgeable individuals who follow the field from a position of more neutral interest or involvement. Although the meeting was to some extent an occasion for spirited debate, no attempt has been made to "score" that debate. This report will describe the controversies only to the extent that they bear on OTA's inquiry into the status of the field. II. DESIGN AND CONDUCT OF EXPERIMENTS Controversy in the area of experiment design and conduct appears to center around the two issues of flaws in methodology and replicability of experiments. METHODOLOGY One of the most persistent and perhaps most damaging charges leveled by the critics is that various flaws in the design and conduct of nearly all parapsychology experiments render their results scientifically less meaningful than they are reported to be -- and, in fact, often account for the results. Parapsychologists dispute this charge, saying that most of the alleged flaws have little or no effect on the experimental findings that indicate "psi," and that much research in psychology (the home discipline of many critics) -- and indeed in many other fields of science -- exhibits similar flaws without encountering comparable criticism and questioning of the results. They say the question of flaws is highly subjective, with individual views differing widely over what distinguishes an experimental "flower" from an experimental "weed." Further, the parapsychologists claim, this criticism is a red herring intended to tarnish the image of parapsychology within the scientific community and the public and to impede broader awareness of findings and progress in the field. Those at the workshop asserted strenuously that they and their colleagues make every effort to conduct their experiments in the most rigorous manner possible. The real issue, they say, is what constitutes a flaw. The types of flaws that have been brought into question include purported procedural. flaws such as: o inadequate precautions against "sensory leakage" (for example, allowing subjects in a telepathy experiment to examine the same "target", such as a photograph, that had been handled earlier by a person serving as "sender") o inadequate security provisions (e.g., to prevent tampering with equipment) o improper randomization techniques (such as hand shuffling of cards) o feedback (such as failure to randomize targets before presenting them to the subject for judging) o incomplete documentation of experimental procedures o inconsistency of conditions and procedures used in an experiment. Also at issue are various purported analytical and statistical flaws, including: o multiple testing or analysis (i.e., to find the test or analysis that gives the most positive "effect size" for psi) Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 o underestimation of the effective error rate and overestimation of the actual significance level o erroneous use of particular statistical procedures. In their joint communique, Honorton argued that there is "no significant correlation between indices of study quality (flaws] and study outcome." Hyman agreed in most cases, but insisted that there is a positive correlation in the case of poor randomization, feedback, and inadequate documentation.9 (This assertion was rebutted in a later analysis.)10 Workshop participants generally agreed that in any event the "flawless," perfect experiment is very rare in any field, if it exists at all. The point of the issue, critics say, is that the presence of a flaw usually implies inadequate controls for error. The most stringent among them believe that an apparently successful experiment cannot be considered to have demonstrated a true anomaly (i.e., a possible psi effect) unless it can be shown that the experiment is completely flawless. The parapsychologists counter that any such flaws should be an issue only if they could have systematically influenced the result. Another point made by parapsychologists is that, by and large, the critics have done very little experimentation in parapsychology. Therefore, they have little awareness of the difficulties encountered or of what the truly significant flaws might be. Indeed, the parapsychologists note, there could be additional flaws that have not yet been considered and which critics could identify through actual experimentation. The NRC report distinguished three types of criticisms relating to flaws in experiment design and conduct.11 First is what is called the "smoking gun," in which observed findings are said to be definitely attributable to a specific flaw or other factor (such as deliberate fraud) that is shown to be present. Second, and more common, is the "plausible alternative" allegation that a particular flaw is present and could have produced the reported results. Third is the "dirty test tube" concept, which alleges not that a particular discovered flaw produced the positive result but simply that the presence of flaws in the experiment demonstrates a general sloppiness which brings the results into question. In other words, if results have been obtained under conditions that fail to meet "generally accepted standards" for scientific research (such as clean test tubes), that fact alone casts doubt on the accuracy and validity of the results. The authors of the NRC report claimed that, while they could find no instances of the "smoking gun" or "plausible alternative," even the best parapsychology experiments exemplify the "dirty test tube" problem (an allegation that has been heatedly disputed by parapsychologists). Participants agreed that the burden of proof in the first two categories should be on the critic making the allegation, as it is in other sciences where an experiment is challenged on these grounds. In the case of the "dirty test tube," the critics feel that the burden of proof should be on the experimenter. This position brings from the parapsychologists the objection that it is non- 9 Hyman & Honorton, op. cit., p. 353. 10 Harris, M.J. & Rosenthal, R. "Interpersonal Expectancy Effects and the Human Performance Research." Washington, DC: National Academy Press, 1988. (p. 3) 11 National Research Council, op. cit., pp. 199-200. Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 9 falsifiable -- that is, that there is no definitive defense against the charge of "general sloppiness". Without detailed criteria for what constitutes acceptable research, they say, criticism of an experiment cannot be anticipated or refuted. In their view, it is an example of the unfairly high standards that are set for parapsychology research, in which no amount of care is ever "enough." The critics present at the workshop tended to agree in principle with the unfairness of "non-falsifiability," but they believe that the "dirty test tube" charge is nevertheless valid because the success of the experiments rests on whether a small but significant departure from some statistical norm or observed norm has in fact been seen, making it essential to eliminate extraneous factors. Parapsychologists object that the charge itself is irrelevant -- that if the alleged flaws or "dirt" have no demonstrable connection with the experimental outcome, it is capricious to argue that the experiment is invalid. The litany of charge and countercharge could be detailed much further. Suffice it to say that the controversy surrounding parapsychology is nowhere more heated and complex than in the area of experimental methodologies. REPLICABILITY For critics of parapsychology, a key argument against the existence of psi phenomena is the difficulty of replicating (repeating) successful experiments -- a difficulty that also plagues many accepted areas of the social and behavioral sciences. If the same effect could be produced many times by different experimenters using different equipment but the same procedures, then the likelihood that an extraneous variable is producing the effect would be much lower and the validity level of the anomalous results would be higher. However, positive results are in most cases maddeningly (to parapsychologists) difficult to replicate, even with the same operator. (Because the effects seem to be highly operator-specific, results are even more difficult to replicate with different operators or groups of operators.) Parapsychologists attribute this replicability problem to the elusiveness and weakness of the psi "signal," and to its vulnerability to subtle subjective factors, most of them unknown. Critics contend that the near non-replicability merely demonstrates the effects of randomness combined with multiple flaws in experiment design and conduct. The parapsychologists are well aware that experimental flaws, both actual and alleged, become less important as a phenomenon becomes more replicable. Therefore, in recent years they have placed great emphasis on replicability. As one noted in his presentation, "The final criterion of the success or failure of my research program is the extent to which I'm able to develop procedures that I can articulate with sufficient precision that other people are able to obtain similar results." In the case of ESP-oriented experiments (such as remote perception or "ganzfeld" psi experiments), the number of these more standardized experiments is still not large. There, parapsychologists have relied upon what is termed "meta-analysis" of many studies, taken as a group, to give statistical confirmation for a pattern of positive results. The rationale is that "an Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 experiment or an effect may be considered replicated if a series of replication attempts provides statistically significant evidence for the original effect when analyzed as a series."12 It is also contended that this approach allows the analyst to assess methodological flaws to determine their effect empirically across the data base. Using the meta-analytic approach, parapsychologists have calculated an overall "success" rate, in achieving statistically significant effects (P .05) are too often interpreted erroneously by critics as insignificant results. An example was given to demonstrate that sample size (n) has a substantial impact on the statistical outcome; if n is small there is very little chance for a "successful" outcome, even if relatively high actual probabilities of success based on ESP are assumed. Meta-analysis of all the forced-choice precognition experiments conducted from the 1930s on showed that Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 13 those with significant results had, on average, two and one-half times larger sample sizes than those with nonsignificant results. Thus, large values of n impart considerably greater "power" to the analysis. Power is, essentially, the probability that the experiment is going to succeed. (This concept of power is now being taught at the graduate level in statistics.) Table 2 illustrates how power increases with increasing n. At least one of the critics present argued that there is no correlation between sample size and the probability of obtaining a significant outcome, but Utts refuted that claim. It was agreed that the spread of data -- for example, one or two "outliers" (data points far outside the general cluster) -- can greatly weaken the correlation. The statistician recommended that parapsychologists stop focusing on the arbitrary, "sacrosanct" p