Document Type: 
Document Number (FOIA) /ESDN (CREST): 
Release Decision: 
Original Classification: 
Document Page Count: 
Document Creation Date: 
November 4, 2016
Document Release Date: 
May 19, 2000
Sequence Number: 
Case Number: 
Publication Date: 
June 14, 1995
Content Type: 
PDF icon CIA-RDP96-00791R000200190041-2.pdf83.94 KB
SAIC-(415) 322-7960-Created: Wednesday, June 14,199522:38- Page 1 of 2 --------------- ------------- Appr4wed-Release-?{I0-3f@9M 6 rt:fA=FZDP56-'0D7911R660200190041-2 The Cognitive Sciences Laboratory 330 Cowper Street, Suite 200, Palo Alto, CA 94301 Voice: 415.327.2007 - Fax 415.322.7960 ~. An Employee-Owned Company e-mail: pQemorandum SG1I SG1I Date: 14 June 1995 To: From: Edwin C. May, Ph.D. Re: Project Evaluation o 1 axe evaluation. I am asked to find "the 10 primary studies or reports developed under the subject program ...that.make the best case for establishing the validity of the paranormal phenomena known as Remote Viewing." I am happy to do this as soon as possible. I have a concern, however, about a problem that may be raised that could call any results into question. Dr. Robert Rosenthal in the Psychology Department at Harvard and others have defined a statistical circumstance known as the "file drawer" problem. It is standard practice in behavioral sciences (including the study of anomalous cognition-remote viewing) to use a p=0.05 criteria as a measure of statistical significance. That is, statistical significance is claimed when, given that the null hypothesis (i.e. no RV in this case) is TRUE, there is a 5% chance that a repeated experiment of the same number of trials would yield a statistical deviation as large as in the original, or larger. We use that criterion in our laboratory. The file draw problem is this: Under the null hypothesis 5% of the studies will be statistically significant. If only the significant studies are published (or in our case given to AIR), it is problematical to assess the validity of the research, because it might be that for every study published there are 19 others (in the file drawer) that did not reach statistical significance and were not published. If this were the case, there would be no evidence for an anomaly. Rosenthal and others provide ways of assessing the potential magnitude of this problem. It might be argued that if I am allowed to pick the best 10 studies, and if it turns out that there were 200 others that were not "good," one could be seriously mislead. This is a worst case scenario; however, critics would correctly argue that there is an undetermined file drawer problem with this new approach, and if it is not addressed, the validity of the evaluation can be questioned. I can assure you that Ray Hyman will raise this point. Might I suggest an alternative that address the concerns of the time constraint, yet avoids the file drawer problem. In 1989, SRI published (now downgraded to unclassified) a meta-analysis of all the SRI work from 1973 through 1989. This study was part of the me new tasking with regard to providing support for the project P. d t Page 1 Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2 SG1I ApprovFax I ransmlttalRCover'Sneet041-2 To: - ORD From: Edwin C. May, Ph.D., SAIC Fax Phone Number: (415) 322-7960 Date: Wed, Jun 14, 1995 - 22:38 Transmitting (3) pages, Including cover sheet. If there Is difficulty with this transmission, please call: (415) 322-7960 Note: Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2