The Weiler Psi

Parapsychology Journalism: The People, The Theory, The Science, The Skeptics

The Bem Precognition Meta-Analysis Vs. Those Wacky Skeptics

To get you in the proper mood to evaluate the latest skepticism regarding psi research, I present a two minute clip from the otherwise somewhat forgettable movie  Erik The Viking.  The island is sinking and the mayor and many townspeople are in complete denial:

This all started from a previous blog post some years ago that made this statement:

The point of all this is not to mock parapsychology for the sake of it, but rather to emphasise that parapsychology is useful as a control group for science. Scientists should aim to improve their procedures to the point where, if the control group used these same procedures, they would get an acceptably low level of positive results. That this is not yet the case indicates the need for more stringent scientific procedures.

From there, a blog by physician, Scott Alexander titled: “The Control Group is Out of Control” has dragged this convoluted reasoning out from under a rock, where it should have stayed:

The results are pretty dismal. Parapsychologists are able to produce experimental evidence for psychic phenomena about as easily as normal scientists are able to produce such evidence for normal, non-psychic phenomena. This suggests the existence of a very large “placebo effect” in science – ie with enough energy focused on a subject, you can always produce “experimental evidence” for it that meets the usual scientific standards. As Eliezer Yudkowsky puts it:

Parapsychologists are constantly protesting that they are playing by all the standard scientific rules, and yet their results are being ignored – that they are unfairly being held to higher standards than everyone else. I’m willing to believe that. It just means that the standard statistical methods of science are so weak and flawed as to permit a field of study to sustain itself in the complete absence of any subject matter.

The reasoning goes something like this:

1.  Psychic ability cannot possibly exist, yet . . .

2.  Parapsychology routinely gets positive results, therefore . . .

3.  The statistical methods themselves must be flawed.

It’s pretty obvious that this line of reasoning is, well, batshit crazy.  Faced with the realization that parapsychological research is statistically and methodologically sound, these otherwise very intelligent people have gotten themselves caught up in an immature, playground-quality argument.  If they can’t win, well then, the whole game must be stupid.

The renewed interest in this idiotic line of reasoning was occasioned by a new meta analysis of Daryl Bem’s “Feeling the Future” experiments which involved 90 experiments.  (You can find a description of the experiment here.)  A little background is required to understand why this set of experiments, and not say, the Ganzfeld, are so difficult for skeptics to brush off.  For starters, there is Bem himself.  There is no way to dismiss him as a fringe parapsychology nut.  He is a social psychologist and professor emeritus at Cornell University. He is also the originator of the self-perception theory of attitude change.  One thing that he has always been good at is getting published in the mainstream journals.  The author or co-author of over 70 scientific papers, editor of five different mainstream journals and a reviewer for many more, author, or co-author of 7 psychology textbooks as well as psychology related books, Bem is not someone who can be taken lightly.  He is well known for being a good experimenter.

In early 2011, his paper, “Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect“, published in the prestigious Journal of Personality and Social Psychology came out to the usual mixed bag response.  What was hard to ignore was that the JPSP is one of the most rigorously refereed journals in the entire field of psychology, with a rejection rate of 82% in 2009.  This paper was accepted based on the collective assessment of the two editors and four reviewers who vetted it for the Journal.  Moreover, authors’ names and other identifying information are removed from a manuscript before it is sent to reviewers so that their evaluations will be based purely on its merits and not be influenced by knowledge of an author’s reputation or the prestige of his or her institutional affiliation.

The skeptical trash talk machine kicked into high gear.  In January of 2011, James Alcock, in an article for the unapologetically biased Skeptical Enquirer, wrote a factually challenged, over-the-top piece slamming not only Bem’s paper, but all of parapsychology as well.  Bem responded to this.  The more moderate mainstream skeptical response was that there weren’t enough replications to accept the results.  What was particularly interesting was that no one was attacking the methodology this time.  It was already vetted by the JPSP and was therefore beyond reproach.

In 2010, skeptic Richard Wiseman and wife Caroline Watts set up a registry for replications of Bem’s experiments.  It was ingenious attempt to grab control of the replications and make it appear as if the experiment was a complete failure.  In 2012 Wiseman gathered up his meager results and wrote up a paper and shopped until he found a journal to accept it and was published in March of 2012. Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s ‘Retroactive Facilitation of Recall’ Effect.  This failure to replicate got a great deal of press, as noted in the comment section of that paper.

However there were six studies that were pre-registered, not three.  Bem provided a detailed comment to Wiseman’s paper, which he summed up in this Skeptiko interview:

What Wiseman never tells people is in Ritchie, Wiseman and French is that his online registry where he asked everyone to register, first of all he provided a deadline date. I don’t know of any serious researcher working on their own stuff who is going to drop everything and immediately do a replication… anyway, he and Ritchie and French published these three studies. Well, they knew that there were three other studies that had been submitted and completed and two of the three showed statistically significant results replicating my results. But you don’t know that from reading his article. That borders on dishonesty.

As of 2012, as far as the mainstream press knew, Bem’s experiments had started out promisingly, but no one had been able to replicate them.  In fact, other, somewhat similar types of precognition experiments had been around for many years with successful replications.  An early description of one can be found in Dean Radin’s landmark book “The Conscious Universe” (pg. 118-124) published in 2000.

In 2012, Julia Mossbridge, Patrizio Tressoldi and Jessica Utts published a meta analysis of similar studies dating back to 1978.   (Update:  The experiments have some significant differences.  See Michael Duggan’s comment below for clarification.)  Studies into subliminal precognition had been around a long time and weren’t anything new.  As with most science, this new experiment wasn’t a bolt out of the blue, but just another iteration of an already tried and true area of study.  In parapsychology, this effect was already taken for granted.

What Bem did mostly was to use his knowledge and experience to tighten it up, improve the protocol, make sure it was easily replicable and most importantly, get it published in a high profile way.  There never was any real chance of failure.  Over the long term, the experiment was eventually going to get a sufficiently large number of successful replications; it was only a matter of time.

From a scientific perspective, this happened rather quickly.  In early 2014 the first comprehensive meta analysis was made public, (although it is still under review.)  A quote from the abstract sums it up:

In 2011, the Journal of Personality and Social Psychology published a report of nine experiments purporting to demonstrate that an individual’s cognitive and affective responses can be influenced by randomly selected stimulus events that do not occur until after his or her responses have already been made and recorded (Bem, 2011). To encourage exact replications of the experiments, all materials needed to conduct them were made available on request. We can now report a metaanalysis of 90 experiments from 33 laboratories in 14 different countries which yielded an overall positive effect in excess of 6 sigma with an effect size (Hedges’ g) of 0.09, combined z = 6.33, p = 1.2 × 10-10. A Bayesian analysis yielded a Bayes Factor of 1.24 × 109, greatly exceeding the criterion value of 100 for “decisive evidence” in favor of the experimental hypothesis (Jeffries, 1961).

An effect in excess of 6 sigma is greater than the proof that was required for the Higgs Boson.  It’s very, very significant.  And so now there is a replicated, bulletproof experiment that demonstrates the existence of psychic ability.  The methodology was vetted by a prestigious mainstream journal, so there is no wiggle room there.  You can’t pull the “selective reporting” (a.k.a. filedrawer effect) card because the study hasn’t been around long enough for a bunch of unreported studies to have taken place.  And just as importantly, care was taken to provide exact replications.  It’s not really possible to dispute the results of this experiment in any truly scientific way, which is why skeptics have resurrected the whole “statistical science itself must be flawed” argument.  They have completely run out of any remotely sane options for disagreement.

So what now?  Well, it’s always possible that some group of skeptics will get together to do a Bayesian analysis and set their prior probability number to something impossible and then claim that the meta analysis actually showed no effect.  It wouldn’t be the first time that has happened.  Daryl Bem, Jessica Utts and Wesley Johnson did an excellent and very readable paper on how this game was played with Bem’s original study.  There,Wagenmakers, Wetzels, Borsboom, and van der Maas (2011) did something ridiculous:

The first and most familiar is the prior odds that H0 rather than H 1 is true. It is here that Wagenmakers et al. (2011) formally expressed their prior skepticism about the existence of psi by setting these odds at 99,999,999,999,999,999,999 to 1 in favor of H0.  [The odds that psychic ability does not exist.]

If they pulled the same stunt with the meta analysis, that huge number would have to be exponentially higher and be so ridiculous that even they might balk at it, but the question is, would anyone examine the skeptical paper to find out if the prior probability has been set to a plausible number?  Probably not.  What the media would focus on would be that someone disagreed.  It’s complete garbage, but it would make the results of the meta analysis “controversial” and that’s all that the mainstream press would need to blow it off.  Aaaand, it wouldn’t be the first time this has happened.


24 comments on “The Bem Precognition Meta-Analysis Vs. Those Wacky Skeptics

  1. Vortex
    July 12, 2014

    Craig, Parapsychological Association has just published the preliminary program of events for its 57th Annual Convention:

    I looked at it, and I can’t find your name on the list of speakers. Does it mean that you changed your mind and are not going to speak there? Or is it just the early version of the program, so you haven’t been included (PA states that this preliminary program is “subject to change”)?

    So, Craig, are you going to the PA convention? Or not? It is up to you to choose, of course, but your refusal to participate would be a true disappointment for me (and, I think, many other Skeptiko forum members and psi proponents and general).

    • craigweiler
      July 12, 2014

      No, that’s correct. I’m not on the program. What I had prepared was rejected, and to be truthful, their reasoning was sound. I was offered a poster session and turned it down. I’m not at all good at that sort of thing.

      It’s not a refusal to participate, by any means. I still want to attend.

  2. Vortex
    July 7, 2014

    Craig, here are some good news to save you from the mental crisis: there was a parapsychology workshop at US National Academy of Sciences center, with the participation of Dean Radin. Radin wrote a blog post about it:

    And I started a Skeptiko thread about it:

    Craig, I hope to see your reply either here on your blog, or there on my thread, or both. This is not a time to be silent. Great change is coming – and it was you who said it all the time, didn’t you?!

    I really hope to hear from you, Craig. I miss your commentary.

  3. hey craig,

    hope your absence from blogging here means you are busy doing wonderful things.
    awaiting the next installment.


    • craigweiler
      June 24, 2014

      Hi Billy,
      I don’t know what’s up with me. Mentally, everything has ground to a halt and I haven’t written much of anything recently.

      I suppose that part of it is that some minor injuries have been accumulating from my day job and I haven’t been feeling all that great. But part of it is that I’ve just hit a wall mentally.

      • Mark
        June 24, 2014

        Hmmm…sorry to here that, Craig. Best wishes. I hope everything gets better for you.

      • I hope you feel better craig.

        trust the process re: healing and writing. maybe it’s a time to think & communicate less and simply feel more.

        our wishes are with you. you’ve provided so much this last year especially you are due a solid rest.

      • Vortex
        July 1, 2014

        Craig, if you’re tired of writing, you are fully entitled to have a rest – until August and annual convention of Parapsychological Association. You will be speaking here – and having a close encounter with most people active in the field.

        Would be great for you – meeting all the persons whom you have only known previously from their papers, books, interviews and videos.

        I hope you will recover then, and provide us with a couple of posts about the convention (couple of photos is also desirable!)

  4. Jordan Wm. Burrill
    June 6, 2014

    Hi Craig!
    The theory of particle phenomena such as the appearance of the Higgs is very interesting to consider! I recently found an article in which group meditation was reported to collapse a wave, superposition state. It really (from my perspective), not be so surprising to find – if consciousness is coupled to the ZPE ground fields matrix. It actually should be expected! This would really send the woo woos into a tizzy! 😀

    Cheers Craig! 🙂

  5. John Amenta (@jamenta)
    May 26, 2014

    Just came across this SPR article on CSICOP by George Hansen. Might be somewhat off topic, but I thought it was well done enough to share:

    • John Amenta (@jamenta)
      May 26, 2014

      “In examining the scientific status of CSICOP, sociologists Pinch and Collins (1984) described the Committee as a “scientific-vigilante” organization (p. 539).”

    • John Amenta (@jamenta)
      May 26, 2014

      “Commenting on an article in SI, medical professor Louis Lasagna (1984) wrote: “One can almost smell the fiery autos-da-fe of Torquemada and the Spanish Inquisition” (p. 12).”

  6. Jordan Wm. Burrill
    May 8, 2014

    Hi Craig!
    Sorry it’s been a while since I bothered you! 😀

    But, There is another perspective that should be considered. We can call it ‘the reverse placebo effect’. ‘Intention’ (as Dean Radin’s experiments in telekinesis shows), can affect results directly, in the outcome of an experiment.
    Expectation equals – intention equals – influence on the study!
    I remember a story told by Stargate RVer Joe McMoneagle (in “Mind Trek”), about a visiting congressman who was also a confirmed disbeliever in RV. He was offered an opportunity to participate in a session – which he accepted (for his own reasons). He failed the session and crowed about his failure as proof of this program being a waste of tax dollars. Until, he was actually shown the results of his session. He had consistently failed at acquiring and describing his targets far below random 50/50 chance! His expectation/intention decided his results! To this he was reported to be much more sober when leaving SRI that day.
    So, this MUST be factored into any ‘legitimate’ set of experimental protocols for any study of this area – to simply be taken seriously! This means double or triple blind protocols done by open-minded experimenters (separated from whom ever is funding it), for it to be taken seriously.
    That is why part of my ascension training with my students includes EFT skills, to ferret-out any unconscious, sabotaging, inhibiting misbeliefs, first.


    • craigweiler
      May 8, 2014

      Hi Jordan,
      I am aware of the experimenter effect and it’s addressed in a great deal of the psi literature and to a certain extent, in the “control group” article.

      It’s not a situation unique to parapsychology. I’ve even seen suggestions that the Higg’s Boson might be an artifact of this effect.

      The only thing that’s unique is the way it’s being used to portray parapsychology as a fake science.

      It’s not even clear whether double or triple blinding will solve the problem. Consciousness is not well understood.

      Something to bear in mind here is that the effect is real and how it was accomplished is irrelevant.

      • Stephen Baumgart
        May 8, 2014

        Hi Craig,

        Have you ever heard of the pentaquark controversy in high-energy physics? A few labs (like Spring-8) reported the exciting discovery of a new particle composed of 5 quarks called the “pentaquark”. However, as more and more labs starting looking for the particle, collecting data over much longer runs of data at higher statistics, the signal went away during later experiments. No one has found a convincing explanation as to why the original discoveries were spurious. It makes one wonder…

        Note: I’ve worked with people on both sides of the pentaquark debate. But since no one was familiar with things like experimenter effects or the decline effect, those explanations were never brought up.

        • craigweiler
          May 9, 2014

          I had not heard of that, but it’s an excellent example. The rest of the science community could learn a great deal from parapsychology

  7. Mark
    May 6, 2014

    I hope I’m not taking us too far off topic, (go ahead and refuse this comment if I am, Craig) but the video about Hy-Brasil reminded me of this article that I read, recently:

    What does this article, linked to just above, have to do with the main post? Well, for one thing, the article has to do with the possibility of time travel explaining The Rendlesham Forest UFO Incidents, and there is a comment in the comments section that indicates that a book by Gorgina Bruni has more evidence of time travel being the cause of all sorts of paranormal occurrences in that area going back centuries, at least. Also, although it does not appear in the article, itself, I believe that the notion of The Legend Of Hy-Brasil is mentioned in connection with The Rendlesham Forest UFO Incidents in an interview with Ryan Sprague that is posted on a website called, though I don’t really feel like looking it up. Supposedly, Hy-Brasil is an Atlantis-like island that’s really out there, somewhere, though, if I remember the interview properly, Hy-Brasil struck me as being more like that island on the TV show called: “Lost” than Atlantis.

    • craigweiler
      May 6, 2014

      The comment is indeed decidedly off topic, but so what? it’s interesting nonetheless.

  8. Vortex
    May 6, 2014

    Craig, you should look at the Dean Radin’s blog post about the very recent (2014) presentiment “Frontiers of Psychology” meta-analysis by Julia Mossbridge, Patrizio Tressoldi and Jessica Utts, which continue the line of investigation marked by their previous meta-analysis of 2012:

    There are 85 comments already, and the debate is hot… Well, “the skeptical silliness”, as Alex Tsakiris would deservedly call it, has reached its new heights (or depths?). One should look at the comments by “Terry the Censor” and other skeptics there. It appears that they have simply no reasonable arguments at all – so they have to invent the wild pseudo-arguments with ever-growing degrees of absurdity.

    You really should look at it – sometimes I didn’t know whether I should laugh or cry when I read these “criticisms”.

    (BTW, the meta-analysis itself is here:

    Its comment section have to be read – a lot of intersting things discussed there.)

  9. Michael Duggan
    May 6, 2014

    Always a pleasure to help Craig, and thanks for all your hard work promoting the field generally.

  10. Oligopsony
    May 5, 2014

    This is a total nitpick, but the (pen) name of the author in question is Scott Alexander, not Scott Campbell, and he’s a physician, not an academic.

    • craigweiler
      May 5, 2014

      Corrected, thanks.

  11. Michael Duggan
    May 5, 2014

    Hi Craig,

    The data are very interesting indeed, although I would like to emphasise our meta-analysis is still under review, despite the early draft posted at the social sciences research network- SSRN. I would also like to clarify an area of apparent misunderstanding: the experiments meta-analysed by Julia Mossbridge, et al, and featured in the Conscious Universe investigated a phenomenon known as presentiment, in which unconscious bodily processes, for example, electrodermal activity, react several seconds before a randomly applied stimulus. These experiments are very different from those investigated by Bem and replicated by others. These examine implicit precognition in which the participants behavioural measures, for example, reaction speed, are inflenced from the future, but the data are generated from a covert precognition test, usually disguised as a regular psychology experiment. BOTH research paradigms are produing very exciting data, and we hope to have our meta-analysis published soon.

    • craigweiler
      May 5, 2014

      Thanks for the clarifications. I’ve made a note of the fact that the meta analysis isn’t published yet and I’ve directed readers to your comment about the differences in the studies. I appreciate your help.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: