Precognition Basically Proven, Skeptics Prove Nothing Can Convince Them

This story began six years ago in 2011 when Daryl Bem published the results of nine experiments in the Journal of Personality and Social Psychology.  

I’m writing about it now because we’re coming up on the publishing of a meta analysis of 90 experiments. But before I get into that a little history is in order.

The original experiments caused a larger than ordinary uproar among skeptics for a couple of reasons. First, this is a really prestigious journal. The JPSP is one of the most rigorous journals in all of psychology and had a rejection rate of 82% in 2009. Author’s names and identifying info are stripped to so that the study can be reviewed purely on merit.

Second, this was not an exotic parapsychology experiment, but rather an ordinary psychology experiment with some pieces flipped around. (Bryan Williams created an excellent 20 page summary which you can find here.) It made the experimental design very hard to criticize. Not that skeptics didn’t try.

James Alcock, a psychology professor in Toronto, wrote a long article criticizing the study and pretty much everything else in parapsychology. That it was written for and published by the Skeptical Inquirer, one of those zealot atheist magazines that praises children for being skeptical of Santa Claus is telling. (I’ve covered the parent organization here.) You can find rebuttals to that article here and here.

Ray Hyman, a member of the aforementioned atheist organization, attempted the shaming strategy:

“It’s craziness, pure craziness. I can’t believe a major journal is allowing this work in,” Ray Hyman, an emeritus professor of psychology at the University Oregon and longtime critic of ESP research, said. “I think it’s just an embarrassment for the entire field.”

But the experimental design, being ordinary in almost every aspect, wasn’t a particularly good path for attack, so the skeptics resorted to their old standby, the Bayesian Analysis.

At around the same time as Bem’s paper, Wagenmaker, et. al, published a paper that purported to demonstrate that the study actually had null results, i.e., nothing happened. To do this though, they had to do something extraordinary. Tamper with the number that they use to gauge expectations. In this case, Wagenmaker set the prior probability to 99,999,999,999,999,999,999 to 1 odds in favor of H0. In other words, that is the probability that psychic ability does not exist in their minds. It looks like they came up with this number by simply holding down the 9 key and counting to five. There is certainly no real world explanation for it. Precognition shows up in warfare often enough for it to be seriously studied. And they could have maybe referenced previous psi studies to get this number?

It’s a clear case of torturing numbers to get the result you want.

Bayesian analysis also requires you to set what you think the effect size will be: This is more complicated, so here’s a direct quote from this rebuttal:

Consequently, no reasonable observer would ever expect effect sizes in laboratory psi experiments to be greater than 0.8—what Cohen (1988) terms a large effect. Cohen noted that even a medium effect of 0.5 “is large enough to be visible to the naked eye” (p. 26). Yet the prior distribution for H1 that Wagenmakers et al. (2011) adopted places a probability of .57 on effect sizes that equal or exceed 0.8. It even places a probability of .06 on effect sizes exceeding 10. If effect sizes were really that large, there would be no debate about the reality of psi. Thus, the prior distribution Wagenmakers et al. placed on the possible effect sizes under H1 is wildly unrealistic.

So what Wagenmaker is basically saying here is that psychic ability can’t possibly ever ever ever ever ever exist, but if it did, we’d have Magneto and Professor X dueling overhead. Setting these kinds of expectations is a great way to make psychic ability statistically disappear in a Bayesian analysis.

Wagenmaker rebutted the rebuttal here where he claimed that the problem wasn’t with Bem’s study, but with the whole of psychology. Bem was merely caught up in a failed system.

instead, our assessment suggests that something is deeply wrong with the way experimental psychologists design their studies and report their statistical results. It is a

disturbing thought that many experimental findings, proudly and confidently

reported in the literature as real, might in fact be based on statistical tests

that are explorative and biased (…). We hope the Bem article will become a

signpost for change, a writing on the wall: psychologists must change the way

they analyze their data.”

 

 Got that? Since Bem is following ordinary psychology experimental protocol and he MUST be wrong, then the only logical conclusion is that EVERYONE must be wrong.

If your jaw dropped on that whopper, it gets better. Others took this seriously . . . for awhile. Since all of experimental psychology would apparently not change to make one positive result go away, the whole thing kind of faded into the background.

This was also a complicated issue so it didn’t quite take the shine off of Bem’s experiments. Something more was needed. Something that as Bem put it, “borders on dishonesty.”

Enter Richard Wiseman. Wiseman has never completed a successful psi experiment and instead has apparently made a career of trashing other people’s work. He trashed the Feilding report on Eusapia Palladino (but backed off later on being carefully questioned), Rupert Sheldrake’s dog experiment (even though he got the same results), botched a meta analysis of the Ganzfeld experiments (by doing is calculations wrong), attempted to discredit the Girl with the X-Ray eyes (by requiring her to meet too high a standard according to critics) and generally has a wake of controversy a mile long that follows behind all of his dealings with parapsychology. You can pretty much guess what happened next. I wrote at the time:

In 2010, skeptic Richard Wiseman and wife Caroline Watts set up a registry for replications of Bem’s experiments.  It was ingenious attempt to grab control of the replications and make it appear as if the experiment was a complete failure.  In 2012 Wiseman gathered up his meager results and wrote up a paper and shopped until he found a journal to accept it and was published in March of 2012. Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s ‘Retroactive Facilitation of Recall’ Effect.  This failure to replicate got a great deal of press, as noted in the comment section of that paper. However there were six studies that were pre-registered, not three.  Bem provided a detailed comment to Wiseman’s paper, which he summed up in this Skeptiko interview:

“What Wiseman never tells people is in Ritchie, Wiseman and French is that his online registry where he asked everyone to register, first of all he provided a deadline date. I don’t know of any serious researcher working on their own stuff who is going to drop everything and immediately do a replication… anyway, he and Ritchie and French published these three studies. Well, they knew that there were three other studies that had been submitted and completed and two of the three showed statistically significant results replicating my results. But you don’t know that from reading his article. That borders on dishonesty.”

As of 2012, as far as the mainstream press knew, Bem’s experiments had started out promisingly, but no one had been able to replicate them.

And that’s where things have pretty much stayed for the past few years. We are now up to speed on the Bem “feeling the future” experiment.

In March of 2017, this flared up again, as an article in the Chronicle of Higher Education did a hatchet job on Bem, which forced him to respond.  They wrote:

This isn’t the first time Cornell has had to cope with a blow to its research reputation. In 2011, Daryl Bem, an emeritus professor of psychology, published a paper in which he showed, or seemed to show, that subjects could anticipate pornographic images before they appeared on a computer screen. If true, Bem’s finding would upend what we understand about the nature of time and causation. It would be a big deal. That paper, “Feeling the Future,” was widely ridiculed and failed to replicate, though Bem himself has stood by his results.

It’s how the system works. Wiseman played to everyone’s desire to stay with the status quo and they ran with it, ignoring all the problems with Wiseman’s replications. Bem responded:

Bartlett asserts that my experiments failed to replicate. He is incorrect: In 2015, three colleagues and I published a follow-up meta-analysis of 90 such experiments conducted by 33 laboratories in 14 countries. The results strongly support my original findings. In particular, the independent replications are robust and highly significant statistically.

And here is the meta analysis: Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events

Positive results and rebuttals to criticisms are roundly ignored in place of the usual hash of skeptical misinformation. Gah, I get tired of this.