Can You Win Randi’s Million Dollar Challenge?

Related Post:  A Critical Look at Randi’s Million Dollar Challenge

Update:  11-30-11:  There seems to be some confusion over the length of time Pavel spent dealing with the challenge.  Skeptics are criticizing this article because they start the date from mid 2008, when Pavel’s protocol was accepted, which would put his time spent at less than a year and a half.   I calculated from when he submitted his first application.  The skeptics are not counting the time it took to establish the protocol, only the time after the acceptance of the protocol.  I can certainly understand this reasoning and I think that this version of events belongs in the discussion.  If anyone has the date when the protocol was accepted, I will include it here.  Hat tip to John Mather.

Update 6-27-11:  Minor error:  The challenge is not administered on the general forum.  There is a special forum specifically for administering the challenge and this is what I am referring to in this post.  Hat tip to Calde Silk.

Do you have what it takes?  Can you win the Million Dollar Challenge by demonstrating YOUR psychic ability?  Before you rush over to the JREF (James Randi Educational Foundation) site and submit your application, you might want to first have a look at the experience of someone else who tried to take the challenge.  In this case I have gone through the forum postings for Pavel Zibarov, who first submitted his application around May of 2007.  On August 20th, 2009, Over two years and almost 900 forum posts and innumerable e-mails and letters later, he finally withdrew from the challenge when faced with an  arbitrary, unexplained and unfathomable ultimatum from Randi just as the protocol was getting settled. It made the preliminary testing a complete waste of time.

Pavel’s journey down the JREF rabbit hole sheds light on what has always been a somewhat enigmatic challenge; we know that many people apply, but beyond that, very little is known about the process itself.  A lot of people make assumptions about the challenge, many of which are incorrect.  The forum postings though, do illuminate it all somewhat.

Investigating through the forums

I should mention here that the forum postings are the only way to track an applicant’s progress.  The challenge is run by volunteers and it is done very much on the cheap.  All the information is through forums.  It is clear from reading the forum posts that structurally, the challenge process is a completely disorganized mess.  The applicant deals primarily with the volunteers, except when he/she is dealing with the staff, who apparently don’t always notify the volunteers about what they’re doing.  (They sometimes find out what has transpired from the applicant.)  Randi can simply swoop in at any moment and change whatever he wishes without notifying anyone else or justifying his decision.  As far as I could tell, the volunteers are left to fend for themselves and have no authority to move an applicant forward no matter how much work they’ve done with that individual.  One of the signs that this is a really badly run challenge is that very, very few applicants ever get to the testing stage.  Isn’t that the object of the challenge?

The website is not set up to provide clear, easy to follow status reports on the applicants.  As I said, you have to slog through forum posts to find out anything.  This is especially troublesome given the often long lag times between responses from JREF staff.  It would be hard to bring a new person up to speed quickly on an applicant the way it’s set up.  The process sometimes leaves people frustrated such as this encounter where a JREF muckety muck suddenly jumps in the middle of the Zibarov application forum with this bombshell:  Here’s the link: ( This discussion begins at the bottom of the page)

Folks, I’m going to step in here.

This protocol has been going on forever. It seems like Pavel keeps trying things and then failing, and people are trying to modify the test so that he has a chance of winning. This is NOT what the challenge is about. The claimant should have a clear claim and confidence that he can accomplish what he claims before he applies.

RemieV recently went over a protocol with me that looked like it might be workable, and then I see here that ANOTHER one has been proposed and Pavel likes it. Now we’re essentially back to square one.

It is now incumbent upon Pavel, who I admit seems sincere, to state what he can do and allow us to test it. It seems that he applied way before he was ready, and now the JREF is expending too many resources to try to make something work.

There are other applicants waiting.

The JREF will accept one more full protocol from Pavel (with Startz’s kind help, if he’s willing) and then we will move on to another candidate.

Startz (JREF volunteer) responds:

Jeff:

I am willing to help.

In fairness to Pavel, he has presented statistically sound protocols. JREF has been rather unresponsive as to what objections they have so that Pavel can revise them in accord with JREF’s wishes.

Let me be more pointed than, as a fan of JREF, I wish were necessary. JREF has asked for communications to be done by email. When I have done as JREF has asked, JREF has not had the courtesy to return emails. If JREF were one of my PhD students, rather than an organization with a long, successful track record, I would say this in a less pleasant way.

Remie has sensibly pointed out that negotiations are better done by email than through this public forum. Following this wise advice, I have (on Pavel’s behalf) sent in protocols by email (while posting informational copies to the forum). JREF’s responses have been through the forum. There is no reason this could not have been settled in a week of back-and-forth email messages. Nearly all the delay time has been on JREF’s end, not Pavel’s

PZ (Pavel Zibarov, applicant) responds:

with all my respect to everyone.. the same was when I was trying to negotiate, sent propositions, asked questions regarding protocol, requirements etc… and there was not much result, first of all it is a weeks for a reply and second.. I have always answered every question that was asked or proposed by JREF (Remiev) and in my tern also posted question..and there was no reply…and as I have also written before, the replies would be really helpful in moving on with things.., yes JREF is busy, but to find out after weeks of waiting that the protocol is can not be accepted cause it is too complicated, and when I have asked like, why? what is the problem, what would be good enough etc.. nothing… so weeks was wasted.. and I was waiting in order not to waist myself and time of the others with the other protocols etc.. than have started to propose the other options hoping that one of them would suite.. again not knowing exactly what it should be suited in..a prat of time frame ( though I never asked for over 8 hours test) or the 1.000 odds to be covered for the first test.

One thing everyone agrees on is that the JREF staff is painfully slow to respond to correspondence and this is the chief reason that this application dragged out for over two years.  PZ is basically being blamed for a problem essentially created by JREF.

Keeping in mind that this challenge has been running for almost 10 years, it’s certainly odd to see that the process is so difficult and slow.

Setting Up a Protocol

After the applicant has fulfilled the basic requirements for submitting an application, which includes a media presence and a letter from an academic authority, the application is still not accepted and this is where the volunteers come in.  It is their job to get a workable protocol from the applicant.  This is where the system completely breaks down and only the tiniest percentage of applicants makes it through to testing.  It’s not clear what that percentage is.  In fact, this information seems to happen off line.  All that can be discerned from the forums is that the vast majority drop out.

A quick perusal through the applicants demonstrates a fairly predictable mix of psychic abilities, (clairvoyance, medical diagnosis, telekinesis, etc.) with some strange stuff thrown in.  Most of these could be quickly and easily handled by developing standardized tests for those types of abilities since they fall into fairly obvious categories.  It’s not like this has never been done before; there is plenty of literature available to show how best to conduct such experiments.  It would certainly relieve the volunteers of a great deal of effort; but alas, they have to re-invent the wheel with every new applicant.

By forcing all the applicants to make a specific claim and set up a protocol, JREF is making the process as difficult as possible for people who have no experience in this area.  Psychics are not scientists and most have experience in the arts or other creative endeavors.  Speaking of which, the volunteers do not understand how to deal with the psychic personality.  It is truly the blind leading the blind.  On the one hand are the psychics, who excel in non linear thinking, as evidenced by the often rambling sentences they put together.  On the other hand are the JREF volunteers who are completely unprepared psychologically to handle the challenges of dealing with psi and psychics.  I suspect that many applicants wash out because of these basic psychological differences.  Perhaps they expect to be encouraged and supported in their attempt to win the prize?  If so, they are in for a rude awakening.  Witness this exchange between PZ and a JREF forum poster: (begins post #113)

GK

We have heard this statement – sometimes in a different phrasing – many, many times in this forum, pavel_do.

How many failures in controlled tests would it take for you to admit you do not have the claimed ability?

Five?
Ten?
100?
1000?
Googoolplex?

PZ

well as i said the time will show us.. maybe there will be no need for 2nd test.. I mean ill pass it first time.. IF ill fail ill try ones again.. depamd on the results and maybe mistakes.. To avoid it ill do my best to be ready 100%..

jj

Pavel, you have just said that:

• If you pass the test, your powers are proven.
• If you fail the test, you’ll try once again.

Now, I’ll add a few more lines to this:

• If you fail the test again, you’ll try once again.
• If you fail the test for the third time, you’ll try once again.
• If you fail the test for the fourth time, you’ll try once again.
• etc….

And I suppose that, after a while, things would get to:

• If you fail the test for the Xth time, you’ll stop thinking you have paranormal abilities.

Now, Pavel, I’ll repeat GzuzKryzt’s question:

PZ

I have ability, and i will prove it, as long as test will be reasonable, i can see and perform results that will be more than expected odds etc.. but if thay will want from me to perform it 10 times in range or put me in glas box with 100 people around and some toehr things its obviesly can affect results.. (…)

P2

Pavel_do, how many failures in controlled tests would it take for you to admit you do not have the claimed ability? Just hypothetically, you know. Just if.

Thinning Out the Applicants

In this situation, Pavel is being goaded during negotiations.  As these will be the same people, or at least the same mindset of the people administering the test, (although it never came to that) it is hard to imagine that test being fair.  A test under these circumstances would almost certainly yield no result (i.e. failure).  In any case, psychics are for the most part sensitive, thin skinned people whose feelings are easily hurt and the excessive doubting and negative comments will be very disturbing for them.  There is no indication that I have seen that the skeptics understand that they are being insulting and or that they are aware of their complete lack of empathy.

I am sure that quite a few applicants are simply unprepared for a scientific test and rightly come to understand that they will very likely not perform as they claim under the scrutiny of skeptics. Still more will undoubtedly be put off by the exceedingly poor communication and response times from JREF staff.  (This is a fairly standard complaint from the volunteers and the applicants.)  In any event, the challenge is clearly not designed help people through the process and get as many tested as possible.

It is hard to tell from the forums exactly what is happening.  Emails between JREF and the applicant typically don’t get published.

Applicants are generally greeted with a “put up or shut up” attitude that most will find quite off putting.  It’s subtle at times, but it’s there.  As most have no idea how to create a workable protocol, they are basically at the mercy of the volunteers, who might know how to do a basic test, but do not understand the nuances of psi testing.  Without getting into the specifics of it, it can be a very touchy process if you want success at it.  By the way, not mentioned in the application are the requirements that the preliminary challenge be completed within eight hours and also that no other observers be present.  There is no outsider checking the work of the JREF testers.

Pavel Throws in the Towel

By his account, Pavel pushed very hard to move the process forward, adapting his protocol and simplifying it in order to move the testing forward.  Bear in mind, all this time, he is re-submitting his application with every new protocol.  It is a hoop that has never been streamlined out of the process and it allows JREF total control over the process by withholding a contractual agreement until Randi decides to go forward.  Until then, there is no agreement.  Despite two years of sincere efforts, JREF consistently refused this simple act of good faith.

As for Pavel, he and the volunteers worked out a very basic protocol which involved 100 trials with 50% odds with the objective being to reach odds against chance of 1,000 to 1.  That is actually kind of fair, although I would have used 200 trials just to be sure.  Psychic ability can be kind of streaky.

In the end, Randi changed the protocol and gave an ultimatum:  Just 20 trials to reach those odds.  It was a take it or leave it proposition with no room for negotiation.  Pavel refused and withdrew his application, wasting two years of not only his efforts, but those of the JREF volunteers.

One of the forum denizens, who apparently knows her way around scientific protocol had this to say about the change:

I think you have misunderstood (or at least, I hope you have). Your success rate will probably be expected to correspond to the same probability given for success in the other test. However, you are right to be concerned. Reducing the number of tests means that you need a much higher success rate to reach the same level of probability. And this success rate will be higher than the success rate you have actually claimed (through your own testing). It sets you up to fail even if your claim is true and accurate. Which would make the critics of James Randi and the Challenge right. This is very disappointing.  (…)

(segue to another post)

Let’s look at what has happened here. Pavel sent in a protocol which gave him a 50% chance of passing – not ideal, but some people may find it acceptable. Randi suggests cutting the number of trials. Now I doubt that Randi intends to change Pavel’s odds of winning to 1:17 instead of 1:1000 by staying with the same success rate, so it is likely that he will be asked to guess at least 18 correctly in order to pass the preliminary. If Pavel’s real success rate is 0.67, then the chance that he will achieve a success rate of 0.90 is only about 2%. You can see that Randi has the ability to ensure that claimants will fail based on dictating how long a Challenge test will last. [my bold italics]  Of course, I cannot speak to his motivations for doing so. I suspect (or rather hope) that it is done out of naivety.

Got it? First, the application will not be signed by Randi until he agrees to the challenge and he won’t agree to the challenge unless he’s sure that you’ll fail it.

I, for one, don’t believe that Randi does this out of naivety.  I hope that you have a clearer understanding of what you’re up against if you attempt this challenge.  But even if you make it through all of this and somehow manage to secure for yourself a genuine shot at the preliminary challenge, here is what you will face:

You will be alone in a room full of people who want you to fail.  Previous testing on psychic ability makes it very clear what will happen: you will fail to achieve significant results.  Psychic ability is greatly diminished in the presence of skeptics; this has been demonstrated in number of studies, most recently Wiseman and Schlitz, 1991.  A very important piece of parapsychology protocol that will be missing from your test is that you will not be made comfortable nor given the opportunity to feel optimistic about your success.  It will be all numbers and protocol with these people who will feel certain of your failure.

Conclusion

Even if you succeeded, you now have to go on to the actual million dollar challenge version and beat odds of 1,000,000 to one.  You will have to change the protocol because you will need a way to generate much higher odds against chance.  Endless repetition is also a detriment to psychic ability.

The ultimate outcome will be a colossal waste of your time combined with your humiliation.

My conclusion is this:   Can you win Randi’s challenge?  No, of course not.  No one can.  It has nothing to do with whether you have psychic ability; it’s just that the game is completely rigged.  You can find more useful ways to spend your time and money.