Came across this site that exposes audio snakeoil myths and regularly challenges the promulgators to a perfectly legitimate $1M prize if they can prove their claim. Oddly, no one has claimed the prize yet. Regular mention of ABX testing, so this site has done its homework.

http://www.randi.org

and a recent challenge (navigate to "A FIRM OFFER")...

http://www.randi.org/jr/080504string.html#8

Use search engine with terms such as "audio, snakeoil, etc". Informative and enjoyable stuff with a regular newsletter...

I've been reading James Randi's site for years, long before I found HA.org even. Good stuff - wish more people were as rational as he is.

There was a thread about this on rec.audio.high-end with lots of very interesting comments by industry heavyweights, including John Atkinson (Stereophile) and Tom Nousaine (Audio Critic).

John's main complaints are that a) Randi never contacted him or Stereophile, and he got a number of wrong names/terms in the prize offer, and b) it's a publicity stunt which Stereophile should not stoop to the level of responding. Both of which are ad hominem arguments, although I'm not sure if any man of the trade would respond any differently than him.

I think the only way the hifi industry will ever seriously be challenged about this is if the AES starts rolling heads. A LOT of high-end manufacturers have membership, and a public statement claiming fraudulence on the part of the hifi industry is a pretty sobering thought. This wouldn't faze the true believers like PWB or possibly Stereophile, but it would go very far in educating the general public.

I personally don't put much credit in tests that failed under Randi's supervision. If the test succeeds, the Randi foundation loses one million dollars. Thus James Randi's interest is to make it fail at all cost, even if it must cost him 500,000 $ to do so !
Let's remember also that James Randi is a professional illusionist. There would be nothing easier for him than to trick people with false results, replacing one list of values with another.
Thus I'd be more interested to see his tests supervised by an independant third party, that would itself be a professional magician. I think that contestants for the prize would have better do like this anyway.

The way Randi's million dollar challenge works is not quite the way you think. As it says on his own page:


In other words, you make your application and state what wacky thing it is you can claim to do. Then they help to define the test, with your approval. If you can pass, you win. The real key here is that for most of the claims made that are questionable, the people making the claims failed to apply at all. If you're not even willing to try to define a valid scientific test for your claim with the JREF, that pretty much sums it up, I think. I mean, if you really can tell a difference, and can prove it in a rational scientific manner, then wouldn't you give it a shot? If the JREF made unreasonable conditions to win the prize, you could publicize the fact. The fact that this has never occurred sorta points to something.

Randi is basically a professional sceptic and debunker. I doubt he'd pull a fraud when that could cost him quite a bit more than a million bucks, in the long run.

If this is a fraud, he will never pay a cent. If he is honest, it might cost him a few millions dollars in the long run.
There is one example of experiment whose result was scientifically published (in Nature), then tested under supervision of Randi, and failed : the famous experiment of J Benveniste : http://www.bbc.co.uk/science/horizon/2002/...athytrans.shtml

I don't accuse James Randi of frauding, because there is no evidence whatsoever that it is the case. My point is that financial bias can be even stronger than expectation bias, and that from my point of view, an experiment setup to win or loose 1,000,000 $ for the contestants is not an objective one. What's the point in removing expectation bias from an analysis, introducing scientific methods, if you introduce another bias at the same time ?