What's the problem with double-blind testing? Options

[mirrorsawlljk]

I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable. I'm afraid I don't understand his angle of attack. I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test.

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing. Is there an audio magazine that isn't filled with this kind of thinking?

[krabapple]

I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable.  I'm afraid I don't understand his angle of attack.  I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test. 

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing.  Is there an audio magazine that isn't filled with this kind of thinking?

By all means read it..it's always good for a laugh. I have the current issue at home and am waiting for a free hour or two to enjoy it.

I've interacted with editor John Atkinson online and very briefly in person -- he's mastered the art of balancing himself on a rehtorical tightrope between scientific evaluation of claims (he has a background in physics) and audiofoolery. This manifests itself in his magazine as the schizophrenic devotion to engineering jargon and inclusion of bench measurements -- sometimes extremely detailed -- accompanying sighted 'reviews' that are literally whimsical. Atkinson knows who his subscriber base is and I doubt he's about to alienate them by embracing valid scientific methods for subjective comparison...ever. The purely 'subjective' experience will ALWAYS trump ANY measurement or scientific method, to that crowd.

But Stereophile is a model of restraint compared to The Absolute Sound, which is absolutely off the deep end, and has been forever.

Peter Aczel's 'The Audio Critic' (now online only) is the closest thing to a purely objective audio magazine you'll find. (www.theaudiocritic.com). Lesser lights include The Sensible Sound (about half of the people there seem to drink the hi-end cool-aid) and Sound & Vision (David Ranada is a DBT advocate, Ian MAsters is a no-nonsense objectivist, Ken Pohlmann is a god of digital, but the mag as a whole seems ever more tilted towards marketing hype -- I don't see much by Ranada in there recently)

This post has been edited by krabapple: Oct 19 2005, 20:15