What's the problem with double-blind testing? Options

[mirrorsawlljk]

I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable. I'm afraid I don't understand his angle of attack. I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test.

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing. Is there an audio magazine that isn't filled with this kind of thinking?

[duff]

Seems to be a lack of understanding of something fundamental here: if you encode something to a lossy format it *does* lose information, it's not "compressed" at all. You've actually thrown away a lot of information! How do you know that the auditory system doesn't find it *easier* to decode since there's less "data"?

Lossless and lossy compression schemes are just that, compression. It is in this sense that I mean that an mp3 file, for example, is compressed..

The question of whether perceptual systems have an easier or harder time resolving stimuli that are impoverished (relative to a complementary uncompressed stimulus) is an empirical one that has been addressed in a variety of ways in perception research (mostly vision). I've been talking about ways to address this specifically with audio lossy schemes, and I'm presenting an idea regarding how this effect might not be addressed fully with an ABX paradigm that relies on decision tasks. A reaction time paradigm might be better suited to study the processing differences listeners likely experience.

The reason compressed data requires more effort on the part of whatever computational system is perceiving it, is because more inferential processing is required. If you are missing data in a signal, some mechanism in your brain must essentially interpolate, from the signal, the missing information.

[stephanV]

The reason compressed data requires more effort on the part of whatever computational system is perceiving it, is because more inferential processing is required. If you are missing data in a signal, some mechanism in your brain must essentially interpolate, from the signal, the missing information.

Why? How can our brain even tell there is something missing? And why MUST it interpolate this data?

These are again merely assumptions on your part.

This post has been edited by stephanV: Oct 21 2005, 17:33

[duff]

The reason compressed data requires more effort on the part of whatever computational system is perceiving it, is because more inferential processing is required. If you are missing data in a signal, some mechanism in your brain must essentially interpolate, from the signal, the missing information.

Why? How can our brain even tell there is something missing? And why MUST it interpolate this data?

These are again merely assumptions on your part.

It's not that our brains "know" there is something missing, it's that relative to a less noisy signal, the brain has to do more work in order to generate the best representation of the sound it can.

I will post studies that support a number of things I have claimed, including quite notably the neural dissociation of sensory and decision processes. The "assumptions" I'm making are rooted in my training as a cognitive psychologist who studies speech processing. Many of the things I've said are well supported in experimental studies...I'm not just making it up. Also, what I'm saying is testable. I'm just trying to explain the logic of why I believe a different methodology might be necessary to rescue at least some of the claims made by audiophiles. I'm not an audiophile.

Why does it not necessarily apply? Because the purpose of lossy audio compression is to throw away unperceivable parts of the audio. So if the sensors aren't missing anything, then why should it make any difference?

Again, the main point here is that fatigue effects caused by processing issues (not accessible to systems that verbally report differences in ABX listening tests) could have residual effects on listeners in more long term ways. Reaction time experiments could verify this processing claim I'm making, and a positive result (where an ABX test shows nothing) could indicate a relationship between perceptual processing and vague ideas of discomfort experienced by some listeners of compressed audio.

This post has been edited by duff: Oct 21 2005, 20:19

[stephanV]

Why? How can our brain even tell there is something missing? And why MUST it interpolate this data?

These are again merely assumptions on your part.

It's not that our brains "know" there is something missing, it's that relative to a less noisy signal, the brain has to do more work in order to generate the best representation of the sound it can.

Now you are saying something completely different. I don't see how our brain would try to represent anything else than what it is perceiving. I don't see how a wave form produced by an mp3 decoder is conceptually different from one might encounter in other situations. But ok, I will wait for studies.

This post has been edited by stephanV: Oct 21 2005, 20:24