What's the problem with double-blind testing? Options

[mirrorsawlljk]

I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable. I'm afraid I don't understand his angle of attack. I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test.

I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing. Is there an audio magazine that isn't filled with this kind of thinking?

[duff]

So sorry for the long delay. I didn't forget about this thread....

I have compiled a number of articles that provide anybody here resources and empirical support for the various claims David and I have been making. I claimed that there is a distinction between sensory and decision processes. This recent Nature article provides empirical support for this. Cognitive scientists have understood this distinction for decades, but here is recent evidence demonstrating the neurological basis for it. This is a fundamental issue in our criticism of the validity of ABX testing as the last word on ultimate differences in the auditory processing of lossy audio. Acoustic differences that could matter for a listener’s overall experience might not be audible.

Here are two articles that show the difference between decision processes linked to the discrimination of stimuli versus sensory processes that are affected when there is no discrimination. There is clear enhanced brain activity that occurs during the presentation of noisier stimuli even when it cannot be distinguished from a less noisy counterpart.

Effects of Low-Pass Noise Masking on Auditory Event-Related Potentials to Speech
The Effects of Decreased Audibility Produced by High-Pass Noise

Woodinside claimed that no filtering processes happen after the ear. This is definitely incorrect, as filtering processes do happen in the auditory cortex (one of the many analogues between auditory and visual processing), but I’ll assume he meant that the relevant filtering for audio codecs involves processes that happen only in the ear. This is fine, as there’s no reason to distinguish peripheral (inner and outer ear) processes from central processes (auditory cortex) when talking about metabolic costs of neural effort. The neurons must work harder (wherever this happens) to resolve the signal and encode it for later processing. And this does not deal with the likely possibility that the resulting representations might differ depending on stimulus quality (not a necessary feature of our argument).

There has been quite a bit of skepticism regarding the claim that compressed or degraded stimuli require more effort to process. I could cite a list of papers a mile long showing reaction time increases as a function of stimulus complexity (which heavily imply that processing difficulty increases), but there is also work showing that neurons respond to increased attention demands. Additionally, there seems to be specialized neural systems for resolving degraded signals, which in turn contribute to learning. These processes require effort, and have associated metabolic costs. This is exactly the sort of thing we have been arguing.

[Woodinville]

Woodinside claimed that no filtering processes happen after the ear. This is definitely incorrect, as filtering processes do happen in the auditory cortex (one of the many analogues between auditory and visual processing), but I’ll assume he meant that the relevant filtering for audio codecs involves processes that happen only in the ear. This is fine, as there’s no reason to distinguish peripheral (inner and outer ear) processes from central processes (auditory cortex) when talking about metabolic costs of neural effort. The neurons must work harder (wherever this happens) to resolve the signal and encode it for later processing. And this does not deal with the likely possibility that the resulting representations might differ depending on stimulus quality (not a necessary feature of our argument).

I'm sorry, but you're simply misusing the term "filtering" from the point of view of a signal-processing person. You are not, I trust, suggesting that the "filtering" that goes on in the CNS results in anything like the outcome of a linear filter, are you?

Psychologists often take words with specific meanings and add new meanings. Personally, I wish they'd use a different word, but in any case, the spatial capture is not a filter in any sense that I would normally use the word. I might use a filter to do spatial capture, that's a different problem altogether.

What you are, once again, ignoring, is that if there is NO INFORMATION AVAILABLE about a given distortion at the auditory nerve, you're arguing for something paranormal to happen later.

The evidence is in for outer hair cells, they depolarize at higher levels. So, there is no more effort going on in the basilar membrane, either. So, missing information means "no stimulus", ergo if anything it's "less effort" in the periphery. I think it's safe to say that the "cochlear amplifier" mechanism has gone FORD. (That's "found on road dead" for non-'mercans)

There is simply no evidence, no mechanism, and no non-sighted listener responses that in any fashion support this contention of "extra effort". None. That's what this all boils down to. One one side we see a bunch of suppositions, hypotheses, and speculation, based on the idea that there must be something wrong with DBT's because they don't observe some things that people think exist.

You show two sets of evidence for noise WELL OVER THRESHOLD. You're using data from a completely different, and irrelevant, situation in order to make your argument.

On the other side we see a century of work that shows that humans are easily fooled into thinking things exist when they don't, especially through the auditory system.

That is really what I see on the table here, supposition vs. a century of evidence. If there is something here, some extraordinary evidence for its existance, that addresses the known flaws of human perception, has to come forth.

This post has been edited by Woodinville: Nov 10 2005, 20:24