What we measure is what we hear Options

[Yahzi]

I've heard audiophiles claim time and time again that what we measure does not reflect what we hear in a listening evaluation. I think Floyd Tool is the pioneer in conducting the studies for this, but who else as in a position of authority has led research into this field? It can't be mere opinion. I feel measurements largely do correlate with subjective sound quality but I don't think *all* measurements do. Some are more relevant than others.

What is your position?

[pisymbol]

I've heard audiophiles claim time and time again that what we measure does not reflect what we hear in a listening evaluation. I think Floyd Tool is the pioneer in conducting the studies for this, but who else as in a position of authority has led research into this field? It can't be mere opinion. I feel measurements largely do correlate with subjective sound quality but I don't think *all* measurements do. Some are more relevant than others.

What is your position?

I am more of a lurker on this board but I would like to contribute the following:

The problem with measurements is well the metrics! Performance is only relevant when you define your metrics. And here in lies the rub:

Without sounding too coy is that as far as I am aware of there are no measurements that describe "hearing" or rather more ambiguously "sound in which your brain interprets it." So truth be told, the audiophile is partially correct, citing THD, FR, spectrum analysis, etc. etc. all do not tell me a damn on how I actually perceive. They sure do give some obvious indicators and are invaluable tools to determine audibility in the first place (and what isn't audible but placebo). This is why many of the adaptive predictive models had to be fine tuned and then *listener* verified via sophisticated DBTs (e.g. MP3).

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

This post has been edited by pisymbol: Dec 31 2012, 02:08

[AndyH-ha]

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

Not really, perhaps depending somewhat on what you mean by "hear." These are often interpretations of what their expectations and beliefs create inside their heads. They very often do not "hear" the same thing if they don't know which piece of equipment they are listening to and they will "hear" what they think they are listening to even when they are actually listening to something else.

[pisymbol]

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

Not really, perhaps depending somewhat on what you mean by "hear." These are often interpretations of what their expectations and beliefs create inside their heads. They very often do not "hear" the same thing if they don't know which piece of equipment they are listening to and they will "hear" what they think they are listening to even when they are actually listening to something else.

Subjectivity can still provide valuable data points (especially negative impressions).

[Ethan Winer]

Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

[Nessuno]

Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.
What can really change and by large amounts are expectations in sighted tests, when the tester knows what he's listening to and could link to the listened device side effects of other precedent or contemporary experiences, different knowledge, true or supposed, even the mood of the moment...

This post has been edited by Nessuno: Dec 31 2012, 19:18

[pdq]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

[Nessuno]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

A failed ABX proves nothing with respect to the codec, or better it doesn't prove that there is no difference between source and encoded sample, but proves that should a difference exists, the individual is not able to discern it, that is the sample is transparent to him.

[pdq]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

A failed ABX proves nothing with respect to the codec, or better it doesn't prove that there is no difference between source and encoded sample, but proves that should a difference exists, the individual is not able to discern it, that is the sample is transparent to him.

I agree with the first part of your statement. However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not. The failure could have been due to reasons other than transparency or lack thereof.