What we measure is what we hear Options

[Yahzi]

I've heard audiophiles claim time and time again that what we measure does not reflect what we hear in a listening evaluation. I think Floyd Tool is the pioneer in conducting the studies for this, but who else as in a position of authority has led research into this field? It can't be mere opinion. I feel measurements largely do correlate with subjective sound quality but I don't think *all* measurements do. Some are more relevant than others.

What is your position?

[slks]

It depends on which measurements we're talking about. A volume change of 6 dB is certainly noticeable, while increasing top-end frequency response from 20 to 30 khz is almost certainly not noticeable.

I don't know of any prominent researchers in this area, but it's fairly simple to do your own studies with the wide variety of audio tools out there, and an ABX test. The only thing with doing it yourself is that you need to make sure you use the proper methodology, or your results will be invalid.

This post has been edited by slks: Dec 24 2012, 18:00

[Yahzi]

So you are saying that measurements can explain what we hear assuming you are using the correct measurements, but not all measurements are appropriate for the task so it depends what exactly you are measuring for. Am I understanding this correctly?

This post has been edited by Yahzi: Dec 24 2012, 18:23

[Ethan Winer]

^^^ Yes, you understand correctly. I've done a lot of such testing using commonly available audio tools, and several articles on my web site explain the method and offer files you can download to assess audibility for yourself:

Converter Loop-Back Tests
Artifact Audibility Report
The Truth About Record Levels
Hearing Below the Noise Floor
Dither Report
Converter Comparison
Perception - the Final Frontier

--Ethan

[AndyH-ha]

To sum of one aspect of "what we measure is NOT what we hear"
Some aspects that can be measured, such as distortion, noise level, and frequency response, make a very significant contribution to what we hear -- up to a point. Beyond that point, improvements in the equipment of an order of magnitude, or two orders of magnitude, might be possible and measurable, but those improvements make no difference in what we hear. They are beyond human abilities to differentiate. Also, if the same level of improvement is not maintained throughout the signal chain, from the source to the output transducers and listening environment, they can be readily swamped, washed out, by those parts of the signal chain that can't meet the same standards.

[Arnold B. Kruege...]

I've heard audiophiles claim time and time again that what we measure does not reflect what we hear in a listening evaluation. I think Floyd Tool is the pioneer in conducting the studies for this, but who else as in a position of authority has led research into this field? It can't be mere opinion. I feel measurements largely do correlate with subjective sound quality but I don't think *all* measurements do. Some are more relevant than others.

What is your position?

My position is that if there is an audible change in an audio signal then there is a readily measurable change in that signal. But, just because there is a measurable change, there is no guarantee of an audible change. This is because modern audio measurements are far more sensitive than the human ear.

The idea that there are audible changes for which there is no corresponding measurable change has been tested carefully for over 30 years and it has been found that the idea that this is so is a result of people not doing proper bias controlled listening tests.

IOW, human bias can lead to the perception of an audible change for which there is no physical explanation.

[Porcus]

Two totally different points:

(1) Those annoying spectrogram-comparisons of a lossy version against the original. Easy to quantify huge differences which are not heard, or at least not heard anywhere near what the measurements indicate. On the other hand, could hide audible artifacts.

What you measure is not what you hear – unless you find a much smarter algorithm. And such one has already been applied:

(2) Training a psy model. Correlating to actual listening tests. Predicting what actual humans hear. Actually implement it in a lossy codec. It measures what we actually hear; we hear something which sounds quite a bit like [lossy version].

The “problem” about the latter one, is that it does not directly return to you any “difference is this big” measure. That is not what it is designed for; it is designed to return the lossy signal.

[Woodinville]

Many people have worked on measuring what you hear. No perceptual coder (MP3, AAC, AC3, etc) would work if many people had not only done that, but also done a good job of it.

Floyd has done some very good work on preference, which is a different (but good) thing. Work on this goes all the way back to the 1920's, articulation, critical bands, etc, of course.

Look at www.aes.org/sections/pnw/ppt.htm for the "heyser lecture" slide deck for some discussion on that subject.

[jkauff]

Many people have worked on measuring what you hear. No perceptual coder (MP3, AAC, AC3, etc) would work if many people had not only done that, but also done a good job of it.

My own experience makes me wonder how well human hearing has been measured. I'm sure average human hearing has been well-studied, but how many outliers are there in the world population? My daughter, for example, can hear the high frequency tones used by cat-repelling devices. She's demonstrated this many times, but my wife and I can't hear them at all (nor can most people). I wonder if the hearing range of various human populations has been well-studied (such as African tribes with certain members who can hear low frequency tones emitted by elephants).

Is it fair to say that this good work has been done primarily based on averages of 1st World populations?

This post has been edited by jkauff: Dec 29 2012, 15:21

[dhromed]

Your daughter is not an unmeasured outlier; she's just young.

[greynol]

I doubt she is as well. What frequency is the cat repelling device?

This post has been edited by greynol: Dec 29 2012, 15:31

[garym]

Your daughter is not an unmeasured outlier; she's just young.

Yes of course. Recall the "ring tones" that only kids in school could hear but none of the teachers could hear it. More here:

http://www.nytimes.com/2006/06/12/technology/12ring.html

[greynol]

We hear about people in their early 20s and younger who can hear up to 22kHz every once in a while.

Pure tone testing doesn't really tell the story, however.

[sld]

Your daughter is not an unmeasured outlier; she's just young.

Yes of course. Recall the "ring tones" that only kids in school could hear but none of the teachers could hear it. More here:

http://www.nytimes.com/2006/06/12/technology/12ring.html

Or the udial.wav that used to circulate around here to test for resampling aliasing...

[Mach-X]

Ouch.

[Yahzi]

Correct me if I'm wrong but we hear using our brains, not our ears?

[Soap]

Correct me if I'm wrong but we hear using our brains, not our ears?

Your point?

If you're addressing one of the dozens of comments how about quoting the relevant one for some context?

Your brain can't hear more than what the ears are physically capable of processing, and demonstrably hears less.

[Yahzi]

I just thought I'd ask the question and I thought instead of starting a new thread I might as well ask in this one. Besides the "we can't measure what we can hear" arguments there are also arguments that our ears are instruments and are the only things that we use in a listening evaluation. Hence the question I asked - "we hear using our brains, not our ears?" Given all the systematic biases that plague human hearing, sight affecting sound .. mood, just thought I might as well get reassurance on that point.

[Speedskater]

If you restrict the use of the words "hear" or "see" to only the processes inside the brains of humans and animals, you have eliminated about 99.99 % of the common usage of these words.

[jkauff]

I doubt she is as well. What frequency is the cat repelling device?

I don't know. Devices differ--some use ultrasound, some don't. Cat hearing range, however, is 45-64,000 Hz.

[Soap]

Cat hearing range, however, is 45-64,000 Hz.

Is it fair to say that this good work has been done primarily based on averages of 1st world cat populations?

[pisymbol]

I've heard audiophiles claim time and time again that what we measure does not reflect what we hear in a listening evaluation. I think Floyd Tool is the pioneer in conducting the studies for this, but who else as in a position of authority has led research into this field? It can't be mere opinion. I feel measurements largely do correlate with subjective sound quality but I don't think *all* measurements do. Some are more relevant than others.

What is your position?

I am more of a lurker on this board but I would like to contribute the following:

The problem with measurements is well the metrics! Performance is only relevant when you define your metrics. And here in lies the rub:

Without sounding too coy is that as far as I am aware of there are no measurements that describe "hearing" or rather more ambiguously "sound in which your brain interprets it." So truth be told, the audiophile is partially correct, citing THD, FR, spectrum analysis, etc. etc. all do not tell me a damn on how I actually perceive. They sure do give some obvious indicators and are invaluable tools to determine audibility in the first place (and what isn't audible but placebo). This is why many of the adaptive predictive models had to be fine tuned and then *listener* verified via sophisticated DBTs (e.g. MP3).

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

This post has been edited by pisymbol: Dec 31 2012, 02:08

[AndyH-ha]

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

Not really, perhaps depending somewhat on what you mean by "hear." These are often interpretations of what their expectations and beliefs create inside their heads. They very often do not "hear" the same thing if they don't know which piece of equipment they are listening to and they will "hear" what they think they are listening to even when they are actually listening to something else.

[pisymbol]

But frankly, audiophiles reviews and impressions still contain intrinsic value even if most of it "can't be trusted" since they are interpretations on how they "hear" which as far as I know is how everyone on this thread hears too...

Not really, perhaps depending somewhat on what you mean by "hear." These are often interpretations of what their expectations and beliefs create inside their heads. They very often do not "hear" the same thing if they don't know which piece of equipment they are listening to and they will "hear" what they think they are listening to even when they are actually listening to something else.

Subjectivity can still provide valuable data points (especially negative impressions).

[db1989]

Subjectivity can still provide valuable data points (especially negative impressions).

Please explain how subjective impressions, negative or positive, are of any use to people who are interested in valid, i.e. objective data about perception.