What we measure is what we hear Options

[AndyH-ha]

Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)

[Woodinville]

Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)

A properly designed and operated subjective test IS a form of measurement.

Audiophile tests generally (almost always) need not apply, of course.

But, a subjective test, run as a DBT, using a proper design, with anchors and controls, is a measurement. A measurement that is a true pain in the behind to do, perhaps, but it IS a measurement, and it most often even gives a useful noise level for the measurement.

[greynol]

TL;DR:
MUSHRA and other ABC-style tests are subjective tests. This is because the person taking the test assigns a score to each stimuli in order to declare a personal preference.

[Satellite_6]

So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.

[Martel]

What's the "best measuring" frequency response of a headphone? I wonder...

[Nessuno]

So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.

AKA "expectation bias"... it rarely misses a shot!

P.S. What you said is not wrong in principle, but fact is that human ear is not linear neither has infinite precision...

This post has been edited by Nessuno: Dec 31 2012, 16:20

[Soap]

What's the "best measuring" frequency response of a headphone? I wonder...

The one which best matches your ear shape, of course!

[Kees de Visser]

A properly designed and operated subjective test IS a form of measurement.

I thought the food industry is doing this all the time:
http://www.ift.org/

Sensory evaluation has been defined as "a scientific discipline used to evoke, measure, analyze, and interpret reactions to those characteristics of foods and materials as they are perceived by the senses of sight, smell, taste, touch, and hearing" (IFT, 1975).

Perhaps we should be less scared to be subjective. All humans "suffer" from it. The real art is in finding out which variables are responsible.

Happy new (multisensory) year !

[greynol]

I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.

[Arnold B. Kruege...]

Many people have worked on measuring what you hear. No perceptual coder (MP3, AAC, AC3, etc) would work if many people had not only done that, but also done a good job of it.

My own experience makes me wonder how well human hearing has been measured. I'm sure average human hearing has been well-studied, but how many outliers are there in the world population? My daughter, for example, can hear the high frequency tones used by cat-repelling devices. She's demonstrated this many times, but my wife and I can't hear them at all (nor can most people).

Since you haven't given the actual operating frequency of this device, your claim lacks global meaning.

It is well known that children generally have better and more extended HF hearing than adults. Human ears have geometry that determines how they work and when they grow, the geometry shifts towards lower frequencies.

I wonder if the hearing range of various human populations has been well-studied (such as African tribes with certain members who can hear low frequency tones emitted by elephants).

We don't need to jump down every rabbit hole to know where to find rabbits every time we need to. ;-)

Perhaps one of the most cogent comments on this thread is from JJ, who makes the point that since lossy coders provide substantial amounts of data reduction with minimal sound quality loss all things considered, and they work based on what we know about human hearing, we must know quite a bit of useful information about human hearing.

Is it fair to say that this good work has been done primarily based on averages of 1st World populations?

More to the point there has been no need to develop special lossy encoders for every country, continent or people group. This is particularly true when good unbiased listening tests are used to determine what people are and are not hearing.

[Ethan Winer]

Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

[Arnold B. Kruege...]

Subjectivity can still provide valuable data points (especially negative impressions).

The above is like saying that radioactivity can lead to thermonuclear explosions.

Subjectivity can provide valuable data points, but the frequency of incidence of this happening can be disappointingly low.

Consider all of the high end audiophile publications. Thousands or tens of thousands of pages of subjective impressions per week or month that would simply cease to exist if proper listening test protocols were used.

[Nessuno]

Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.
What can really change and by large amounts are expectations in sighted tests, when the tester knows what he's listening to and could link to the listened device side effects of other precedent or contemporary experiences, different knowledge, true or supposed, even the mood of the moment...

This post has been edited by Nessuno: Dec 31 2012, 19:18

[greynol]

Maybe we should take a close look at the meaning of the word perception. From where I stand, and from having an opinion shaped by what I have read by many of the fine members here and elsewhere with demonstrated expertise in the area, I will add that expectation bias does in fact alter our the perception of what we hear. The suggestion of a difference in X (breath/openness/dancability/PRAT but also non-BS qualities) can result in the actual and very real perception of that difference in X to the individual, even when the stimulus is held constant.

So no, changing perception does not have to affect DBT results necessarily. In the case of having placebophiles with strong expectation bias we often see the reverse: DBT affects the perception, or maybe not until the results of the test are revealed. That's still no guarantee since some people think that getting 13 right out of 20 means they passed the test. Still others will continue to cling to their faith in fairy dust, especially when the perceived differences once again return when the constraint of a DBT has been lifted.

This post has been edited by greynol: Dec 31 2012, 22:09

[jkauff]

Next, on to wine tasting....

[greynol]

Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum.

This post has been edited by greynol: Dec 31 2012, 22:14

[Woodinville]

I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.

Amen to that. Sorry, but it had to be said twice.

[Woodinville]

What's the "best measuring" frequency response of a headphone? I wonder...

On whose head, with what haircut, under what circumstances, with what ear canal volume and shape?

Martel has a real point here.

[Woodinville]

This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.

To some extent that's true.

Even something so simple as measuring one's hearing can be very different on day 1 vs. day 3 of getting off of the plane, or if you spent 8 hours on the show floor vs. ran through it with earplugs.

But this kind of thing can be managed, and that's also what controls are for.

[Woodinville]

Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum.

Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit.

[greynol]

You're right, I just don't know that we should drop comparing/correlating audio measurements and actual hearing which we hardly touched upon in order to move onto expectation bias in wine tasting. If anyone should be suggesting the direction of the discussion it should be the original poster.

[Nessuno]

Maybe a little OT, but I recall a "rule of thumb" more than a woman told me they follow when they buy perfumes is they don't evaluate more than three or even two different fragrances a time, or they would not be able to discern real differences and end up taking home something they could even not like any more days after.
I guess this has everything to do with temporal masking and even perception system fatigue.

P.S. Happy new year full of good music to all HA fellows!

[Arnold B. Kruege...]

Feel free to start a new topic, replacing the word hear in the subject line with the word taste. Be sure to post it in the off-topic forum.

Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit.

I'm reminded of Mielgard, et al classic book about Sensory Evaluation.

http://books.google.com/books?id=F_A-YtWXF3gC

[Ethan Winer]

but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.

There are several different issues. One is that our perception changes from moment to moment. Another is we can't focus on everything all at once when complex music is playing. Another is that some things really are so obvious that they won't be missed whether sighted or blind. Another is expectation affecting our opinion when we know which [device] we're hearing. Another is that moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones). Another is that our sensitivity changes at, as you said, borderline conditions. A real change may be noticed one time but not another. Especially for an old man like me.

--Ethan

This post has been edited by Ethan Winer: Jan 1 2013, 19:10

[Martel]

... moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones)

Almost all headphones "suffer" from a similar problem - seating them on/in one's head/ear (or any headphone measurement device). Two people are not very likely to position the same headphone exactly the same way on their heads due to different head/ear shapes, preferences etc. This is going to change the perceived/measured sound and introduce additional variable(s) into the mix.