What we measure is what we hear Options

[Yahzi]

It seems this discussion ain't over yet. Thanks to all those who replied to my original question, much appreciated. Please carry on with the interesting discussion... I'm learning as I go along and I appreciate the input.

[Nessuno]

Let alone those quite obvious issues which actually affect ear sensibility, most of which are temporary (except aging, of course ) and as Woodinville pointed out could be managed and taken care of in a controlled environment (and even at home: you'll never try to ABX a critical sample while having a cold or just coming back from a night spent on the dance floor! ), maybe, as said by Greynol, we should agree on the term perception, which by the way comes from latin to take (CAPERE) by mean of (PER).

Now, let me put my previous statement it in another, rather hyperbolical way: if someone spent the price of a sport car in a hi-end amplifier and this fact let him enjoy his music better than before, that's good. It nevertheless doesn't change the amount of information his ear takes from the air motion, even if the new amp measures "far better" than the one he had before and so something in the air motion has actually changed, because those changes are below the treshold of his ear perception, even he is in perfect healt, relaxed, in a good mood and willing to hear some improvements. If instead he had spent far less money in a new pair of speakers which measure "better but not that much" and so his expectations could be lower, still his perception threshold would have been actually triggered by the larger amount of changes a new transducer takes, even if his head was in the exact place relative to point sources etc etc...

Both times his brain has felt a change, but the first time was not a perceived one (not by mean of ears at least).

All that said, my vision of the issue at hand and interpretation of the terms used might as well be wrong because this is not my working field of interest so I don't read specific literature (and also I'm not a native english speaker).

[greynol]

The point I was making is that the ears (outer or inner) don't perceive; rather it is the conscious mind does the perceiving. Further, perceptions can be altered though expectation bias. People who are persuaded into "hearing" (or "not hearing") something will actually experience it, regardless of the stimulus to the ear. To them it is very real and is perceived as such.

[Woodinville]

The point I was making is that the ears (outer or inner) don't perceive; rather it is the conscious mind does the perceiving. Further, perceptions can be altered though expectation bias. People who are persuaded into "hearing" (or "not hearing") something will actually experience it, regardless of the stimulus to the ear. To them it is very real and is perceived as such.

This is really the whole point. Perception is very much influenced by everything, including your internal thoughts, and pure randomness.

In order to make useful measurements, one has to establish understanding in the person doing the sensory evaluation first. This is called "training" in some contexts.

[Nessuno]

The point I was making is that the ears (outer or inner) don't perceive; rather it is the conscious mind does the perceiving.

Ok, but "perceptual" codecs are focused on discarding exactly those stimula that the ears (and/or lower brain functions) are not able to discriminate and blind tests are used just to put "consciousness" out of the equation.

But now it's me just quibbling , I think the sense of what has been said upon measure vs. hear is clear and also upon hearing abilities not actually changing by expectations, so I stop here...

[Nessuno]

In order to make useful measurements, one has to establish understanding in the person doing the sensory evaluation first. This is called "training" in some contexts.

So hi-end magazines are simply "training" audiophiles on behalf of expensive gears makers?

[Woodinville]

In order to make useful measurements, one has to establish understanding in the person doing the sensory evaluation first. This is called "training" in some contexts.

So hi-end magazines are simply "training" audiophiles on behalf of expensive gears makers?

No, training is much more than reading an article with imprecise, untestable preferences that are stated as facts.

[Woodinville]

The point I was making is that the ears (outer or inner) don't perceive; rather it is the conscious mind does the perceiving.

Ok, but "perceptual" codecs are focused on discarding exactly those stimula that the ears (and/or lower brain functions) are not able to discriminate and blind tests are used just to put "consciousness" out of the equation.

Perceptual codecs, by and large, use the auditory periphery sensitivity to throw out information.

If the periphery can not capture the information, it never reaches the cognative level.

You must separate cognative effects from peripheral effects, please. Cognative effects are as plastic as plastic gets (i.e. they can change, adapt, etc). The periphery is very nearly fixed, with simple feedback mechanisms from the CNS.

[Nessuno]

You must separate cognative effects from peripheral effects, please. Cognative effects are as plastic as plastic gets (i.e. they can change, adapt, etc). The periphery is very nearly fixed, with simple feedback mechanisms from the CNS.

You are augmenting the detail level (and I'm liking to go deeper in a field I don't know in finer details) but this doesn't change the whole picture.
Correct me if I'm wrong: we could see it, for sake of simplicity, like a two block system. The first block which houses the transducer (ear) and its low level driver logic (peripheral auditory system), the second block which houses "business logic" (cognitive functions like, say, construct vectors of stimuli and match them with previously known patterns).
The second block takes its inputs mainly from the first, but is able to change its functions if properly instructed and is also able to aggregate other inputs, from for example sight, to select different functions and give different interpretations to a same input from the first block.

Now, for example, spectral or temporal masking is a resolution limit of the first block, you cannot "train" someone to improve upon this aspect, but you can instruct the second block to overcome some of this limitation integrating its input with other stimuli: when an individual sees, for example, a certain cable between the amp and the speakers he is listening to and he was told those cables possess a "superior resolution power", he actually perceives (in the sense Greynol gave to this term) "firmer bass", "silky trebles", "deeper soundstage" etc etc... even if swapping cables haven't changed anything at first block level.

Relating to what you call "training", correct me once more if I'm missing the point, is actually like improving functions of second block to better recognize patters in stimuli, but using only input from the first block.

So, to go back to the topic of this thread, better measures of a device doesn't directly translate in better perception, as long as their influence is limited to the first block (and the improvement in measurable quantity is beyond its resolving power).

This post has been edited by Nessuno: Jan 2 2013, 12:16

[Martel]

It's nearly impossible to measure subjective feelings of an individual (even sticking electrodes into one's brain doesn't help much). They need to describe/tell you what they hear/feel (until we finally invent the telepathy). Unfortunately, words do not carry nearly as much information as is contained within the feelings they're trying to describe. Hence all those audiophile metaphors (which translate very poorly between individuals and carry almost no useful information as a result).

When I was younger (teen-age) I used to bask in the placebo effect coming from comparing MP3 encoders through a spectrogram. Cut-off at 18kHz was "not good enough for me and sounded like crap", even though the headphone I was using at that time (HD 490 live) had a significant treble roll-off (roughly comparable to Porta Pro) - I just put them on again recently and the treble was virtually non-existent. I ended up using some weird SW called SCMPX, which went all the way to 22kHz on 192kbit MP3 (and likely made worse MP3s overall).
Yet, the associated feeling of superiority (due to extended frequency range that I saw) was really there, it made me enjoy my MP3s much more. While I try to spend as little money as possible to get my equipment, I understand the audiophiles spending fortune on their equipment - for them, just spending big money on equipment is enough to boost their satisfaction level. Due to the physical limits of the human auditory apparatus, this seems to be one of the few ways to improve the perceived sound beyond some objective supremum.

I know I enjoy some fast black/death metal much more when I'm drunk. Some of my friends told me they enjoy techno more when high on weed. Now measure that.
(I'm sorry if this is not appropriate - moderation remove this last sentence in case you find it too offensive)

This post has been edited by Martel: Jan 2 2013, 13:26

[heatfire]

Can only measure that we hear it not how or what we hear;)
What we experience in the sound is just subjective.

[Arnold B. Kruege...]

...if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, especially at borderline conditions.

No argument. Stuff happens.

How do we explain ABX tests with other than perfectly consistent results?

They happen all the time - do they demand an explanation?

This from JJ looks to me like as good of an explanation as any:

"Perception is very much influenced by everything, including your internal thoughts, and pure randomness."

[Arnold B. Kruege...]

It's nearly impossible to measure subjective feelings of an individual (even sticking electrodes into one's brain doesn't help much). They need to describe/tell you what they hear/feel (until we finally invent the telepathy).

I know I enjoy some fast black/death metal much more when I'm drunk. Some of my friends told me they enjoy techno more when high on weed. Now measure that.

I'm told that modern brain imaging can be an effective means for measuring the subjective feelings of an individual, especially pleasure.

Google on fmri

[pdq]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

[Nessuno]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

A failed ABX proves nothing with respect to the codec, or better it doesn't prove that there is no difference between source and encoded sample, but proves that should a difference exists, the individual is not able to discern it, that is the sample is transparent to him.

[heatfire]

There is one measurement some "audiophiles" use that really really is not what we hear.. and that is price.. so a 100dollar amp cant play 1000dollar headphones, might not.. or might do it, money aint the answer to the question.
Not even sure they are audiophiles though, more salesmen then reviewers.

[pdq]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

A failed ABX proves nothing with respect to the codec, or better it doesn't prove that there is no difference between source and encoded sample, but proves that should a difference exists, the individual is not able to discern it, that is the sample is transparent to him.

I agree with the first part of your statement. However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not. The failure could have been due to reasons other than transparency or lack thereof.

[Nessuno]

However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not.

Right, by definition transparency cannot be proved in absolute term, only statistically. But if a tester fail in a statistically relevant number of runs, then well, let's say there is a low probability that tomorrow he could as well succeed in a statistically relevant number of runs, all conditions being equals. May we not, then, draw the conclusion that this sample is transparent to him/her?

The failure could have been due to reasons other than transparency or lack thereof.

As, for example? Please, keep in mind in precedent posts we've already ruled out obvious temporary impairing conditions (illness, fatigue from previous long listening tests, exposition to very noisy environment, very bad mood etc…)

[Kohlrabi]

However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not.

Right, by definition transparency cannot be proved in absolute term, only statistically. But if a tester fail in a statistically relevant number of runs, then well, let's say there is a low probability that tomorrow he could as well succeed in a statistically relevant number of runs, all conditions being equals. May we not, then, draw the conclusion that this sample is transparent to him/her?

To say what pdq said in other words: The reason a failed ABX test doesn't prove anything is because that is not the intention of the ABX test in the first place. You conduct the test to reject the null hypothesis that two samples sound the same, and this can only be achieved by a successful test.

This post has been edited by Kohlrabi: Jan 3 2013, 01:25

[Arnold B. Kruege...]

if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail

Which is exactly why a failed ABX test proves nothing.

If you are going to be that absolutist about it, then nothing proves anything.

Of course, in Science, proof is not of the essence.

All findings of Science are provisional, and only relevant until something more relevant is found.

So even though a failed ABX test proves nothing, it means something.

[Kohlrabi]

So even though a failed ABX test proves nothing, it means something.

Sure, a lot of failed tests with lots of different listeners and circumstances means that it is very, very likely that the samples sound the same and the null hypothesis holds. Still, that's not a proof in the meaning of the word, but likely good enough for any real life considerations.

Science is, after all, the aim to find the most likely explanations for things occurring in nature. To accept that our modern life (or science in general) is based on not-yet-rejected hypotheses doesn't give me sleepless nights, nor do I think about it every waking hour.

It might feel a bit annoying to split hairs over these semantics, but to me understanding the purpose of the ABX test really helps to understand the meaning of its results and their interpretation.

This post has been edited by Kohlrabi: Jan 3 2013, 01:34

[pdq]

The failure could have been due to reasons other than transparency or lack thereof.

As, for example? Please, keep in mind in precedent posts we've already ruled out obvious temporary impairing conditions (illness, fatigue from previous long listening tests, exposition to very noisy environment, very bad mood etc…)

For example, the tester performed too few trials to be statistically significant, but a greater number of trials would have shown that he was able to hear a difference.

In any case, I am not fundamentally in disagreement with anything that has been said here on the subject, so I will stop nit picking.

[Nessuno]

However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not.

Right, by definition transparency cannot be proved in absolute term, only statistically. But if a tester fail in a statistically relevant number of runs, then well, let's say there is a low probability that tomorrow he could as well succeed in a statistically relevant number of runs, all conditions being equals. May we not, then, draw the conclusion that this sample is transparent to him/her?

To say what pdq said in other words: The reason a failed ABX test doesn't prove anything is because that is not the intention of the ABX test in the first place. You conduct the test to reject the null hypothesis that two samples sound the same, and this can only be achieved by a successful test.

Stricto sensu, neither a successful test proves anything: a tester can, in theory, guess every time the correct answer without even listening. Of course statistics as well as real life experience lead us to consider this a very unlikely thing to happen, so we accept a successful test as a proof. This is also the reason why a single run cannot prove anything either way.
Now, if a tester fails in a statistically relevant number of runs etc etc... (as per above) what could it possibly mean, according to statistics and real life experience? Will you still accept the hypothesis that the two samples sound different to him?

Edit: yep, I think the whole sense of what has been said is clear, so I stop nitpicking too...

This post has been edited by Nessuno: Jan 3 2013, 02:26

[Woodinville]

However, a failed ABX test does not prove that the sample was transparent to the tester, only that (s)he was unable to show, statistically speaking, that it was not.

Right, by definition transparency cannot be proved in absolute term, only statistically. But if a tester fail in a statistically relevant number of runs, then well, let's say there is a low probability that tomorrow he could as well succeed in a statistically relevant number of runs, all conditions being equals. May we not, then, draw the conclusion that this sample is transparent to him/her?

To say what pdq said in other words: The reason a failed ABX test doesn't prove anything is because that is not the intention of the ABX test in the first place. You conduct the test to reject the null hypothesis that two samples sound the same, and this can only be achieved by a successful test.

Stricto sensu, neither a successful test proves anything: a tester can, in theory, guess every time the correct answer without even listening. Of course statistics as well as real life experience lead us to consider this a very unlikely thing to happen, so we accept a successful test as a proof. This is also the reason why a single run cannot prove anything either way.
Now, if a tester fails in a statistically relevant number of runs etc etc... (as per above) what could it possibly mean, according to statistics and real life experience? Will you still accept the hypothesis that the two samples sound different to him?

Edit: yep, I think the whole sense of what has been said is clear, so I stop nitpicking too...

When, at the same time, the subject performs well beyond chance in the positive control stimulus, and randomly in the negative control stimulus.

[knutinh]

Good work carried out by Toole and others aside:
Are there good measurement methods available that can prove that the spatial characteristics of two audio events will be indistinguishable in a blind test?

I think that measuring the behaviour of loudspeakers in a room (or microphones) in a perceptually meaningfull manner is quite hard. It is easy to find _some_ difference, it is hard to tag the relevant differences while discarding the irrelevant differences.

People tends to move their heads, and HRTFs can vary quite a bit from person to person. Sticking a single omni B&K microphone at "sweet spot" does not tell us all there is to know about a loudspeaker/room system. Toole advocates empirical weighting of on-axis/off-axis loudspeaker measurements done in anechoic chambers, together with physical analysis of room geometry, but I dont think that approach is necessarily "perfect"?

-k