Bayesian approach to ABX - call for comments Options

[Gag Halfrunt]

Hello all,

I happened across this:Hopkins Research - ABX methods for measuring human perception of change

Really not sure what to make of it. Anyone care to comment?

[ojdo]

As far as I read it it seems to be a nice introduction to the ABX test method. What do you mean by "not sure what to make of it"?

[Gag Halfrunt]

As far as I read it it seems to be a nice introduction to the ABX test method. What do you mean by "not sure what to make of it"?

Not the test methodology part, the interpretation of the results of that test. As in evaluating the probability of hearing a difference, instead of the probability of correctly identifying X. It strikes me as putting the cart before the horse.

[ojdo]

probability of hearing a difference, instead of the probability of correctly identifying X

Apart from semantics, what is the difference? In order to be able to get a significant test result you definitely must be able to hear a difference between A and B. If not, you are stuck at 50% correct guesses.

[MLXXX]

As far as I read it it seems to be a nice introduction to the ABX test method. What do you mean by "not sure what to make of it"?

Not the test methodology part, the interpretation of the results of that test. As in evaluating the probability of hearing a difference, instead of the probability of correctly identifying X. It strikes me as putting the cart before the horse.

In ABX testing, for each answer attempt ("trial") the unknown X will [by design of the testing equipment] be either A or B. If the subject claims to hear a difference between A and B, the subject ought to be able to correctly match X to A or to B, i.e. correctly identify X.

When it boils down, the probability of correctly identifying X strongly correlates with the probability of truly hearing a difference, i.e. "hearing a difference" that is not illusory.

When operating at the limits of human perception, the subject will often be unable to distinguish between what is real and what is imagined.

I note that:-

Failure to give a correct answer for a trial can be due to:
* the subject's mind or ears are playing tricks on them (the subject thinks they perceive a difference but it is illusory and happens to be wrong)
* the subject makes what seems to them to be a pure guess and the guess is wrong
* the subject makes a mistake in recording their answer (can occasionally happen if the subject rushes their answer or is very tired)

The giving of a correct answer for a trial can be due to:
* the subject's mind or ears are playing tricks on them (the subject thinks they perceive a difference but it is illusory, and happens to be right)
* the subject makes what seems to them to be a pure guess and the guess is right

probability of hearing a difference, instead of the probability of correctly identifying X

Apart from semantics, what is the difference? In order to be able to get a significant test result you definitely must be able to hear a difference between A and B. If not, you are stuck at 50% correct guesses.

I think the article explores situations where the stimulus is barely perceptible and from time to time the subject just manages to detect it, doesn't quite manage to detect it, or falsely imagines they have detected it. In these circumstances over a large number of trials the perecentage of correct answers will lie somewhere between 50 and 100%. If the underlying ability to truly detect is sporadic, the number of trials required for a statistically significant result can be very high. The article suggests acceptance in practice of a lower standard of statistical significance than for more straightforward testing where the stimulus is clearly perceptible almost every time it is presented.

___

For myself, I find I can will my brain to hear a difference even if I am comparing bit for bit identical files using foobar2000. This is a bit scary. I can tell my brain "the next time you press the X button, the sound will be slighter lower in pitch"; and it does then sound lower in pitch for my hearing. Or I can will it to sound slightly higher in pitch. A relevant part of my brain cooperates, despite the fact that I know intellectually that the sounds being played by fb2k each time I press the X button for a particular trial, must be the same.

It is no wonder that sighted tests can leave people convinced they are hearing a difference, if they have preconditioned their brains to expect a difference.

This post has been edited by MLXXX: Sep 13 2009, 07:25

[Arnold B. Kruege...]

For myself, I find I can will my brain to hear a difference even if I am comparing bit for bit identical files using foobar2000. This is a bit scary. I can tell my brain "the next time you press the X button, the sound will be slighter lower in pitch"; and it does then sound lower in pitch for my hearing. Or I can will it to sound slightly higher in pitch. A relevant part of my brain cooperates, despite the fact that I know intellectually that the sounds being played by fb2k each time I press the X button for a particular trial, must be the same.

I find that no act of will is required at all to perceive a difference when there is none.

The usual explanation is that human perception was developed in a context where false positives had a far lower cost than false negatives.

IOW, hearing that an enemy is sneaking up on you when there is none costs little or nothing, but not hearing the enemy when he is actually sneaking up on you can extract the highest possible price.

It is no wonder that sighted tests can leave people convinced they are hearing a difference, if they have preconditioned their brains to expect a difference.

Also, without some check on reliability, claims of audible differences are generally settled in the favor of the one with the loudest voice or most persistence.

Without some reliable means of evaluating results, it is easy for even well-meaning people to claim that everyody who can't hear what they claim to hear simply have "cloth ears".