Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: ABX Debate (Read 17608 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

ABX Debate

Reply #25
Quote
[a href="index.php?act=findpost&pid=357266"][{POST_SNAPBACK}][/a]
Quote
For example, If I am comparing 32kbps MP3 versus 44.1/16 wav and I truly expect there not to be a difference, I can bias the result of the test in such a way as a negative result is produced.

Does that count as properly performing an ABX test  - deliberately saying the files are indistiguishable when you can actually hear a difference (selecting B when you know the correct answer is A)?

To me that is more akin to making a concerted attempt to bias/sabotage the test. Any experiment will become useless if testers makes a decision to ignore, or say the opposite, to what their senses are telling them.
[a href="index.php?act=findpost&pid=357474"][{POST_SNAPBACK}][/a]
It might not be a conscious effort. Expectation bias can play a huge role in non-blind test results without the tester conciously being biased. That is why we choose a double blind testing methodology. Surely if the tester expects that there is no difference, he can be biased towards a negative hypothesis, even subconciously?

In my mind the ABC/HR methodology is stronger than ABX against this bias. ABX doesn't prevent this from happening at all. It doesn't mean that positive ABX results are any less valid - it's still extremely good at removing expectation bias in the positive direction. ABX result statements like "there is a 99.99% chance the tester can distinguish between these samples" are still valid.

In my mind, ABX results like "the tester could not distinguish between these samples" are not truly the results of a double-blind test. I might be wrong about this, but it seems logical to me.

ABX Debate

Reply #26
Quote
In my mind, ABX results like "the tester could not distinguish between these samples" are not truly the results of a double-blind test. I might be wrong about this, but it seems logical to me.
[a href="index.php?act=findpost&pid=357495"][{POST_SNAPBACK}][/a]



Do you know the term "controls"?

Quite aside from that, are you saying that the subject knows which signal 'x' is?
-----
J. D. (jj) Johnston

ABX Debate

Reply #27
Quote
Quote
In my mind, ABX results like "the tester could not distinguish between these samples" are not truly the results of a double-blind test. I might be wrong about this, but it seems logical to me.
[a href="index.php?act=findpost&pid=357495"][{POST_SNAPBACK}][/a]



Do you know the term "controls"?
[a href="index.php?act=findpost&pid=357665"][{POST_SNAPBACK}][/a]

Indeed, but how is this relevent?
Quote
Quite aside from that, are you saying that the subject knows which signal 'x' is?
[a href="index.php?act=findpost&pid=357665"][{POST_SNAPBACK}][/a]
No, the subject doesn't know what X is. However, the subject expects that the difference between A and B is not audible and is therefore biased towards selecting X at random. The point I am trying to make (perhaps not clearly) is that expectation on the part of the subject can bias them (consciously or subconciously) into not putting their "best effort" into identifying X. Surely the ABX methodology is based on the hypothesis that the test subject made their best effort?

ABX Debate

Reply #28
Expectation depends completely on what the test subject knows about the material before the test starts. I don't see how ABC/HR is different from ABX in this regard, as the effort required to come to a positive result is not different and the effort required to select at random will be equal between the two too.
"We cannot win against obsession. They care, we don't. They win."

ABX Debate

Reply #29
Quote
Quote
Quote
In my mind, ABX results like "the tester could not distinguish between these samples" are not truly the results of a double-blind test. I might be wrong about this, but it seems logical to me.
[a href="index.php?act=findpost&pid=357495"][{POST_SNAPBACK}][/a]



Do you know the term "controls"?
[a href="index.php?act=findpost&pid=357665"][{POST_SNAPBACK}][/a]

Indeed, but how is this relevent?
Quote
Quite aside from that, are you saying that the subject knows which signal 'x' is?
[a href="index.php?act=findpost&pid=357665"][{POST_SNAPBACK}][/a]
No, the subject doesn't know what X is. However, the subject expects that the difference between A and B is not audible and is therefore biased towards selecting X at random. The point I am trying to make (perhaps not clearly) is that expectation on the part of the subject can bias them (consciously or subconciously) into not putting their "best effort" into identifying X. Surely the ABX methodology is based on the hypothesis that the test subject made their best effort?
[a href="index.php?act=findpost&pid=357684"][{POST_SNAPBACK}][/a]



Then how do the controls not detect this?
-----
J. D. (jj) Johnston