I have come up with an variant of ABX testing that is useful in discovering which differences in the encoded vs. original file are really significant. Basically, a person is given files X and Y and is left to determine which one is compressed and which one is not without references to known files. In order to do this one must not only be able to notice differences in the file but must be able to determine whether those differences are compression artifacts or part of the original file, and to determine that one file clearly sounds superior to the other. The belief is that any differences that can be perceived but don't result in a discernible loss of subjective quality are insignificant for many purposes.
In order to do this test with existing software, use WinABX and use the ABXY mode and simply don't click on A or B, only listen to X and Y. I guarantee that you will find it substantially more difficult to successfully XY a file than to ABX it.
Full Version: Alternative test to ABX
is this progress? Can it be used to tell who the real US president is?
QUOTE(dsimcha @ Nov 2 2004, 10:20 PM)
In order to do this one must not only be able to notice differences in the file but must be able to determine whether those differences are compression artifacts or part of the original file
isn't that exactly what ABXing is?! if the 'differences' are part of the original, there a re no differences. if the differences are artifacts, well, then there are differences... so...this method is not really new, but just described differently, or am I wrong here?QUOTE
and to determine that one file clearly sounds superior to the other
ABC-HR?QUOTE(fairyliquidizer @ Nov 2 2004, 10:43 PM)
is this progress? Can it be used to tell who the real US president is?
no president indications (it's Bush anyway...), but you can automate it to milk cows while making coffee at the same time. with sugar of course.edit: sugar
QUOTE(Digga @ Nov 3 2004, 02:58 AM)
QUOTE(dsimcha @ Nov 2 2004, 10:20 PM)
In order to do this one must not only be able to notice differences in the file but must be able to determine whether those differences are compression artifacts or part of the original file
isn't that exactly what ABXing is?! if the 'differences' are part of the original, there a re no differences. if the differences are artifacts, well, then there are differences... so...this method is not really new, but just described differently, or am I wrong here?QUOTE
and to determine that one file clearly sounds superior to the other
ABC-HR?No, ABC-HR would not do this. The point is to prove that one can not only discern between the 2 files and say that they do not sound the same. They must be able to say that one clearly sounds better than the other, and to say that one is encoded and the other is not. With ABX, if there is any perceivable difference between A and X, you know that X is B. With XY, there can be a perceivable difference but if one does not clearly sound better than the other you cannot say which is encoded. Therefore, XY testing would test for the significance of perceived differences.
(Enough with the presidents crap, we all know we just voted for the lesser of two evils, whoever that might be anyhow)
edit: sugar
QUOTE(dsimcha @ Nov 3 2004, 06:22 PM)
QUOTE
and to determine that one file clearly sounds superior to the other
ABC-HR?No, ABC-HR would not do this. The point is to prove that one can not only discern between the 2 files and say that they do not sound the same. They must be able to say that one clearly sounds better than the other, and to say that one is encoded and the other is not.
QUOTE(dsimcha @ Nov 3 2004, 09:22 AM)
No, ABC-HR would not do this. The point is to prove that one can not only discern between the 2 files and say that they do not sound the same. They must be able to say that one clearly sounds better than the other, and to say that one is encoded and the other is not. With ABX, if there is any perceivable difference between A and X, you know that X is B. With XY, there can be a perceivable difference but if one does not clearly sound better than the other you cannot say which is encoded. Therefore, XY testing would test for the significance of perceived differences.
If I understand the point of your method, it is that just perceiving a statistical difference (ABX) is not a real-world test. That difference may be so small that it doesn't really matter in normal listening. Your idea is that if you can clearly determine which one is encoded, that means the difference is large enough to be a "significant" difference, not just statistically significant.
The problem with this method is that the encoded version may actually sound better than the original, for example, if the encoding process reduces background hiss or reduces distortion (which I have noticed happening on some samples).
ff123
QUOTE(ff123 @ Nov 3 2004, 11:08 PM)
The problem with this method is that the encoded version may actually sound better than the original, for example, if the encoding process reduces background hiss or reduces distortion (which I have noticed happening on some samples).
Exactly my first thought. Remember the myth about vorbis sounding "better than the original" because of the hf-boost ?
The danger is that with your method, it may easily happen that people do not rank transparency, but insteat "taste".
But:
1. Tastes differ (in the vorbis-example the artifact which others liked caused slight headaches to me)
2. "Coloration" of music imho is the job of a DSP or EQ. A codec should try to reproduce the original signal as transparent as possible - people can then afterwards tune it to their taste with DSPs and EQs. Therefore, "taste" imho isn't a desirable rating method for testing the "quality".
- Lyx
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.