High bitrate MP3 vs. Lossless ABX Tests?, Are there any studies...? Options

[Street Samurai]

Hi All,

I've been corrected before on this site for making the statement: "I can tell the difference between an MP3 and lossless". I've never ABX'ed myself but I was wondering if there has been a statistically significant (with regard to population) ABX study done with (say) Lame insane vs. Lossless?

Clearly for myself it makes more sense to do a personal ABX but I was wondering in general about the population.

I searched manually through the posts in this forum (since search seems useless for the term 'mp3') and didn't see anything.

Thanks for help,

ss.

This post has been edited by Street Samurai: Jun 2 2005, 01:02

[Woodinville]

Hmm, there's a bit of confusion in this thread.

All of the MPEG verification tests were made against the original, which is the same thing to compare to as a bitwise-exact coding method.

All of the MPEG verification tests I'm aware of, which would include MPEG-1, MPEG-2 BC, MPEG-2 NBC, various and sundry MPEG-4 verification tests, have shown that at least some listeners appear to distinguish between the original and the coded at the test rate, even for rates like 256 and 192 for layers 1 and 2, and for 192 as Layer 3, if I recall correctly.

The CRC test for the american DAR project, the one done a long time ago when the audio quality was of the 160kb/s variety, also showed that 160kb/s PAC and some higher rate of (I think it was AC3, but I frankly don't remember) could be distinguished from the original.

So, I think it's confirmed via some very sophisticated tests, done by world experts, that the usual rate of most codecs is not "transparent" in the ABX sense, although MPEG has used a standard wherein anything above the 4.0 CCIR scale mark is considered 'acceptable'.

[Lyx]

So, I think it's confirmed via some very sophisticated tests, done by world experts, that the usual rate of most codecs is not "transparent" in the ABX sense, although MPEG has used a standard wherein anything above the 4.0 CCIR scale mark is considered 'acceptable'.

Well, not only the bitrate average matters, but also the encoder and method. Is Fhg CBR competitive against LAME VBR at high-bitrates? How high was the amount of positive ABX-tests compared to total trials? What kind of samples were used?

The most sophisticated tests done by world-experts can only tell something about "what" they tested. If the tested encoder/settings were inferior compared to high-bitrate LAME VBR, then the tests only concluded just that: That with an inferior encoder some people were able to tell the difference.

- Lyx

This post has been edited by Lyx: Jun 5 2005, 02:17

[Woodinville]

The most sophisticated tests done by world-experts can only tell something about "what" they tested. If the tested encoder/settings were inferior compared to high-bitrate LAME VBR, then the tests only concluded just that: That with an inferior encoder some people were able to tell the difference.

- Lyx

Well, in those tests, the encoders were built by the algorithm proponents, with everything on the line, based primarily on encoded quality.

I don't think it's possible to get evidence at this point to show exactly what was tested, but the people doing the encoding were the inventors of the coding methods, and the ones with the most to lose if the test results weren't good. The encoders were generally not restricted to real time, not restricted to simple, and were supposed to show the "ultimate" quality.

That's all I can offer you.

None the less, I don't think there is a 128kb/s MP3 encoder that meets the "ABX transparent" standard, or at least I haven't meant one, even in tests where all of the degradations were reported as being over 4.0 on the scale, the ID for the 'reference' signal in an ABC/hr test wasn't even close to any reasonable random hypothesis. The MPEG test results, at least, as well as the CRC results that are on much, much newer encoders, have been published. I don't know the citations offhand, but the results were not ambiguous in any real fashion.