Public AAC Listening Test @ ~96 kbps [July 2011]: Results, Results and post-test discussion Options

[IgorC]

After the long time of preparations, discussions and realization of the test the results are finally here.

http://listening-tests.hydrogenaudio.org/i...-a/results.html

Summary: Apple won, FhG is the second, Coding Technologies is the third and Nero is the last

I appreciate all people who has supported the test and participated in it.

This post has been edited by IgorC: Aug 23 2011, 20:12

[Sebastian Mares]

It appears to me that the low anchor was way too bad. Shouldn't the low anchor be at around the same quality as the contenders, but "slightly" worse than all of them?

[Garf]

It appears to me that the low anchor was way too bad. Shouldn't the low anchor be at around the same quality as the contenders, but "slightly" worse than all of them?

Not sure about this one, I thought it should "calibrate the scale". (Because the overall quality is so high, it's less needed at the upper end)

If you don't use an anchor, what happens is that for a minor distortion users will tend to slam down the slider. The anchor serves as a reminder "what really bad really is".

It would be more useful if the anchor stayed the same throughout the tests, I guess. Probably the opportunity to test ffmpeg in one swoop was interesting. No idea if it was understood it is *this* bad.

FWIW, this is a somewhat relevant and interesting paper I hadn't seen before:
http://www.acourate.com/Download/BiasesInM...teningTests.pdf

[Nezmer]

Probably the opportunity to test ffmpeg in one swoop was interesting. No idea if it was understood it is *this* bad.

The AAC and Vorbis encoders in FFmpeg/libav were written to produce valid bitstreams without implementing any sophisticated optimisations. So, the results here shouldn't be a surprise.

[Garf]

Probably the opportunity to test ffmpeg in one swoop was interesting. No idea if it was understood it is *this* bad.

The AAC and Vorbis encoders in FFmpeg/libav were written to produce valid bitstreams without implementing any sophisticated optimisations. So, the results here shouldn't be a surprise.

Sorry, but this just isn't true for the ffmpeg AAC encoder. Have you actually looked at it? It's reasonably sophisticated, more sophisticated than FAAC for example. It has a real psymodel, 3 different quantization loop algorithms, proper short block switching, etc.

Even so, there's no particular reason to believe a non-sophisticated AAC encoder must terribly suck. Again, FAAC is good reference.
http://listeningtests.t35.me/html/AAC_at_1...est_results.htm

As far as I can tell, the problem is that it is utterly riddled with bugs and was probably never properly tested and debugged. It might be misdesigned too, but I feel like I'm sticking out my neck here because I could be wrong on that - maybe the current design works fine if you fix the bugs.

The ffmpeg AAC encoder is crap because it's buggy and insufficiently tested. Not because it's missing sophisticated algorithms.