~96 kbps Listening Test? Options

[dsimcha]

I've recently discovered my inability to reliably ABX the latest AoTuV beta against FLACs from my music library at -q2 (nominal bitrate 96 kbps). The last ~96 kbps listening test I've found was done in 2005, before Aoyumi's significant improvements to Vorbis's noise normalization, and probably before some substantial improvements to other codecs as well.

I'd be very interested to see what the landscape of modern encoders at 96 kbps is like. Specifically, it would be interesting to use something like LAME -V4 or -V5 (where LAME is widely regarded as being near-transparent) as a high anchor and see if a more modern codec at ~96 kbps can compete with it. I believe ~96 kbps is very close to perceptual transparency, at least with Vorbis.

[Polar]

IgorC has also been active in this testing field in a few 2009 threads here, amongst which http://www.hydrogenaudio.org/forums/index....showtopic=66949

I'm just as curious as you btw, about Vorbis' and AAC's current-day performance at 96k. My own not so profound tests (little time, bad headphones) suggest quality at least as good as LAME -V5. Let's hope coining our interest here triggers some more.

[dsimcha]

IgorC has also been active in this testing field in a few 2009 threads here, amongst which http://www.hydrogenaudio.org/forums/index....showtopic=66949

I'm just as curious as you btw, about Vorbis' and AAC's current-day performance at 96k. My own not so profound tests (little time, bad headphones) suggest quality at least as good as LAME -V5. Let's hope coining our interest here triggers some more.

Yeah, LAME -V5 is about what I'd say Vorbis -q2 is equivalent to, too. This is based on ABX tests I did about a year ago. I'd call both "almost transparent". I can ABX some stuff but it's ridiculously hard and there's a lot more stuff that I can't ABX than stuff that I can. I only hear artifacts when I'm listening very hard for them and they're never very annoying. One more quality setting down (-V6 for LAME or -q1 for Vorbis) and the artifacts become much easier to notice in ABX tests and occasionally annoying enough to affect my enjoyment of the music.

This post has been edited by dsimcha: Jun 14 2011, 16:31

[mudlord]

Would be interesting to see more tests with CELT

[klonuo]

With 0.11.2 or 0.11.4? Or with some future version with non back compatible bit-stream?
Looking from outside, 64kbps tests seem like bless to "CELT is out!, Help wanted"

[IgorC]

Looking from outside, 64kbps tests seem like bless to "CELT is out!, Help wanted"

You're right. The main idea of the last test was to figure out how new CELT/Opus performs against actual codecs.
Does it change anything?


Now if the next test will be only AAC codecs someone will see it like promoting AAC or something like that.
But the fact is that there are too many good AAC encoders. First we should figure out what is the most optimal AAC encoder and only then do multiformat (MP3, Vorbis, AAC...) test.

[klonuo]

Does it change anything?

Not to me. I have noting to do with resource management for listening tests. Only feel sorry for HA members trying to compile latest CELT as a result. Not just because it is unfinished, but when they seek valid help in that same thread - developers vanished...

[dsimcha]

Does it change anything?

I suspect that CELT doesn't change much because at ~96 kbps because Vorbis and possibly AAC are fairly close to transparency. After all, perceptual transparency is as good as it gets and for our purposes the low latency of CELT isn't very useful.

[C.R.Helmrich]

First we should figure out what is the most optimal AAC encoder and only then do multiformat (MP3, Vorbis, AAC...) test.

Or the other way around I'd like to first find the best non-AAC encoder and then compare it to a few AAC encoders at the same bit rate. So we could run a 96-kbps pre-test incl.

• LAME
• GXLame (if the Sheep of Death promises that his latest version can compete with LAME)
• Vorbis
• CELT 0.11.latestgreatest

This pre-test would also give us some time for an AAC pre-test (i.e. excluding relatively bad AAC encoders). Opinions?

Chris

[IgorC]

Multiformat test without AAC isn't enough conclusive.
At least one AAC encoder should be included.

It can be:
~96 kbps
1. LAME (GXLAME or any other low bitrate MP3 encoder)
2. Vorbis (AoTuV 6)
3. AAC (Apple or Nero. To be discussed)
4. Opus (To be discussed)