New Listening Test in 2007, MP3 @ 128 kbps or Multiformat @ 64 kbps? Options

[Sebastian Mares]

Now that the multiformat test at 48 kbps is over, what would you like to see next? If the MP3 test wins, the pre-test discussion shouldn't last long because we already had one before the 48 kbps test started.

[Firon]

I'd like to see either a 64kbps or a 96kbps test. A 96kbps test would be great since a lot of songs are already transparent to me at this bitrate using Vorbis.

This post has been edited by Firon: Dec 16 2006, 18:12

[guruboolez]

Same preference. 80 and 96 kbps comparison could be interesting - and were never done before.
To answer to the alternative: 64 kbps multiformat instead of 128 kbps layer 3.

[haregoo]

64k is too close to 48k. I guess the result won't suprise us. I'd like to try 96k multi or 128k MP3.

[singaiya]

I'd like 64 or 80 personally.

[Sebastian Mares]

Thing is that at 64 kbps, WMA Professional 10 can be tested in VBR mode and we can also (more or less) compare the outcome to Microsoft's test.

[senab]

I'd like a 64/80 multi test

[Firon]

Thing is that at 64 kbps, WMA Professional 10 can be tested in VBR mode and we can also (more or less) compare the outcome to Microsoft's test.

Well, I suppose that way we can defeat the claims about "CD quality at half the bitrate of 128kbps MP3!"
I'd still like a 96kbps test though.

This post has been edited by Firon: Dec 16 2006, 20:50

[Junon]

Thing is that at 64 kbps, WMA Professional 10 can be tested in VBR mode and we can also (more or less) compare the outcome to Microsoft's test.

In theory a 80 kbps test would be possible as well. I just encoded a few samples using the Q25 setting:

CODE

Aerosmith - Get a Grip - Eat the Rich: 91 kbps
Alanis Morissette - MTV Unplugged - I Was Hoping: 84 kbps
Apocalyptica - Cult [CD2] - Hope Vol. II feat. Matthias Sayer:  79 kbps
Billy Joel - Greatest Hits Volume I - The Entertainer: 87 kbps
Creed - Weathered - My Sacrifice: 85 kbps
Def Leppard - Euphoria - Demolition Man: 85 kbps
Faith No More - King for a Day / Fool for a Lifetime - Ricochet: 85 kbps
Foo Fighters - In Your Honor [CD1] - No Way Back: 89 kbps
In Flames - Come Clarity - Come Clarity: 82 kbps
Nightwish - Oceanborn - Gethsemane: 85 kbps
Threshold - Wounded Land - Consume to Live: 91 kbps
Tori Amos - Little Earthquakes - Little Earthquakes: 86 kbps
Trans-Siberian Orchestra - Beethoven's Last Night - Mephistopheles: 82 kbps
Trivium - The Crusade - The Rising: 84 kbps
Vaya Con Dios - The Best of Vaya Con Dios - Nah Neh Nah: 80 kbps
Within Temptation - The Silent Force - Memories: 86 kbps

All files' average bitrate: 85 kbps

I agree to the statement that 64 kbps are too close to the latest test's bitrate, a 128 kbps MP3 test is of no interest to me. I'd prefer a 80 kbps test instead, hence I abstain from voting.

Full edit: Now, having calculated a few files' average bitrates (filesize/runtime), I noticed that the above list was completely wrong. Both Winamp and foobar obviously display the maximum bitrate in their "bitrate" fields when checking the properties of a WMA file. Winamp calculates the correct average one on playback as well as in the "Current Bitrate" and "Optimal Bitrate" fields, hence I corrected all results.

This post has been edited by Junon: Dec 17 2006, 00:22

[Firon]

That's the point of VBR though. Even Vorbis does that (huge bitrate "overshoots").
There's no reason to exclude the ones that go over a lot.

[kwanbis]

i rather see a multformat at 164 or 192 ... yes, i know the dificulty.

[molnart]

I use lossy compression only for portable purposes, so mp3 is the only interesting format for me. Alternatively, there could be a test where mp3@128 kbps would be compared to other lossy codecs at lower bitrates (64-96kbps).

[Junon]

i rather see a multformat at 164 or 192 ... yes, i know the dificulty.

Judging by the high anchor's results of the latest 48 kbps test I don't think there's any need for such a test. Quite a lot of the testers obviously sensed 96 kbps AAC to be transparent.

@all

Please note the corrected table above. Both Winamp and foobar displayed incorrect results.

[Kirby54925]

96k!!!!!!!! Seriously, 128k's been done a lot already, so let's push the envelope down even further so that at some point in the future, we can declare 96k to be the new 128k =)

[emtee]

Personally, I think 64 is too close to the latest listening test. I'm afraid I lack the motivation to go through it again, especially considering the results won't be too surprising. I'd gladly participate on a 96 kbps multiformat listening test, though.

[Mike Giacomelli]

100k would be nice. IMO this is the sweet spot for flash based players. 64k is just too low, and 128k is more then is needed judging from the last test.

[IgorC]

64k is too close to 48k. I guess the result won't surprise us. I'd like to try 96k multi or 128k MP3.

I have another experience at 64 kbit/s. Vorbis is closer to Nero and WMA10pro is best performer.

As many people already mentioned 64 is not enough. Todays the encoders have a good quality at 96-128. So it will be interesting for me to see 80 kbit test. Also it would be useful to see how performs Vorbis against AAC and SBR vs LC performance.

Vote for 64 or 80.

This post has been edited by IgorC: Dec 17 2006, 02:19

[Jillian]

I'll vote for 64kbps, but still want ~80.
Like Junon, Q25 is possible.

[Sebastian Mares]

Three things:

1. At bitrates other than 64 kbps, we cannot compare the results to Microsoft's. Maybe this isn't such a big problem because you can never compare results 1:1 since other testers and samples were "used", anyways.
2. Considering a test at 64 kbps, do you think WMA Standard should be tested again? If not, comparing 3 contenders only could be a waste of resources.
3. At bitrates over 64 kbps (e.g. 80 or 96 kbps), is HE-AAC or LC-AAC the better choice?

This post has been edited by Sebastian Mares: Dec 17 2006, 09:59

[LadFromDownUnder]

I think it would be good if a listening test has some practical outcome, rather than being academic interest. Given this goal, codecs should be chosen that are relatively widely in use on portable devices (why would someone choose low bitrates when they have hundreds of gigabytes of storage at their disposal on their home PC?). Also, given that flash RAM storage is all the rage right now, typical portable capacities are coming in at 1 GB to 4 GB, so low bitrates are desired.

I voted for the 64 kbps test, but I would brefer to see something between 64 and 128 kbps, with codecs widely available NOW on flash RAM portable devices.

Assuming 1 GB of storage (multiply as appropriate), 128 kbps amounts to 17 hours of play time, 96 kbps is 23 hours, 80 kbps is 28 hours, and 64 kbps is 35 hours.

I'm not fussed whether it's VBR, ABR, or CBR, so long as it's the best quality at an agreed average bitrate (I expect it will be VBR or ABR). I'm not fussed whether it's more than one vendor's implementation of a format (a couple of MP3 contenders, a couple of AAC contenders, and a Vorbis contender?).

That's my thoughts.

[Sebastian Mares]

Well, all contenders from the 48 kbps listening test are also "portable-friendly", with a lot of players supporting WMA Standard, some players supporting Ogg Vorbis (Cowon iAudio products, some Trekstor), some supporting HE-AAC (Nokia mobile phones and maybe others I don't know of) and one supporting WMA Professional (Zune AFAIK). Therefore, at 64 kbps, we could use the same contenders. At 80 or 96 kbps, maybe we could also use MP3.

This post has been edited by Sebastian Mares: Dec 17 2006, 10:15

[matthias1985]

I'd like to try 96k with several encoders, because the last 128k test showed that this bitrate is nearly transparent for the most users.

[breez]

I agree with other posters requesting 96k or 80k test. Voted for 64k.

[Sebastian Mares]

Between 96 and 80 kbps, I would be more happy to conduct an 80 kbps test, 96 kbps being too close to 128 kbps. Also high anchor in 48 kbps test already was almost transparent.

[spockep]

100k would be nice. IMO this is the sweet spot for flash based players. 64k is just too low, and 128k is more then is needed judging from the last test.

Not when some people participating in the last test gave high achor a 5 automatically. This is without really listening to the high anchor sample. What a shame. It really invalidates the high anchor score IMO.