Now that the multiformat test at 48 kbps is over, what would you like to see next? If the MP3 test wins, the pre-test discussion shouldn't last long because we already had one before the 48 kbps test started.

I'd like to see either a 64kbps or a 96kbps test. A 96kbps test would be great since a lot of songs are already transparent to me at this bitrate using Vorbis.

Same preference. 80 and 96 kbps comparison could be interesting - and were never done before.
To answer to the alternative: 64 kbps multiformat instead of 128 kbps layer 3.

64k is too close to 48k. I guess the result won't suprise us. I'd like to try 96k multi or 128k MP3.

I'd like 64 or 80 personally.

Thing is that at 64 kbps, WMA Professional 10 can be tested in VBR mode and we can also (more or less) compare the outcome to Microsoft's test.

I'd like a 64/80 multi test

Well, I suppose that way we can defeat the claims about "CD quality at half the bitrate of 128kbps MP3!"
I'd still like a 96kbps test though.

In theory a 80 kbps test would be possible as well. I just encoded a few samples using the Q25 setting:



I agree to the statement that 64 kbps are too close to the latest test's bitrate, a 128 kbps MP3 test is of no interest to me. I'd prefer a 80 kbps test instead, hence I abstain from voting.

Full edit: Now, having calculated a few files' average bitrates (filesize/runtime), I noticed that the above list was completely wrong. Both Winamp and foobar obviously display the maximum bitrate in their "bitrate" fields when checking the properties of a WMA file. Winamp calculates the correct average one on playback as well as in the "Current Bitrate" and "Optimal Bitrate" fields, hence I corrected all results.

That's the point of VBR though. Even Vorbis does that (huge bitrate "overshoots").
There's no reason to exclude the ones that go over a lot.

i rather see a multformat at 164 or 192 ... yes, i know the dificulty.

I use lossy compression only for portable purposes, so mp3 is the only interesting format for me. Alternatively, there could be a test where mp3@128 kbps would be compared to other lossy codecs at lower bitrates (64-96kbps).

Judging by the high anchor's results of the latest 48 kbps test I don't think there's any need for such a test. Quite a lot of the testers obviously sensed 96 kbps AAC to be transparent.

@all

Please note the corrected table above. Both Winamp and foobar displayed incorrect results.

96k!!!!!!!! Seriously, 128k's been done a lot already, so let's push the envelope down even further so that at some point in the future, we can declare 96k to be the new 128k =)

Personally, I think 64 is too close to the latest listening test. I'm afraid I lack the motivation to go through it again, especially considering the results won't be too surprising. I'd gladly participate on a 96 kbps multiformat listening test, though.

100k would be nice. IMO this is the sweet spot for flash based players. 64k is just too low, and 128k is more then is needed judging from the last test.

I have another experience at 64 kbit/s. Vorbis is closer to Nero and WMA10pro is best performer.

As many people already mentioned 64 is not enough. Todays the encoders have a good quality at 96-128. So it will be interesting for me to see 80 kbit test. Also it would be useful to see how performs Vorbis against AAC and SBR vs LC performance.

Vote for 64 or 80.

I'll vote for 64kbps, but still want ~80.
Like Junon, Q25 is possible.

Three things:

1. At bitrates other than 64 kbps, we cannot compare the results to Microsoft's. Maybe this isn't such a big problem because you can never compare results 1:1 since other testers and samples were "used", anyways.
2. Considering a test at 64 kbps, do you think WMA Standard should be tested again? If not, comparing 3 contenders only could be a waste of resources.
3. At bitrates over 64 kbps (e.g. 80 or 96 kbps), is HE-AAC or LC-AAC the better choice?

I think it would be good if a listening test has some practical outcome, rather than being academic interest. Given this goal, codecs should be chosen that are relatively widely in use on portable devices (why would someone choose low bitrates when they have hundreds of gigabytes of storage at their disposal on their home PC?). Also, given that flash RAM storage is all the rage right now, typical portable capacities are coming in at 1 GB to 4 GB, so low bitrates are desired.

I voted for the 64 kbps test, but I would brefer to see something between 64 and 128 kbps, with codecs widely available NOW on flash RAM portable devices.

Assuming 1 GB of storage (multiply as appropriate), 128 kbps amounts to 17 hours of play time, 96 kbps is 23 hours, 80 kbps is 28 hours, and 64 kbps is 35 hours.

I'm not fussed whether it's VBR, ABR, or CBR, so long as it's the best quality at an agreed average bitrate (I expect it will be VBR or ABR). I'm not fussed whether it's more than one vendor's implementation of a format (a couple of MP3 contenders, a couple of AAC contenders, and a Vorbis contender?).

That's my thoughts.

Well, all contenders from the 48 kbps listening test are also "portable-friendly", with a lot of players supporting WMA Standard, some players supporting Ogg Vorbis (Cowon iAudio products, some Trekstor), some supporting HE-AAC (Nokia mobile phones and maybe others I don't know of) and one supporting WMA Professional (Zune AFAIK). Therefore, at 64 kbps, we could use the same contenders. At 80 or 96 kbps, maybe we could also use MP3.

I'd like to try 96k with several encoders, because the last 128k test showed that this bitrate is nearly transparent for the most users.

I agree with other posters requesting 96k or 80k test. Voted for 64k.

Between 96 and 80 kbps, I would be more happy to conduct an 80 kbps test, 96 kbps being too close to 128 kbps. Also high anchor in 48 kbps test already was almost transparent.

Not when some people participating in the last test gave high achor a 5 automatically. This is without really listening to the high anchor sample. What a shame. It really invalidates the high anchor score IMO.

I think that claim will be harder to defeat than you think. Even at 48k some of the samples were near transparency with HE-AAC, and I doubt that WMA Pro 64k is worse than HE-AAC 48k. Microsoft never claimed that 64k WMA is "CD quality" for every possible sample, remember?

Voted for 64 kbps

I am curious about WMA10 pro and HE-AAC.

I voted for 64kbps. Though, bitrate is close to the last test, I think it much more useful than 80/96kbps. Most of online video or music usually use 48, 64 or 128kbps. I rarely see ones broadcast with 80/96kbps.

My second preference would be mutiformat @128kbps but the test would be very difficult to do.

Well, the last multiformat test at 128 kbps is pretty recent.

I voted 64k test, because I would like to have 3.98 stable ready before a 128k test comes.

My ears aren't quite as good as guruboolez' (guruboolez's?), but there were only two samples where I ranked at least one of the 48k contenders above 3~3.5 (and it still wasn't a 5), which is certainly not transparent. The vast majority of the samples were awful.

64k could be better, but I don't really think it will be -significantly- better than 48.

In for 64.

My vote is for a 96 kbps test. While once a benchmark for "acceptable" quality, modern encoders have evolved to the point where 128 kbps is transparent to most listeners for most music. I'd be in interested in seeing whether the great work of developers has progressed audio compression to the point where transparency can be achieved under 100kbps (and which codecs are best at achieving this). This would also be of particular interest to owners of flash-based portables looking to have [close-to-] transparency at the lowest possible bitrate.

Hi!

I vote for 64kbs, but don't leave out Atrac3Plus this time! I use that on my minidisk, and I want to know how it comes out compared to others.



Trondis

Minidiscs don't support ATRAC3+ as far as I know.

Even though 64kbps seems close to 48, it's good to remember that it's a 33% increase. Those 16 bits probably go a lot farther (in terms of perception) than adding 16 bits to say 128. (That's speculation, but seems reasonable to me.)

Judging by the quality increase going from 48 to 64 kbps using ogg vorbis (aotuv b5) I'd say a 64 kbit test would be very interesting. As someone else pointed out it is a 33% increase of the bitrate and I would be curious to find out which codec would benefit the most of it. Since we just had a 48kbps test using the same contenders would make sense.

I would vote for 80 or 96 kbps multiformat test. Considering great results of current encoders at 128kbps, I am interested in their performance in 25% lower bitrate range. From my observations 64kbps still means "very good quality internet radio" for me, but that's not enough for portable usage and pleasant listening on headphones. I expect the results would be too much correlated to those from 48kbps listening test.

I voted for 64kbps, and I still would be interested in test that, mainly because vorbis vs HE-AAC. M$ claim wold also be interesting.

Nevertheless, I prefer an 80kbps one, because we can see the level of transparency in this bitrate (something not adequate for 64kbps), and test some Mp3 codecs, as lame. The 80kbps one would be very good for evaluate codecs for portable use. The downside is that we would need an pre-test for LC vs HE AAC.

For 80 kbps LC AAC there is already "pretest" comparing PNS and IS in Nero codec.

I'd simply include them both into the test. This way the user could see whether it made sense encoding to HE-AAC in the 80 kbps bitrate range. Especially portable players would benefit of that, since no or only very slight differences in quality clearly caused the less decoder-intensive LC-AAC to be way more desirable. For the Nero developers this would also be a useful indicator about the question if their encoder should default to HE-AAC or LC-AAC in case someone used the -q 0.3 commandline setting.


Edit:


Yes, but this test won't answer the question whether LC-AAC or HE-AAC is the better choice for a 80 kbps file.

I guess that question if PNS or IS give any advantage when used in LC should first be answered. Then testing LC against HE can be done. But it seems that question LC against HE at 80 kbps will be answered only after this 64 kbps test

That, of course, is true. Too bad the poll doesn't include any choice about the bitrate range to be used in case someone voted for a multiformat test instead of the MP3 one.

I don't vote for a 96 kbps test because in the 48 kbps test I could only identify the high anchor twice. It got a 4.5 and a 4.8 from me, and in the overall group results it received 4.7. I think most contenders at 96 kbps would be so close to the top, that statistically confident ranking of x being better than y is highly unlikely.

64kbps for me! aoTuV b5 with raised lowpass is a whole different ballgame than 48kbps Vorbis. 64kbps HE-AAC shouldn't surprise anybody (some folks claim they can pick out the SBR any day). WMA10Pro at 64kbps would be an interesting candidate, especially if it can be tested in VBR mode.

As for VBR WMA(pro), why can't one person (the test administrator) make all the encodes of full songs in VBR, cut them down to sample-size, and distribute all the [lossless re-encoded] pieces for testing? That would make a lot more sense than to force an encoder to do CBR mode when it is capable of a much more advanced mode of bit allocation.

Maybe because the test administrator doesn't have enough money to buy 20 CDs (it has to be CD or since I don't know of any online store that offers lossless downloads) or doesn't want to get into trouble by illegally downloading the tracks.

Would like to see 64kbs test. Tried the 80kbs AAC recent test and am amazed at what can be done with so little. My portable only supports mp3 and wma. Here's to progress!

cheers,
herefornow

I voted for 128kbps. My player doesn't support HE-AAC or WMA pro so I would like to see a multi-format test conducted again at 128kbps (or at 96kbps).

I receive about one iTunes gift card every odd week so it would be nice to know how that (128kbps AAC) is stacking up against other encoders at the same setting. Correct me if I am wrong but, wasn't the last 128kbps multi-format test conducted over a year ago (around the holiday season of 2005)?

What about a mixed test: ~128kbps mp3/mpc and ~96kbps aac/vorbis/wma? Should be a roughly competetive constellation.
Sure a big test as a series of mp3 encoders is to participate. But also one of major practical interest IMO which should be allowed some time to proceed.

A new Mp3 test could show if or how much iTunes + Fhg + Xing (Helix) mp3 encoders have improved since most average computer users (n00bs) still use software that uses those encoders.