• LAME 3.98b5: --vbr-new is not needed anymore (used by default)
• iTunes: I don't see any reason to not use the highest quality level.
• Fraunhofer: same than iTunes
• GoGo: I would trust AlexB's advice on GoGo (he has an experience I don't have)
• Real: As you wish (I would personally avoid to install Real and use the CLI tool instead). VBR seems to be the way to go.

I finished on my side a personnal MP3 listening test based on classical samples which features four encoders. I can give the average bitrate of 150 files (16 hours of music), unfortunately not representative of compressed and/or popular music:
• LAME -V5 = 128,73 kbps
• iTunes 128 VBR HQ = 131,54 kbps
• Helix -V65 = 129,12 kbps
• Fhg -m3 -q1 = 127,86 kbps
I'm lucky that VBR settings are so much comparable bitrate-wise. Fhg seems to be downgraded to -m4 with louder music to match the targeted bitrate range; Helix can easily be adjusted if needed; iTunes needs to be tested by someone else (if it's not already done).

For information: both iTunes & Fhg are giving pretty good (but not constant) results here.

Here are my results. For iTunes and RealPlayer I used the standard GUI programs. For the command line encoders I used Speeks' Batchenc frontend. The WMP11 mp3 codec was accessed by Nyaochi's Acmenc and Speek's Batchenc. The test bench was a 2.8 GHz P4. Before the speed tests I tried to disable all unnecessary background processes. The source files were in wave format on one HD and the target HD was separate. I measured the encoding times with a digital stopwatch. I didn't run several passes that would have been needed for lowering the error margin, but I think the results are accurate enough for this purpose. For measuring the bitrates I used Encspot Pro's "Complete Scan" feature (some other programs may not show correct bitrates with files that don't contain a Xing header).

The test file set consists of 25 selected complete tracks of various genres. The total duration is one hour, 31 minutes and 34 seconds.

The encoding speeds and average bitrates:


The individual bitrates of the VBR files:


A chart of the VBR bitrates:


Some readers may also find the following table interesting (Album = the 25-track test set):



I uploaded the test data here (Excel file & Encspot Pro reports):
http://www.hydrogenaudio.org/forums/index....st&p=515301

Surprisingly the quality options have almost no effect to the encoding speeds. The very small encoding time differences may be caused by other things as well (I did only one pass). I wonder if these settings do anything, even though there was a very small bitrate increase.

The Smart Encoding Adjustments and Filter Frequencies Below 10 Hz options seem to slightly affect the speed so perhaps they actually do something.

... I think the setting names Apple has chosen suggest that the best quality would be achieved @ 128 kbps VBR, Quality: Highest, Stereo Mode: Joint Stereo, Smart Encoding Adjustments & Filter Frequencies Below 10 Hz: enabled.

... Fraunhofer q0/q1 difference: I won't test it on my side (I tried and -bored- I quickly gave up last week). I strongly believe that we should use the switch called “high quality”; even if didn't improved anything according to your test, I didn't lowered the quality either. We should always test the HQ mode and let people use the fastest mode if they're ready to take some risks (even minimal ones) by maximizing encoding speed.

I agree that Fhg speed is pretty impressive with -q0 but I'd like to give the same weapons to Fhg's encoder (HQ mode).

Some samples were slightly worse and some were slightly better in the default mode.

So people are interested after all in an MP3 test... Problem is that we never agreed on what settings to use for Helix / Real for example.

The problem is that not even developers know what to recommend and choosing some "weird" settings without doing listening tests first can lead to unfairness.

Could we not just drop this implementation from the list of contenders?

So, does anyone mind using Real to encode 128 kbps VBR?

To be honest, I would personally use Real instead of dropping the encoder just because we cannot agree on parameters. Real uses Helix and doesn't offer any special settings that might mess things up more than they help

Are you already locked in on the encoders? I'm a big fan of WMAstd 2-pass vbr ~128kb.

It is a MP3 Listening Test.

Isn't Real using Helix? Maybe Real uses some switches that should optimize the quality but have impact on the speed.

I would definitely conduct the test if we finally decided which encoders to use. By the way, I once contacted one of the Helix developers and he wasn't able to recommend any setting.

Here are my results. For iTunes and RealPlayer I used the standard GUI programs. For the command line encoders I used Speeks' Batchenc frontend. The WMP11 mp3 codec was accessed by Nyaochi's Acmenc and Speek's Batchenc. The test bench was a 2.8 GHz P4. Before the speed tests I tried to disable all unnecessary background processes. The source files were in wave format on one HD and the target HD was separate. I measured the encoding times with a digital stopwatch. I didn't run several passes that would have been needed for lowering the error margin, but I think the results are accurate enough for this purpose. For measuring the bitrates I used Encspot Pro's "Complete Scan" feature (some other programs may not show correct bitrates with files that don't contain a Xing header).

The test file set consists of 25 selected complete tracks of various genres. The total duration is one hour, 31 minutes and 34 seconds.

The encoding speeds and average bitrates:


The individual bitrates of the VBR files:


A chart of the VBR bitrates:


Some readers may also find the following table interesting (Album = the 25-track test set):



I uploaded the test data here (Excel file & Encspot Pro reports):
http://www.hydrogenaudio.org/forums/index....st&p=515301

As I promised, I tried to do a "quick pretest" with a few samples using foobar's ABX module, but that didn't work. The codecs' quality didn't have immediately obvious differences and the results would have been almost arbitary depending on the used samples. So I felt kind of obligated to do a full-scale personal test.

I used the following codec settings:
FhGc = FIIS 3.4 CBR 128 kbps
FhGv = FIIS 4.1 VBR -br 0 -m 4 -vbri -ofl
FhGv1 = FIIS 4.1 VBR -br 0 -m 4 -q 1 -vbri -ofl
GoG = GoGo 3.13 ABR -a -b 137
GoG2 = GoGo 3.13 ABR -a -b 137 -q 2

Edit: I forgot to mention that I used FhG Mp3sdecoder v. 1.3 for decoding the samples. (Looks like the -ofl option worked. ABC-HR Java didn't adjust the starting position of the FhG VBR samples)

After seeing the results of my previous speed and bitrate test I upped the GoGo ABR bitrate setting from 135 to 137 so that it would better match the codec average. FhG VBR produced still a bit higher average bitrate with the now tested new samples (134 kbps vs. 131 kbps).

I tested 30 varied samples. All samples are created by myself and most of them are brand new. About half of them are from Vangelis' Alexander soundtrack album, which has a lot of interesting music (it's a mixture of classical, electronic and ethnic genres) and I wanted to try if that material would be useful for testing codecs.

The reference samples and my ABC-HR result files are available in this link:
http://rapidshare.com/files/55027379/alexb_test.zip (about 76 MB)

Here are the test results:


Even though I did the test properly, I wouldn't trust the results too much.

Mostly I was able to differentiate the contenders without ABXing because of the lowpass. All encoders seem to use an about 15-16 kHz lowpass frequency at this bitrate/quality level. Beyond that the test was a tough call. The codecs were generally quite good and the slight lowpass didn't make the samples annoying (to me, it seems to be detectable only in a direct comparison with the reference).

I didn't want to spend several days with this test so I rated the samples rather quickly and it is possible that a new test with the same samples would produce slightly different results.

GoGo failed misarably on one killer sample, the sample number 29 (also FhG was bad with this sample, but better than GoGo). On the other hand FhG's VBR has obvious problems with some other samples.

I can make the following conclusions:
All encoders are tied and the higher quality options didn't make the encoders better.

Edit: Fixed a couple of typos and added the MP3 decoder info.

OK, so let's assume the Helix problem is solved (it is actually: if I get feedback from the guys from the audio department I will use their suggestions, otherwise the settings mentioned in this thread)... This leaves us with the last problem: Gogo. The question is whether or not to use an additional -q parameter.

Thinking about it right now, I am wondering if we should even include Gogo in the test... The problem is:

- CBR would tie Gogo's hands and is unfair
- VBR is not mature in Gogo's LAME "engine"
- ABR gives different results when encoding samples and when encoding the corresponding full track - we had this problem when it came to WMA's 2-pass VBR encoding

Seeing that FhG CBR 128 kbps without any quality switch is almost 80x, I would tend to use that instead of Gogo.

BTW, is Gogo 3.13 based on LAME 3.88 or LAME 3.9x? The website states the latter, but when you run the build from RW, it says that it's based on 3.88.

My understanding of ABR is that the the encoder also "memorizes" what bitrate it used on previous frames and allocates bits for future frames taking this "memory" into consideration - sort of like the bit reservoir works, just that it allows larger fluctuations. The problem I see is that an encoded sample might have another bit allocation than if you were to encode the whole track and then cut out the part you want to test.

... ABR is one pass? How can it approximate a desired bitrate then? IMHO it needs to do a quick look on the file at least once.

... ABR is one pass. Well, here is how I think it works, anyone please confirm or correct me. In CBR there is something called bit reservoir. When a frame is to be encoded to say 128 kbps, the encoder may not match this exactly. In some frames there will be empty space, and in some the data just won't fit. So what the encoder can do when there is to much data, is to move this to a frame where there is empty space. How this works in detail I have no clue, but it is referred to as the bit reservoir.

The problem is that there is not always enough space in the bit reservoir when doing CBR. My guess is that in ABR this is skipped and the frame size is instead adapted to what the encoder wants.

Anyone wanna comfirm this, or correct me?

ABR is the same thing as CBR with an unlimited bit reservoir (emulated by changing frame sizes)

- ABR gives different results when encoding samples and when encoding the corresponding full track - we had this problem when it came to WMA's 2-pass VBR encoding

couldn't you encode the entire song, then cut out the sample portion?

I am particularly interested in ABR since that is what I use. My ipod seems to choke on VBR

After seeing the recent LAME 3.98 related posts I think we should test LAME 3.97 and LAME 3.98.