LAME: different compilers producing significantly different bitstreams, Split from “LAME 3.99 is out” Options

[Alex B]

The new 32-bit lame.exe seems to work fine for me on XP Pro SP3 (I have an AMD Phenom II CPU that supports SSE2), but it produces entirely different output than LAME 3.99.3 from the previous "main" bundle. I did not expect that. As far as I can see, the change log does not indicate any further "quality tweaking".

In some cases 3.99.4 seems to produce significantly smaller VBR bitrates than 3.99.3. I tested the settings -V1, -V5 and -V8. When the source tracks were quiet and uncompressed (classical, ambient, etc) the bit rates were 3-12% smaller than before. When the source tracks were loud and compressed (hard rock, metal, etc) the bitrates were about the same as before (or just very slightly smaller).

This post has been edited by Alex B: Jan 28 2012, 16:08

[halb27]

With my test set 3.99.4 produces an average bitrate of 254 (was 259 for 3.99.3), 186 (190), and 125 (126) kbps for -V0, -V2, and -V5.

It would be kind if the Lame dev(s) could give us some information about the changes.

[Alex B]

I uploaded a sample: http://www.hydrogenaudio.org/forums/index....showtopic=93149

In some cases LAME 3.99.4 produces significantly smaller VBR bitrates than 3.99.3.

For example, a 30 s classical piano sample: [attachment=6862:Liszt.flac]
LAME 3.99.4 -V1 => 221 kbps
LAME 3.99.4 -V5 => 106 kbps
LAME 3.99.4 -V8 => 91 kbps

LAME 3.99.3 -V1 => 246 kbps
LAME 3.99.3 -V5 => 119 kbps
LAME 3.99.3 -V8 => 102 kbps

[lvqcl]

Different compilers/options IMHO.

[Alex B]

At least John did not change the announced compiler version:

From: http://www.rarewares.org/mp3-lame-bundle.php

LAME 3.99.4
2012-01-25

Bundle compiled with Intel Compiler 12.1.
Download (754kB)

From: http://webcache.googleusercontent.com/sear...lame-bundle.php

LAME 3.99.3
2011-11-28

Bundle compiled with Intel Compiler 12.1.
Download (630kB)

Also the size difference is interesting:

32-bit 3.99.3 lame.exe = 626 KB
32-bit 3.99.4 lame.exe = 829 KB

[JJZolx]

In some cases LAME 3.99.4 produces significantly smaller VBR bitrates than 3.99.3.

Some smaller, some larger. Occasionally the difference is significant. This has been noted previously. Overall, my Mp3 library is slightly larger when the master FLAC library is transcoded at -V2 using 3.99.3 vs. 3.98.4.

[lvqcl]

From RareWares main page: "Compiles are now MSVC9/ICL12.1 to retain Windows 2000 compatibility." (emphasis mine)

Also, here are my MSVS2010 compiles: http://depositfiles.com/files/bje5um8ql (warning: captcha etc... ).

This post has been edited by lvqcl: Jan 29 2012, 00:25

[bbrabant]

Also, here are my MSVS2010 compiles:

Thanks lvqcl, no problem running your compiles on my system

[Alex B]

Some smaller, some larger. Occasionally the difference is significant. This has been noted previously. Overall, my Mp3 library is slightly larger when the master FLAC library is transcoded at -V2 using 3.99.3 vs. 3.98.4.

We are discussing about 3.99.3 vs 3.99.4, not about 3.98.4 vs 3.99.3/3.99.4.

And my point is this:

... it produces entirely different output than LAME 3.99.3 from the previous "main" bundle. I did not expect that. As far as I can see, the change log does not indicate any further "quality tweaking".

The announced changes:

LAME 3.99.4 January 25 2012
Robert Hegemann
- Fix for tracker item [ 3475581 ] lame crashes at .w64 input file
- Addressing things brought to attention by tracker item [ 3463197 ] 3.99.x problem WFED and PCST frames
- WFED and PCST frames can now be added, to tag podcasts iTunes recognizes
- USER frames are now supported
- COMM frames can now have a description, when passed via --tv "COMM=description=full text"
- possible divide-by-zero exception should be fixed
- adding malformed user-defined-frames could result in abnormal program termination, fixed

[JJZolx]

Ok, then back to lvqcl: Why would a different compilation of the LAME encoder create different audio file encodings?

[Alex B]

Why would a different compilation of the LAME encoder create different audio file encodings?

I'd say that small differences are expected, but not to this extent:

For example, a 30 s classical piano sample: [attachment=6862:Liszt.flac]
LAME 3.99.4 -V1 => 221 kbps
LAME 3.99.4 -V5 => 106 kbps
LAME 3.99.4 -V8 => 91 kbps

LAME 3.99.3 -V1 => 246 kbps
LAME 3.99.3 -V5 => 119 kbps
LAME 3.99.3 -V8 => 102 kbps

and the same sample:

lvqcl's LAME 3.99.4 -V1 => 207 kbps
lvqcl's LAME 3.99.4 -V5 => 97 kbps
lvqcl's LAME 3.99.4 -V8 => 84 kbps

lvqcl's LAME 3.99.3 -V1 => 207 kbps
lvqcl's LAME 3.99.3 -V5 => 97 kbps
lvqcl's LAME 3.99.3 -V8 => 84 kbps

This is quite surprising.

EDIT: added lvqcl's 3.99.3 compile.

This post has been edited by Alex B: Jan 29 2012, 01:21

[C.R.Helmrich]

Indeed. How was the following done, if I may ask?

possible divide-by-zero exception should be fixed

I think doing this (or not doing this?) via modification of floating-point registers controlling exceptions, different compilers might produce different results. But I'm not an expert on this.

Chris

This post has been edited by C.R.Helmrich: Jan 29 2012, 02:13

[lvqcl]

It seems that the relevant changes are somewhere in quantize.c: http://lame.cvs.sourceforge.net/viewvc/lam...athrev=lame3_99

*pxmin++ = xmin; -> *pxmin++ = Max(xmin, 1e-20f); etc.

[JJZolx]

It seems that the relevant changes are somewhere in quantize.c:

I ran a test similar as that run by Alex B, except with a sampling of files from my FLAC library, and only encoded at -V2, and found similar results. Unlike Alex, I did find numerous cases where the RareWares 3.99.4 produced higher bitrate (larger) files than 3.99.3.

What I don't understand about this explanation is that in my tests your compiles of 3.99.3 and 3.99.4 produced essentially the same bitrate in the output, unlike the RareWares compiles of the same releases. And in nearly all cases the bitrates were lower than those produced by either RareWares version.

Some examples (all at -V2)

CODE

Katia Lebeque - Rhapsody in Blue.mp3
RareWares LAME 3.99.4  196 kbps
RareWares LAME 3.99.3  190 kbps
lvcql's   LAME 3.99.4  178 kbps
lvcql's   LAME 3.99.3  178 kbps

The Black Keys - Set You Free.mp3
RareWares LAME 3.99.4  186 kbps
RareWares LAME 3.99.3  202 kbps
lvcql's   LAME 3.99.4  185 kbps
lvcql's   LAME 3.99.3  185 kbps

Yngwie Malmsteen - Leviathan.mp3
RareWares LAME 3.99.4  189 kbps
RareWares LAME 3.99.3  187 kbps
lvcql's   LAME 3.99.4  185 kbps
lvcql's   LAME 3.99.3  185 kbps

[john33]

Sorry, guys, there was a compiler option error on my part. New 32 bit compiles uploaded that should correct this. These will also revert the apparent bit differences, I believe.

[halb27]

Everything's fine now. Exactly the same average bitrate for 3.99.3 and 3.99.4 using -V0, -V2, and -V5 on my test set.
Sorry I thought there were changes in the source code.

This post has been edited by halb27: Jan 30 2012, 14:54

[john33]

Everything's fine now. Exactly the same average bitrate for 3.99.3 and 3.99.4 using -V0, -V2, and -V5 on my test set.
Sorry I thought there were changes in the source code.

Thanks for the confirmation, and sorry to be the cause of confusion!!

[godrick]

Does the nature of the compiler error require users to reconvert any files that used the previous 3.99.4 compiled version, or is the audio quality unaffected?

[john33]

Does the nature of the compiler error require users to reconvert any files that used the previous 3.99.4 compiled version, or is the audio quality unaffected?

There was a small difference in bit allocation which I would very much doubt is in any way audible but I would recommend you test a couple of tracks to determine whether you can detect any difference.

[Alex B]

Thanks John.

The new 32-bit exe compile seems to produce identical bitrates with the 3.99.3 compile from the previous Rarewares "main bundle" (I tested only the Listz sample at -V1, -V5 and -V8). Actually, at -V5 both versions produced identical MP3 data. At -V8 and -V1 the resulting files were not exactly the same.

Regarding my test sample, I am still curious about the rather big difference between the Rarewares and ivqcl's compile:

-V1: 246 vs 207 kbps
-V5: 119 vs 97 kbps
-V8: 102 vs 84 kbps

This post has been edited by Alex B: Jan 30 2012, 17:22

[john33]

Thanks John.

The new 32-bit exe compile seems to produce identical bitrates with the 3.99.3 compile from the previous Rarewares "main bundle" (I tested only the Listz sample at -V1, -V5 and -V8). Actually, at -V5 both versions produced identical MP3 data. At -V8 and -V1 the resulting files were not exactly the same.

Regarding my test sample, I am still curious about the rather big difference between the Rarewares and ivqcl's compile:

-V1: 246 vs 207 kbps
-V5: 119 vs 97 kbps
-V8: 102 vs 84 kbps

Hmmm, I am aware that there are differences between VC9/VC10/ICL compiles in terms of the resulting bitrate, but those discrepancies do seem rather larger than I would have expected. Having said that, I can't offer any other explanation either! I did just check that I had the correct 3.99.4 release code and I do, so that's not a possibility either. If anyone else has any thoughts, let's hear them, please.

[robert]

Thanks John.

The new 32-bit exe compile seems to produce identical bitrates with the 3.99.3 compile from the previous Rarewares "main bundle" (I tested only the Listz sample at -V1, -V5 and -V8). Actually, at -V5 both versions produced identical MP3 data. At -V8 and -V1 the resulting files were not exactly the same.

Regarding my test sample, I am still curious about the rather big difference between the Rarewares and ivqcl's compile:

-V1: 246 vs 207 kbps
-V5: 119 vs 97 kbps
-V8: 102 vs 84 kbps

I suspect it has something to do with LAME's usage of libsndfile.
Alex, can you please decode your testsample with foobar first and feed it into LAME without libsndfile support?

[Alex B]

I don't think libsndfile is involved. I used foobar2000 as a frontend. The encoders received decoded PCM through STDIN.

Here is an example of fb2k's log:

CODE

CLI encoder: lame.exe
Destination file: F:\Test\lame\Liszt.mp3
Encoder stream format: 44100Hz / 2ch / 16bps
Command line: "C:\Soft\LAME\lame.exe" -S --noreplaygain -V 5 - "Liszt.mp3"
Working folder: F:\Test\lame\
Encoder process still running, waiting...
Encoder process terminated cleanly.
Track converted successfully.
Total encoding time: 0:01.109, 27.05x realtime

EDIT

Actually, I think libsndfile is included only in the special rarewares compile. I tested the first, "main", bundle.

This post has been edited by Alex B: Jan 30 2012, 18:51

[mixminus1]

I can confirm AlexB's findings vis a vis the bitrate differences between the Rarewares compile of 3.99.4 and lvqcl's on quieter/"easier" to encode material.

I chose two tracks, one with a FLAC -8 bitrate of 461 kb/s ("Shaolin Spirit (duo)", acoustic guitar), and one with a FLAC -8 bitrate of 1105 kb/s ("Rolodex Propaganda", rock).

All encodes were done from the command line in Windows XP SP3 (running on a 3.1 GHz i3-2100) from WAVs:

CODE

Shaolin Spirit (duo)
V2 V5
3.99.4-20120130 (Rarewares) 170.3 104.8
3.99.4 lvqcl 159.8 95.1

Rolodex Propaganda
V2 V5
3.99.4-20120130 (Rarewares) 200.1 135.8
3.99.4 lvqcl 199.3 135.6

I couldn't detect any audible differences between the Rarewares and lvqcl encodes at either -V setting (although neither were transparent at -V5 with "Rolodex Propaganda", but not too suprising...punk-ish rock with open hi-hats ).

Edit: After confirming that 30s samples of the above tracks demonstrated the same bitrate differences/similarities, I added them to AlexB's previously-linked upload thread.

This post has been edited by mixminus1: Jan 30 2012, 22:07

[DigitalMan]

Assume 64 bit compile is unaffected? Not at machine; can't test ATM.