lame 3.98.4, 3.99 alpha, 32- and 64-bit builds Options

[john33]

Can someone tell me why I'm getting different audiostreams with compiles by tsnr and john33??

Differences found: 4389914 sample(s), starting at 2.1563265 second(s), peak: 0.0983066 at 149.9082993 second(s), 2ch

Also encoding is about 1.5x faster with tsnr's compile

Parameters (foobar2000):

--silent -b 320 -q 0 --noreplaygain - %d

As already mentioned, different compilers and compiler options will produce differing results, but I don't believe anyone yet has been able to hear a difference.

I'd love to know what the compiler options used were that make that compile 1.5 times faster.

Edit: Well, having just run tsnr's 32 bit compile on my development system (e4300 @ 3.0GHz, 4GB RAM, XP Pro SP3) there is virtually no difference in speed between that compile and my own. What system are you using to get 1.5x faster running?

This post has been edited by john33: Mar 23 2010, 18:19

[Steve Forte Rio]

different compilers and compiler options will produce differing results

But I thought that the same LAME version/parameters will result in absolutely the same stream
And now I wonder what the differences (absolute, not audible!) can be between streams produced by different compiles...

What system are you using to get 1.5x faster running?

Pentium 4 631 3,75 GHz (MMX, SSE3), 2Gb DDR2-667 RAM, XP Pro SP3 32bit

This post has been edited by Steve Forte Rio: Mar 23 2010, 18:40

[Alex B]

It is important to minimize all external factors that may affect the encoding speed before making any conclusions.

Just a few days ago I compared the encoding speeds of two 3.98.2 compiles: the MS compile from http://lame.bakerweb.biz and John's Intel compile from rarewares.

I used foobar2000 on XP SP3 for encoding a set of 50 various FLAC tracks. The destination folder was set to be on a separate drive. Foobar was set to use all four cores of my AMD Phenom II CPU. Before testing I disabled the virus scanner and closed all other programs. I encoded the complete set with both encoders twice in a row and noted only the latter runs in order to avoid differences that could be caused by cached file data.

The results:

Rarewares
Total encoding time: 1:04.188, 118.47x realtime

Bakerweb
Total encoding time: 1:23.828, 90.71x realtime

[lvqcl]

WinXP 32bit. Core2 Duo E4600. lame -S --noreplaygain -b320 -q0.

rarewares:

Kernel Time = 0.140 = 00:00:00.140 = 0%
User Time = 30.156 = 00:00:30.156 = 98%
Process Time = 30.296 = 00:00:30.296 = 99%
Global Time = 30.531 = 00:00:30.531 = 100%

tsnr:

Kernel Time = 0.156 = 00:00:00.156 = 0%
User Time = 23.875 = 00:00:23.875 = 98%
Process Time = 24.031 = 00:00:24.031 = 98%
Global Time = 24.328 = 00:00:24.328 = 100%

1.25x faster.

[Alex B]

OK, I'll repeat the my test using the new 32-bit 3.98.4 compiles. I still have the test files ready for use. Give me a few minutes.


john33,

Regarding AMD vs Intel, do you think that your binaries favour Intel processors?

This post has been edited by Alex B: Mar 23 2010, 18:57

[robert]

Interesting read:
Does Intel's compiler cripple AMD performance?

[john33]

I just re-tested on a q6600 with 8GB and XP x64 and, again, the speed difference was minimal. My compiles use the /arch:IA32 command line option.

[Steve Forte Rio]

Everyone talks about the differences in performance but nobody cares, what is the difference between the output files and where it come from??

This post has been edited by Steve Forte Rio: Mar 23 2010, 19:22

[Alex B]

The results using -V2 --noreplaygain. I forgot to mention in my above description that I also always deleted the resulting files before each new run so that the destination drive would be in about the same state.


tsnr's compile:

The first run: Total encoding time: 1:03.266, 120.19x realtime

The second run in a row: Total encoding time: 1:01.578, 123.49x realtime


john33's compile:

The first run: Total encoding time: 1:02.015, 122.62x realtime

The second run in a row: Total encoding time: 1:00.484, 125.72x realtime


On my PC john's compile was slighly faster, but the difference is not significant. It may be caused by anything.

[john33]

Everyone talks about the differences in performance but nobody cares, what is the difference between the output files and where it come from??

From the fact that different compilers produce different object code. In this particular case, I believe that the difference is mainly due the use of different compiler options. Changing the compiler options allowed me to generate a compile that produces an encoded mp3 that is more similar in characteristics, according to the histogram, to tsnr's compile. Basically, the results of math calculations vary slightly when using SSE, SSE2, SSE3, SSSE3, etc.

The differences in encoded files resulting from encoders generated with different compilers has long been discussed but, as I said, I don't believe anyone has ever claimed to hear a difference and that is all that matters.

[timcupery]

does the audio stream itself differ between the two compiles? and by this I mean, if you decode both files to wav files and compare those, are the wav files bit-for-bit equal?

[db1989]

See post #24.

[Steve Forte Rio]

Audio steams are different, so the wave files will be different too and they can not be "bit-for-bit equal".

This post has been edited by Steve Forte Rio: Mar 23 2010, 21:48

[john33]

Whether, or not, the decoded streams are identical is somewhat irrelevant. People are getting hung up on whether the encoded files are identical and, as has been said many times before, different compilers and compiler options will necessarily produce different encoded results. The only concern is whether, or not, that difference can be heard and, so far, no one has even attempted to claim that.

At the end of the day, take a wave file and encode it with a variety of different encoders - different codecs - and each of them will lack a significant amount of data when compared with the original wave file. That is a given and no one would contest that. At the expense of repetition, the only issue is whether a difference can be heard under reasonable listening conditions.

If paranoia has truly set in, then use a lossless codec, otherwise demonstrate via testing that you can discern a difference. We are dealing with a lossy codec here! A lot of data is being thrown away and only when you can genuinely hear that something is lacking is there an issue!

[db1989]

People are getting hung up on whether the encoded files are identical

I mean no offence to Steve, but he seems to be the only person who is concerned.

[Light-Fire]

does the audio stream itself differ between the two compiles? and by this I mean, if you decode both files to wav files and compare those, are the wav files bit-for-bit equal?

Audio steams are different, so the wave files will be different too and they can not be "bit-for-bit equal".

We are talking LOSSY encoding here kids. It doesn't matter if the compiles differ and streams are not bit-for-bit equal. What counts is:

...whether, or not, that difference can be heard and, so far, no one has even attempted to claim that...

[saratoga]

Audio steams are different, so the wave files will be different too and they can not be "bit-for-bit equal".

If you want bit for bit identical, you need to use lossless. Lossy codecs use floating point values, which are only approximate. Different compiles, different CPUs, different operating systems, etc can and do give different output. Thats how computers are supposed to work

If you don't like it, don't use MP3.

[gottogo99]

Another question, for john33: why are there two slightly different versions of lame.exe in the new 3.98.4? The one with libsndfile is one minute newer and 5KB smaller. Seems like they should be the same but I'm no programmer.

[Andavari]

http://www.virustotal.com didn't find anything scary in the exe files. Only "Symantec 20091.2.0.41, 2010.03.23" thinks that the files are suspicous.

The Symantec scanner on VirusTotal tags allot of stuff as "Suspicious.insight" for some particular reason, even if it's deemed clean by all the other scanners.

I wonder if those "MediaFire" compiles were just taken from Rarewares to begin with since LAME 3.98.4 is on Rarewares right now.

[viktor]

sadly x64 is about 15% slower than the x86 built with asm optimizations. it would be great if we could take advantage of the advanced architecture sometime.

http://www.phoronix.com/data/img/results/m...910_final/4.png

xvid x64 is 20% faster than x86, x264 x64 is 15% faster, but that could be due to the lack of optimizations in the x86 version...

[john33]

.....
I wonder if those "MediaFire" compiles were just taken from Rarewares to begin with since LAME 3.98.4 is on Rarewares right now.

No, they weren't, they were made available during the night (my time) before I'd even begun to build them.

[john33]

Another question, for john33: why are there two slightly different versions of lame.exe in the new 3.98.4? The one with libsndfile is one minute newer and 5KB smaller. Seems like they should be the same but I'm no programmer.

There are, in fact, three versions currently offered:

The standard build with the dll and the documentation, a modified, unofficial build that will accept flat wav input and the version that uses the libsndfile dll that accepts FLAC input in addition to the usual inputs. The option to compile against libsndfile libraries to extend the input file options has existed for many years. It recently became more useful as it now takes FLAC input, along with many others. The .exe files differ because the standard one uses the native file reading routines provided within LAME whereas the libsndfile version hands input file handling to the libsndfile-1.dll. For like-for-like input files, the output will be identical for all versions.

[Steve Forte Rio]

Ok, people. Now I see that really getting too worried about such negligible things And it really looks like some kind of paranoia
But it was a metter of principle for me.
Well, now I understand where those differences come from (floating-point values, approximation, etc.), so they are absolutely inaudible (maybe you will find it ridiculous, but I even tried to ABX them )
Thank you

This post has been edited by Steve Forte Rio: Mar 24 2010, 11:26

[john33]

...(maybe you will find it ridiculous, but I even tried to ABX them )
Thank you

Not ridiculous at all, the best way to establish whether the differences matter.

[gottogo99]

There are, in fact, three versions currently offered:

The standard build with the dll and the documentation, a modified, unofficial build that will accept flat wav input and the version that uses the libsndfile dll that accepts FLAC input in addition to the usual inputs. The option to compile against libsndfile libraries to extend the input file options has existed for many years. It recently became more useful as it now takes FLAC input, along with many others. The .exe files differ because the standard one uses the native file reading routines provided within LAME whereas the libsndfile version hands input file handling to the libsndfile-1.dll. For like-for-like input files, the output will be identical for all versions.

Thanks for clearing that up. I figured there was a reason.