The discussion below has become quite involved, as such things have a way of doing, and in an effort to make this thread useful for people who just want the goods, here is a summary:

Pentium 4 binary, 15% faster than mitiok, CRC-identical mp3 frames to mitiok, safe for archival: download

Pentium 4 binary, 21% faster than mitiok, NOT CRC-identical mp3 frames to mitiok. Bitrate varies +-0.5% from reference. Unavailable, takehiro says /Qrcd is bad. I still can't hear a difference, but who am I to argue with the lame devs.

Original post:

just tried it on my p4 2.1 ghz.

normal lame with --aps 4.1x realtime
p4 lame with --aps 4.85x realtime

which is about 15%.. Nice work!

What is the process in getting this build "certified" and approved by the offical LAME community?

Get one of the devs to commit some changes to the Makefile in CVS.

A good start would be if the original poster in this thread would upload the Makefile that he used somewhere.

The information below is depreciated.

Original post:

Nice work nebob. Any speed increase without sacrificing the quality is always welcomed!

Of course this only benefits P4 users and not those Athlon users, but the Athlon already beats the P4 in many areas so I guess this small concession is good enough .

Cheers
AgentMil

Would it be possible to create other LAME compiles optimized for specific CPU's? Say an 'Athlon XP *ONLY*' or a 'Pentium 3 *ONLY*'?

AthlonXP optimizations would only be possible with GCC 3.x, since ICL performs no AMD optimizations (for obvious reasons...)

The reason this binary runs only on the P4 is because of SSE2. At the moment, only the P4 has SSE2, although when the Hammer comes out, it will be able to run it too. If you try running it on anything else, it will crash.

The P3 and AthlonXP don't have a 'distinguishing feature' like this, so even binaries specifically optimized for them will run on other CPUs supporting their instruction set extensions (like the P4), albeit slower. In this way, you can create binaries that will run best on a specific one of these CPUs, but you cannot create binaries that will run *ONLY* on these CPUs and fail to do so on anything else.

One could always optimize for SSE1. That should work for the AthlonXP. For that matter, my 3.90.2 compile should run faster on athlons than on older processors as well... so that's kind of "optimized" for it also.

The only thing is that ICL doesn't support 3dnow, but that's not the only way to increase performance on Athlon chips.

You're right though about ICL not supporting specific optimization profiles for AMD chips.

I don't think that he wants a LAME version that only runs on an Athlon or P3, I think he just wants versions that are optimized for particular CPUs so that you can have the fastest version possible.

Are there any options available to speed up LAME on Athlons or P3s?

The Athlon has 3dNow! and 3dNow2!, both of which do similar things to SSE (and both of which would crash on an Intel chip). Based on some informal tests with gcc3.1.1, enabling 3dnow and 3dnow2 code generations does provide a significant speed increase.

Does anybody know where I can get an ICL-optimized compile for Linux? I'd like to compare the ICL to the gcc3.1.1-athlon-optimized head-to-head, but have only been able to find win32 ICL compiles thus far.

Nope... you'll probably have to roll your own. IIRC, the Intel Compiler was free of charge for Linux or something like that.

It's free of charge and unsupported for open source development, similar to Kylix, I think.

I just compiled this using ICL 6 latest build using these options "-O3 -p80 -rcd -tpp7 -xW" which basically means maximum optimizations with 80 bit floating point, fast float-to-int conversion and optimized for pentium 4 with SSE2. I also compiled another version with the same flags but with "-axW" instead of "-xW" so that it can be used with other processors, you can get the P4 only version at ftp://ftp.sonic.net/pub/users/bfrates/lame_p4.zip and the all CPUs but P4 optimized version @ ftp://ftp.sonic.net/pub/users/bfrates/lame_all_p4opt.zip

edit: btw, I haven't tested either of these builds because I don't have a P4 based computer (I personally don't like the P4 because IMO, it is inferior to the Athlon XP or Athlon MP)

is there any quality loss when using these compiles? how bout theoretical quality loss? how bout a size difference in mp3s made with these compiles and those made with Dibrom's or Mitok's compiles?

I belive there was a lengthy discussion over here on that issue. If I remember correctly, the conclusion was that there is a theoretical difference between files encoded with various optimized executables, that was not practically audible.

andy2kxp: -rcd causes a 4% size increase. Most people are not amicable to this. -p80 cancels many opportunities for vectorization as SSE2 handles at maximum packed 64bit FP.

SNYder: If you trust mitiok's compiles, the output of my binary is CRC identical to them. (meaning byte-for-byte the exact same thing) There seem to be a few bytes difference in the header, but that's it.

EDIT: I am an idiot

Hello!

I hadn't noticed the -rcd switch yet
and became quite curious about it.

So i couldn't avoid doing my own test with ICL4.5
(still the fastest code for my Athlon650).

--alt-preset fast standard:

-O2 -Qxi: 3.2354x , 200.1kbps (same output as mitioks compiles)
-O2 -Qxi -rcd: 3.3591 , 205.0kbps

2.5% bigger, 3.8% faster :eek:

Both files sound ok.

i'm not sure which compile to use now.
my ears don't hear a difference and the
increase in speed doesn't justify the
increase in size in my case either.

BUT...the holy quest for the mathematically perfect mp3-file....

i really don't know what to do.

Hmm, it appears you're right (it's free but unsupported for non-commercial use). After downloading and fighting with it a bit to get it to install (it comes only in RPM format, which sometimes causes problems on Debian systems), I can get it to compile simple programs, but it dies when compiling lame, giving all sorts of errors in /usr/include/string.h. Googling doesn't give me much info, but I'll keep looking around I suppose.

Better yet, if anyone has actually compiled LAME on Linux with icc, give me your makefile. =]

new compiles posted, use links in previous message to download, used flags "-O3 -tpp7 -xW" and "-O3 -tpp7 -axW" respectively.

You guys are going to LOVE this

Most people don't know it, but the lame distribution comes with i387, 3dnow and SSE FFT routines, all in optimized assembler. The first two get assembled and linked into libmp3lame, but the last one does not.

Today, I spent a bit of time poking around the source code, and after some additions and modifications, re-enabled the SSE FFT routine. The product of this is that lame no longer uses the i387 FFT, but the SSE one on qualified processors. This results in a performance increase, but more importantly results in a bitrate drop of about 2.4%. To compensate for the bitrate drop, I turned on /Qrcd resulting in a bitrate increase -- and the two cancelled each other out almost perfectly.

The outcome of all this is an executable that is 21% faster rather than 15% and produces files with bitrate within +-0.5% reference, of theoretically superior quality. I should put the qualifier here that the output is no longer CRC-identical to mitiok, but I can't tell the difference, and I'd wager a month's salary that you can't either.

The old binary with 'safe' performance increases will remain available for those who wish to air on the side of caution, and the new one is available right here.

Enjoy!

*P4 only, like last time. The 3dnow FFT is faster for the athlon anyway, and I don't have a P3 handy to do any testing.

your compile for P3 did not work on my system AT ALL
may be somthing is missing it crashs on my 550MHZ PIII

I'm sorry, I must have missed something, or I'm dumb, or both. Is there an AthlonXP-optimized lame binary for Windows that I can download somewhere? I couldn't quite tell from your last post if there are 3DNow(2) + SSE optimized executables floating around that I could get my hands on without having to compile anything. A cursory Google search didn't turn up anything but an AthlonXP-optimized Lame.dll.

mitiok's [lame 3.92] binaries are as fast as it gets for Athlons, at least as far as I've been able to tell.

http://home.pi.be/~mk442837/

Thanks for the clarification. Do these compiles use SSE? I don't see the SSE buzzword float by when the encoder starts on my AthlonXP, just MMX and 3DNow.

Actually not quite. The binary of 3.90.2 that I compiled is faster than Mitioks. I used a few switches which he didn't feel comfortable with or something like that. It's been so long that I don't even remember exactly, but one of them caused the output to be different than MSVC (something to do with rounding precision I believe), and one of them dropped compatibility with p5's, etc. Anyway, it provided a fairly decent speed increase on faster processors.

Btw... some quick testing with some very interesting results:

[size=9]

[size=9]

Hrmm. 3.90.2 is way faster..

I'll have to look into this some more to see exactly what's going on here. I knew that 3.92 took a bit of speed hit from disabling a few of the ICL options that were being used before (and which I further expanded upon) which caused a difference in bitrate between files encoded with these binaries and MSVC, but I didn't think the speed hit would be that great.

Also, I remember that ICL 5.0 produced slower binaries than ICL 4.5. I wonder if it's the same with ICL 6.0 or not.

Oh, and btw, this is on a 1.7ghz p4.

Also, it should be noted that all of the testing I did for the --alt-presets was on these higher bitrate ICL 4.5 compiles.. I'm starting to wonder if it was such a good idea for the LAME developers to make the changes to the Makefiles which lowered bitrate without doing extensive listening tests first. I did mention once quite awhile ago that with an MSVC compile (lower bitrate), I did hear a difference on fatboy.wav with --aps.

Edit:
Also interesting the bit about the joint stereo differences.. this was an old compile I had laying around, and it may not be exactly the same as the final 3.90.2. Anyone still have this thing available? CiTay?

Edit2:
Hrmm.. maybe it's time to make a list of "validated" compiles..

Fair enough, but what about lame 3.92? My XP runs seti in the corner, but from what testing I've done, mitiok's compile is the fastest.



It's important to distinguish between a compile that uses SSE, and a compile that uses an SSE FFT routine. Most compiles for the athlon xp make use of SSE in the C parts, like the psy modeling or quantization because the intel compiler is instructed to generate vectorized SSE code from the loops in these files. On the other hand, the FFT routines (a building block of mp3 encoding that must be run many times) are already written in hand optimized assembler. Different versions are available -- x87, 3dnow, and SSE. When you download the lame source and build it, the SSE version is left out; you must manually re-integrate it into the source. This is what the (ASM used) indicators that you see during encoding refer to.



Kapish?

Having said that, I can make an executable that does use this SSE on the athlon, but it'll be no faster (I've tried it). SSE was tacked on to the Athlon as an afterthought, not as an integral part of the pipeline like the P3/P4, and as a result, many SSE instructions have significantly higher latencies on the K7. Additionally, lame makes a decision on which routine to use (assuming you've linked in all of them) based on the CPU you have. If you have both 3dnow and SSE, it'll use 3dnow. If you have just SSE, it'll use that. If you have neither, it uses x87 FP.

Also, a clarification:

3dnow!'2' (aka pro aka +) = 3dnow! + SSE. It appears in the Athlon XP, and is just AMD rebranding the SSE instruction set so they don't get sued.

you mean the final 3.90.2 build? it is still at the download link in the 'recommended lame compile' thread. I noticed small bitrate and joint stereo differences in the test track I just did too, and wondered if I should stick with 3.90.2 instead of 3.92. I'm sure we'll all be interested in some more educated responses about which one to use now.

Wow, a lot more information than I was expecting, but I'm certainly grateful for it! Thanks for the detailed explanation, nebob. Well, maybe it wasn't that detailed from a hardcore programmer's standpoint, but that post was right at my threshold of understandability.

I do have a question about the part of your post when you say SSE optimizations do not benefit the Athlon XP... I don't doubt your conclusion that it offers no speed increase, but I thought I remembered reports of a P4-optimized DivX5 compile bringing speed increases to the AthlonXP due to SSE instructions. Perhaps this was because SSE gives a bigger speed increase with operations associated with video processing? Or maybe it was due to inept movie pirates falling for the placebo effect...

this is with my 1.4GHz Athlon (non-XP), differences aren't as big as the example posted above but are still there none the less. Interesting that 3.90.2 doesn't state (ASM used) for 3DNow! but it is still faster.

D:wavCrystal Method, TheTweekend>lame --alt-preset standard 1.wav 3.92.mp3
LAME version 3.92 MMX (http://www.mp3dev.org/)
CPU features: i387, MMX (ASM used), 3DNow! (ASM used)
Using polyphase lowpass filter, transition band: 18671 Hz - 19205 Hz
Encoding 1.wav to 3.92.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=2
Frame | CPU time/estim | REAL time/estim | play/CPU | ETA
14829/14832 (100%)| 2:06/ 2:06| 2:06/ 2:06| 3.0739x| 0:00
32 [ 14] *
128 [ 927] %************
160 [ 3607] %%%%%**********************************************
192 [ 4730] %%%%%%%%**********************************************************
224 [ 3033] %%%%%%%%***********************************
256 [ 1595] %%%%%%%%***************
320 [ 926] %%%%%%*******
average: 201.5 kbps LR: 2353 (15.86%) MS: 12479 (84.14%)

Writing LAME Tag...done

D:wavCrystal Method, TheTweekend>lame --alt-preset standard 1.wav 3.90.2.mp3
LAME version 3.90.2 MMX (http://www.mp3dev.org/)
-- Compiled at http://www.hydrogenaudio.org
-- Check this website for up to date information on the --alt-presets

CPU features: i387, MMX (ASM used), 3DNow!
Using polyphase lowpass filter, transition band: 18671 Hz - 19205 Hz
Encoding 1.wav to 3.90.2.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.4x) qval=2
Frame | CPU time/estim | REAL time/estim | play/CPU | ETA
14829/14832 (100%)| 1:59/ 1:59| 1:59/ 1:59| 3.2456x| 0:00
32 [ 14] *
128 [ 650] %********
160 [ 3022] %%%**************************************
192 [ 4945] %%%%%%%%%*********************************************************
224 [ 3284] %%%%%%%*************************************
256 [ 1865] %%%%%%%%*****************
320 [ 1052] %%%%%%%********
average: 206.7 kbps LR: 2309 (15.57%) MS: 12523 (84.43%)

Writing LAME Tag...done

Hrmm, I guess it is.

I should have checked the obvious At some point, this compile was removed shortly though, and I don't remember putting it back up. Ah well..

Just to clarify, the only compile that I'm recommending is still the 3.90.2 compile.

There hasn't been any significant changes to warrent moving to 3.92 if someone is only interested in using the --alt-presets, and instead it may be possible that some of the changes made to the compile time options were not optimal for quality and speed.

So, if you're asking for my take on it, use 3.90.2 (ICL) until otherwise noted.

Thanks, I believe I will

Can we be sure that quality hasn't *decreased* with the --alt-presets in v3.92 (Mitiok)? What I mean is, there isn't someone out there trying to sabotage the --alt-presets, right?

I've encoded a lot of files with v3.92 and wouldn't be too happy if quality was significantly worse. I really couldn't care less if it's a bit slower, nor if the bitrate varies a little (if those are the only differences, I won't think about it again).

I don't think anyone is actively trying to sabotage quality.

QUOTE

I don't think the quality would be significantly worse either. We're only talking about an average of 5-10kbps difference, so it couldn't be like a night and day difference. I think it might matter most in some of the more critical situations, like on serioustrouble and fatboy. The tuning for those samples was extremely precise. IIRC, a few weeks ago I went back and listened to Mitiok's 3.92 and it didn't sound as good as I remember it should have... I'm not sure what that means yet. So all I'm saying for now, until I have more time to look into the matter, is just use 3.90.2. That's the only compile I have ever recommended officially really..

I don't mean they don't benefit the XP absolutely (they do), I just mean they aren't any faster than 3dnow. Lame uses 3dnow if you have it by default, and forcing it to use SSE instead would not bring any advantage. If you took out 3dnow and made a comparison between x87 FP and SSE, certainly the SSE routines would be faster, though not by as great a deal as the P3/4 because of the Athlon's superior FPU.

Athlon XP 1700+
Oasis - Morning Glory

Hmmph, I'm not posting all that, my CPU speed is too embarassing
Here's a "short" sample comparison. The bitrate differences are quite striking.

DO NEVER USE -Qrcd or -rcd option. see this mail on lame-dev ML.

http://www.geocrawler.com/archives/3/7323/2002/2/0/7856153/

edit: fix typo

Ok. Thanks.

Away with -Qrcd.

You learn something new every day.

Think that URL is dead.

mitiok's site has been at http://mitiok.cjb.net/ for the past couple of years. There is also a mirror.
The official archive has a lot more choices, like specialized DLLs that will work with programs (like PlexTools Pro) that otherwise do not support --presets.