While trying to figure out what settings to use to encode my CD collection, I tried to do a blind test and pick out the WAV from a pair of Lame encoded copies using -V5 --vbr-new and -V2 --vbr-new.

Well, I ripped a number of tracks and can just barely (I think?) hear the difference between the WAV and -V5 --vbr-new file. But I gotta be listening to them one right after another to do it. Albeit, it is my computer's onboard soundcard processing the audio, but my headphones are very revealing Beyerdynamic DT770s, for whatever that's worth.

I even did a spectrum analysis in Cooledit, then imported screenshots into Photoshop as Difference Layers to see if the difference was noticeable visually. Frankly, aside from the obvious low-pass on the Lame encoded files, there's not a whole lot of difference between any of them. From my analyssi, it seems the -V5 Lame file has a tendency to drop out frequencies from soft sounds when other overpowering sounds occur at the same time.

Anyway, thought I'd share a 10 second snippet of one of the songs (Cowboy Junkies Trinity Session -- very high quality live recording in an old hall) back to back to back in 3 formats, all saved as one WAV file along with a screenshot of the spectrum.

Can you hear the difference?

Can you see the difference in the JPEG? (probably not zoomed in enough to make any solid conclusions...)

Anyway, I'm starting to wonder whether I should just use -V5 --vbr-new from now on, since most of my listening is hardly deep enough that I'd really hear these minor differences.

Anyway, here's the file: http://gordo.dyndns.org.100.nyud.net:8090/..._comparison.zip

As you said, -V5 is fine for several people (not the one used to talk about audio compression without knowing anything about ABXing) and for most situation. Just keep in mind that the concept of audio quality that applied for the perceptual encoding area is different from the concept of audio quality in use in other applications like hardware or HI-FI. In our case, -V5 could be fine for most audio material but could still reveal audible issues in some specific case. Robustness (against artefacts, against difficult or strange situation) is a part of the concept of quality. And LAME -V5 isn't as robust as -V2 for example.

In other words, if you're using LAME to build a library supposed to be fully transparent, -V5 is a bit risky and higher settings are most suitable for that purpose (with more bits, LAME could handle more easily specific problems).
I'm using -V5 --vbr-new myself for my portable player, but I'm also listening to them at home with my Beyerdynamic headphone. Surprisingly, these encodings are often transparent on optimal listening conditions and I quickly forget that I'm listening to something else than lossless files! But from time to time, some issues (minor and also major ones) are reminding me the nature of the source of my musical pleasure. On portable playback, the audibility of these issues are really exceptionnal.

Last but not least, the ability to detect artefacts depends a lot of the listener habits or training. I suppose that I can safely say that I'm a trained listener; if I weren't, I'd say that -V5 would sound absolutely perfect to my ears (like were my first CBR128 encodings four or five years ago).

I use -V5 --vbr-new.

According to the recent multiformat listening test @ 128kbps, you are not alone. -V5 --vbr-new has proved itself to be a decent quality for everyday listening, especially if you are talking about using on a portable. If you would mainly listen on your PC using headphones you may do better to go a little higher in bitrate/quality.

NB: I haven't bothered to look at your files.

Edit: Too late, beaten by the master...

Edit 2: If guruboolez is listening to -V5 at home then that is surely testiment in itself. LAME could use that as a strapline...

Interesting, I see what you're saying. That puts a slightly different spin on things. I think you're saying that -V2 has a greater margin of safety against artifacts, whereas you run close to the line with -v5 and may risk artifacts from time to time. I'd assumed that the VBR algorithm would take care of these, but I suppose that's only the case up to a point.

Anyway, Lame VBR V5 is miles better than many of the 128k MP3s I've got on my system, some from as long as 10 years ago... Listening to those is difficult compared to LAME. MP3 has come a long way!

VBR purpose is (in theory) to ensure a constant level of quality. In reality, the perceptual quality isn't as constant as expected. That's why a transparent profile could lead to non-transparent encoding (e.g. distortions or artefacts). Even the highest level are not fully free of problems - especially with MP3 which is handicaped by native limitations (such as pre-echo).
I compared -V5 to -V2 (--vbr-new for both) on 200 samples, and for most of them, a difference was perceptible (link). But for some files, both encodings were eaqually transparent. Robustness... and in rare cases, -V2 encodings revealed weird issues.

Perhaps this is a stupid question, but where is there mention of the original recording in this test? Do the subjects listen to the original track before every encoded track and give comparative subjective opinions?

It's rather surprising that Lame is keeping up with AAC , which is supposed to be a generation newer and better than MP3. How exactly is this possible all within the original spec for MP3, which has been around for more than a decade? Is the format really that accomodating of new techniques and acoustic algorithms?

Do the subjects listen to the original track before every encoded track and give comparative subjective opinions?
Original (=non-encoded file) is available as explicit reference. The tester is free to ignore it or to listen to it as often as needed.

How exactly is this possible all within the original spec for MP3, which has been around for more than a decade?
IIRC, AAC was launched by Dolby in 1997 - nine years ago. Both formats are old. But LAME encoder benefits from a longer maturation period and that should explain how this encoder (thus the MP3 format) could still compete with (or against) his technological successor.

The method used for this test is ABC-HR. Basically for each codec a listener gets presented two files (encoded and original) without knowing which is which. He can switch between the two files for listening and then (try to) rate the encoded one.

I think you should see AAC more as a 'bugfix' release for MP3 than as a real overal quality improvement, except for low bitrates like 64 kbps. (but this is also related to SBR and PS, which are technologies on top of AAC).

AAC by itself (theorically) is quite more efficient than mp3.
A big difference is that with mp3, you have to really "fight" to be among the best contenders at 128kbps, while with AAC it is quite easier to beat mp3. However, once you are ranked at top position, interest to enhance quality is decreasing in the case of a commercial company.

It is competition from new technologies that partially "stimulated" Lame developement. In the AAC world, competition is not that big.

Makes sense.

Wow, what a great forum this is. I'm highly impressed by the quality of the discourse, knowledge of the posters and double blind listening tests you rely on to to back it all up. I will have to spend more time around here!

--Gordo