Condtions:

All files have been converted with dBpoweramp (R 13.2)

In each category, files’ sizes are very similar (less than 4% between them).

Lame encoding has been chosen as “slow (highest quality)”.


Songs: Metal music as said in the title:

Devil To The Metal – Cradle Of Filth (~symphonic black metal)
Summon The Wild – Eternal Tears Of Sorrow (melodic death metal)

Above songs are diversified enough owing to the “symphonic” passages, keyboard is tested, female voice, growls, and so on, not to mention drums blast beats which help me to recognize MP3 and WMA (both don’t sound that good during blast beats)


Note 1: “3/10” can't be seen as a mark, it simply means that I got 7/10 in the ABX test and that I recognized the lossy (or the lossless one) song 7 times out of 10 and that the lossy song tricked me 3 times.

“0/5” means that the lossy song never tricked me and that I made distinction with the original it each time, therefore, sound quality of the considered lossy song might be poor.

Do what you want with these results, it may help, but aren't professional tests at all (not many trials, not many songs, and so on), even though it took me some time (much^^).
I saved that for me, so I don’t mind sharing and explaining a little.


Note 2:

Firstly,“Low” and “medium” don’t say anything, these words are totally arbitrary chosen! (since my “medium”category is the low category of someone else !)

Secondly, I didn’t bother to try bitrates above 170 kbps since it would have taken far too much to time to tell which one is which one, as far as I’m concerned…






Here we go

Low < 128kbps

(only COF song was tested here)

Lame 3.98 MP3 V7 VBR (average 120kbps) – Flac : 0/7. Lame loose.

WMA Pro 10 VBR 50 (average 126kbps) - Flac: 0/7 WMA Pro loose

WMA Pro 10 CBR 128 (1Pass or 2 Pass, both tested) – Flac: 0/7 WMA Pro loose (foreseeable there)

Nero AAC LC VBR .35 (average 118kbps): 0/7
First test (at first, quite hard to tell): 3/10 (up to 10 since I was unsured)
Second test (now that I knew better touchy passages): 0/7 AAC LC eventually loose

Nero AAC HE VBR .45 (yes 45 not 35 in ‘HE mod’ gives a bitrate of 118kbps and similar size to other samples) : 0/5
(I only needed one test and easily recognize AAC HE maybe because I had been use too AAC weak points) (by the way SBR seems to be useless at such a bitrate)


Short conclusion: There are no codecs (among tested) that sound really good at rates under 128kbps with such metal music, on the whole (even though, AAC LC turned out to be firstly tricky), you can easily tell which is the original and which is a poor quality audio sample.




Medium 128 kbps <bitrate< 160kbps

COF song:

Lame 3.98 MP3 V5 VBR (average 142kbps) – Flac : 4/15

WMA Pro 10 VBR 75 (average 149kbps) : 7/15

Nero AAC HE VBR .55 (average 139kbps): 10/15

(AAC LC can’t be tested there since even .45 gives 170kbps and huge size, unlike .55 HE)

ETOS song:

Lame 3.98 MP3 V5 VBR (average 147kbps) – Flac : 2/10 , then, 0/7

WMA Pro 10 VBR 75 (average 149kbps): 3/10, then, 1/7

Nero AAC HE VBR .55 (average 139kbps): 9/20 (I pursued up to 20 since I thought I could tell more easily differences, but I as you can see, I actually failed)



Conclusion: Second song highlights once again that transparency of a codec really depends on the genre, and even on the song. Not every codec is the same according to what you are used to listening to, that’s something not to forget if you have various genres in your library to encode. (all the more so as both songs were “metal music” rather violent )

With regard to AAC HE, I guess SBR parts are simply useless there (too high rates), and that quality is the same as ACC LC at the same bitrate, even though I haven’t been able to test it )

Thus, I’m still surprised how AAC LC seems to be fine, and turned out to be the hardest to recognize.


Furthermore, while ABX test around 120 kbps (“low”) were quick and lossy songs easily recognizable FOR ME, only a few kbps more and it’s a way more hard to tell.

I hadn’t expected such a difference between “low” and “medium”, I found somehow which one was lossy but only because I focus on details as much as I could.

I a nutshell, if I had heard the Flac song one day, and the lossy one the day after (without knowing it was lossy), I guess I wouldn’t have noticed anything. (while during ABX you really care)

Thanks for the test. I think it's generally accepted that it varies by genre and song. Some require a much higher bitrate than others. Though some of us really can't be bothered changing settings per song and even album. I listen to mainly metal but pretty much everything sounds fine with LAME V 3 through my iPhone and Westone UM-2s....I've not ABXed the odd song I think sounds bad yet but am entirely happy overall

I also listen to metal mainly, my favourite band being Amon Amarth. I have been using lame -V 4 --vbr-new for quite some time, until I found some artifacts with theese settings with Amon Amarth - Victorious March song (and then later also some others). I have done no ABX tests, but I did not find it necessary, since the artifacts are just too obvious to me. But I can provide flac sample(s) if someone would like to test it.
Where I'm getting at with my post is, that since that I am using -V 0, which is probably total overkill, but just to be sure and because HDD space is not so much of a problem these days anymore. (and as time will pass, I will maybe abandon mp3 and switch to some better codec like Vorbis, but first all of my hardware players will have to support it - maybe you should repeat your test and include Vorbis?)
Anyway, I also found out that metal music almost never gives the bitrate expectations for -V switches. It is almost always gives higher bitrates than expected. But that is logical (at least to me), as metal songs usually are very complex - percussions, drums, distorted guitars, growls or high-pitched voice, etc.

That will be sfb21 bloat, which appears alot on most metal music and other types of music that has alot high freqs. This issue appears on V2 - V0, you can add -Y (default on V9 - V3) which sorta acts like a smart lowpass filter and lowers the bitrate on V2 - V0.

/mnt, thanks for clearing that up. I'll try -Y on my next encoding.

Some of you may try AAC HE (or AAC LC), I was really impressed while I was used to using WMA Pro VBR 75/90

Thanks for commenting !

@ pyscho: I like AA much too , like Ensiferum, In Flames, Dark Tranquility, Eluveitie and so on..

I do have Victorious March song, I may try to listen to it with different codecs, but in a few weeks since I'm going to England on tomorrow



note: I also would like to test high-pitched voices with Epica (and the beautiful Simone Simons.) to see how encoders perform with it.

Eternal Tears of Sorrow are great... and thanks for the test )))