Here’s the result of my first ABX test.
I listen to mp3 files for about ten years now, and I use Vorbis since 2002 (the first final release). I hate badly compressed songs, but however I never really tried to know if I was able to distinguish an encoded sound. I love to listen high quality music, but what quality do I need ? Is the quality of a high VBR vorbis or mp3 really this good ? I began this test to answer these simples questions… and for fun, of course !
I wasn’t planning to share the results on a forum, so don’t blame me if I made mistakes, or if my comments are not always perfect… and sorry for my average english, I’m french.
I used the following samples from my own collection of CDs:
01 - Alain Souchon - Au Ras Des Pâquerettes - Au Ras Des Pâquerettes
02 - Björk - Homogenic - Hunter
03 - Björk - Homogenic - Pluto
04 - Ben Harper - Diamonds On The Inside - When It's Good
05 - Mogwaï - Mr Beast - Acid Food
06 - American Head Charge - The War Of Art - A Violent Reaction
07 - Guem - Percussions Africaines Pour La Transe - Le Serpent
08 - Dredg - Leitmotif - Lechium
09 - FSOL - The Isness - High Tide On The Sea Of Flesh
10 - Soulfly - Primitive - Back To The Primitive
11 - U2 - War - Seconds
12 - FSOL - Dead Cities - Antique Toy
13 - The Chemical Brothers - Dig Your Own Hole - Elektrobank
14 - Genesis - Foxtrot – Horizons
I tried to choose interesting samples, despite the fact that it was my first attempt to find ones by myself.
I used EAC 0.95 beta 4 with secure mode to rip every songs. Each CDs are originals ones of course.
I wanted to generate encoded samples that I would be supposed to listen in real life. So I used well know compiles of encoders:
- oggdropXPd V1.8.9 Lancer[20061110] with SSE3 and MultiThreading, using aoTuV b5
- lamedropXPd V2.0 using Lame 3.97 final
The settings are quite simples as well. For lamedrop:
- Target Quality : 50 / VBR mode : Standard / Encoding Engine Quality : High (132kbps)
- Target Quality : 80 / VBR mode : Standard / Encoding Engine Quality : High (190kbps)
- Target Quality : 100 / VBR mode : Standard / Encoding Engine Quality : High (240kbps)
And for oggdrop:
- Standard Quality Mode : q1 (80kbps)
- Standard Quality Mode : q3 (112kbps)
- Standard Quality Mode : q5 (160kbps)
Assuming mp3 @ 132kbps is 100%, then 240 is 182%.
Assuming Vorbis @ 80kbps is 100%, then 160 is 200%, so we will see how these encoders behave when the data rate is doubled.
Here is the hardware used for ripping, encoding and ABX:
Intel E6600
Asus P5B Deluxe no Wifi
Hitachi Deskstar T7K250 160 Go P-ATA
Pioneer DVR-110D modded with DVR-A10XL firmware (v 1.41). Used to rip every samples.
Creative Audigy LS
Sony MDR-EX51 (not killer earphones, but quite good, and the intra auricular design is good to ear sharp sounds).
I used the tool ABC/HR 1.1b2 to randomize my samples.
So here are the results:
Each sample is described with a special notation, from ‘very easy’ to ‘perfect’. Very easy means it’s easy to distinguish encoded from original, perfect means it’s impossible
Let’s begin with mp3 samples. I was able to distinguish most of the sample when they were encoded with 132kbps, but the quality was really better with 190kbps. I was extremely surprised when I realised that the advantage of 240kbps was so small over 190.
Mp3 @ 132kbps:
Sample 01: Easy to find: like if a paper sheet was placed between your ear and the speaker.
Sample 02: Very hard: a very soft pre echo is audible.
Sample 03: Easy: take care of the sputtering sound.
Sample 04: Impossible: sounds perfect!
Sample 05: Very easy: again the paper sheet effect.
Sample 06: Very easy: sounds smooth, original sound is way sharper.
Sample 07: Easy: take care of the sputtering sound on the left side.
Sample 08: Easy: paper sheet syndrome again.
Sample 09: Very hard: paper sheet syndrome, but extremely soft.
Sample 10: Average: take care of the sound turning from a side to the other. Paper sheet syndrome.
Sample 11: Very easy: charleston sound is softened. Paper sheet syndrome.
Sample 12: Easy: pre echo on the very sharps ‘clics’ sounds.
Sample 13: Average: take care of the sputtering sounds.
Sample 14: Impossible: sounds perfect!
My comments for 190kbps and 240kbps samples that I can ABX focus on the same weaknesses described above. Some artefacts are extremely hard to ear, and quality is truly awesome starting from 190kbps. But as I said before, I was surprised I was able to recognize 240kbps samples. Quality doesn’t increase that much with 240 against 190.
Vorbis encoded samples were easier to find a low bit rate, but aoTuV b5 seems to take more benefits of higher bit rates than Lame does. Using 112kbps, some samples that sound really bad at 80 become excellent. I found only two samples with 160kbps.
Vorbis @ 80kbps:
Sample 01: Very easy: paper sheet syndrome.
Sample 02: Average: take care of percussions at the end of the sample.
Sample 03: Very easy: the sputtering sound is smoother than the original.
Sample 04: Very hard: sound of hands clapping is a bit messy, smoother. Extremely hard to find.
Sample 05: Very easy: the sputtering big beat is distorted and smooth.
Sample 06: Very easy: nothing is good here!
Sample 07: Easy: when big percussion arrives, the sputtering sound on the left is less audible.
Sample 08: Easy: cymbals are less audible, but not distorted.
Sample 09: Very hard: take care of the sharpness of guitar cords.
Sample 10: Average: paper sheet syndrome.
Sample 11: Very easy: cymbals and charleston. Paper sheet syndrome.
Sample 12: Average: pre echo on the very sharps ‘clics’ sounds.
Sample 13: Very hard: the sputtering sound is a bit smoother.
Sample 14: Very easy: sound is much distorted.
As for mp3, my comments for higher bit rates samples are based on the same weaknesses shown with 80kbps, except sample 14 which becomes transparent with 112kbps. The increase of bit rate allows an incredible benefit on quality.
So, that’s it, I hope you enjoyed the test, and that you will find some interesting sample in my compressed file
Thanks to HA for all the informations gathered here
PS : this is the link to my PCM samples
