Personal listening test mp3 / Ogg Vorbis, No golden ears for me... Options

[hankwang]

Introduction

Since I always have trouble finding my CDs, I am considering to rip my whole CD collection to Ogg Vorbis or MP3. But to which format and bitrate? Everybody seems to agree on the fact that Ogg Vorbis sounds better for a given bitrate, but there aren't yet that many hardware players.

Since I would replace my CDs, transparency is the main requirement. People on the net declare that for full transparency, one would need at least ogg -q6 or mp3 at 300 kbps bitrate. I mostly listen to classical music, which is a bit sparsely represented in the various online tests.

I compared the compressed sound files to the original and tried to describe in what respect their sound differs (for some reason, this is seldom mentioned in online listening test reports), at least as far as I can reliably tell the difference in a blind ABX test. I found it easier to find sensitive spots in short fragments, instead of to switch back and forth during a whole track.

Apparently I have tin ears, because the encoders become transparent at bitrates far below the -q6 or 300 kbps that is mentioned by golden-ear listeners elsewhere, or my music taste is not demanding. (I do use a good headphone and a good sound card and my ears can still hear up to 20 kHz.) If someone knows how I can upgrade my ears, please tell me. :-)

As far as the encoders are not transparent, Ogg Vorbis and MP3 turn out to sound very different. Note that I hardly ever listen to MP3s and the like, so this may be a well-known fact for you. The MP3s tend to give annoying ringing artifacts, often strongly localized incertain music fragments. Ogg Vorbis, on the other hand, tends to distort the stereo image and create an overall hissing background and some coloration. Ogg Vorbis's distortions are much less annoying, because they are comparable to coloration by loudspeakers.

The stereo-image distortion is strongest in the choral and harpsichord solo recordings. Likely, different microphones were recording the same sound source in those cases, which results in strong phase differences between the left and right channels, which are apparently hard to encode in mid/side mode. I suspect that the other recordings mostly have an intensity stereo image as opposed to a phase image. Strangely enough, I didn't observe stereo imaging problems in the MP3s.

Enough said, here are the results.

Encoders

Lame 3.93 --resample 44.1 --abr (xxx+offset)

I tweaked the --abr option such that the final average bitrate was with +/- 1.5 kbps the bitrate that I wanted. Lame wants to downsample at lower rates (below --abr 103), which my ABX test couldn't handle, so I forced a 44.1-kHz sample frequency. Maybe unfair to Lame at low bitrates, but that's life...

oggenc 1.0-7 -q xxx

I used oggenc simply with quality numbers.

Samples and ratings

The encoded samples (30 seconds) [edit] were online, but not anymore. To save space, I didn't put original WAVs, but rather ogg-q7 versions as the references.

antonilla

Juan del Encina, Antonilla dees desposada
Margaret Philpot (alt), Christopher Wilson(?) (lute)
Hyperion CDA 66454
Comment: the lute in the original sounds as an artifact but isn't...
mp3 80k: horrible buzzing (22.05 kHz resampled)
mp3 96k: transparent
ogg q0 (52.7k): some ringing
ogg q1 (68.5k): transparent


bachkyrie

J.S. Bach, Messe in B minor: Kyrie Eleison
Ton Koopman, The Amsterdam Baroque Orchestra and Choir
Erato 4509-98478-2
mp3 96k: strong ringing
mp3 112k: audible
mp3 128k: audible artifact at 21-22 seconds (2nd "Kyrie")
mp3 150k: audible
mp3 160k: transparent
ogg q0 (47.8k): strong distortion in stereo image
ogg q1 (62.8k): stereo image; colored sound
ogg q2 (79.2k): stereo image, colored sound, hissing (mainly in beginning)
ogg q3 (101.8k): audible hissing
ogg q4 (116.4k): transparent

bachwtk

J.S. Bach, Das wohltemperierte klavier: prelude nr. 14
Leon Berben, Harpsichord
Brilliant Classics 99362
Comment: A budget edition; I'm not really a fan of harpsichord
music, but since harpsichord is reputedly hard to compress, I
thought I'd give it a try.
mp3 96k: dull (missing high frequencies)
mp3 112k: dull
mp3 128k: somewhat dull
mp3 150k: transparent
ogg q0 (65.4k): stereo image, dull sound
ogg q1 (79.2k): stereo image, a bit colored
ogg q2 (96.5k): fuzzy, hissing middle register
ogg q3 (131.6k): nearly transparent
ogg q4 (162.0k): transparent


mignon

Hugo Wolf, Goethe Lieder: Mignon
Geraldine McGreevy (soprano), Graham Johnson (piano)
Hyperion CDA67130
mp3 96k: small artifacts on consonants ("ein *S*anfter wind")
mp3 112k: very small artifacts
mp3 128k: transparent
ogg q0 (38.1k): ringing and hissing in piano intro
ogg q1 (51.9k): ringing in piano
ogg q2 (74.7k): some hissing in piano
ogg q3 (96.2k): nearly inaudible coloring
ogg q4 (109.1k): transparent

salsa

Roberto Roena, Mi Mambo
Nascente NSCD 039 "Salsa Moderna"
Lots of percussion and trumpet

mp3 96k: dull
mp3 112k: somewhat dull
mp3 128k: somewhat dull percussion
mp3 150k: transparent
ogg q-1 (53.2k): dull/distorted percussion (cymbals)
ogg q0 (68.0k): transparent
ogg q1 (82.0k):
ogg q2 (95.5k):
ogg q3 (118.8k):
ogg q4 (137.2k):

This post has been edited by hankwang: Feb 4 2005, 00:18

[odious malefacto...]

For mp3, use HA's recommended LAME build of 3.90.3 with the switch "--alt-preset standard" and try again.

(But... good luck with harpsichord. Maybe use "--alt preset insane" for that difficult to encode instrument.)

Have you considered going "lossless"?

This post has been edited by odious malefactor: Jan 18 2004, 21:03

[guruboolez]

(But... good luck with harpsichord. Maybe use "--alt preset insane" for that difficult to encode instrument.) 

hankwang tried with --abr 150 and find it transparent. I don't think that VBR or CBR 320 are useful... for the moment.

hankwang > golden ear are not necessary for hearing artifacts at bitrate > 128 kbps. Common ears need some training before artifact perception. It's like jpeg or DVD artefacts: some people don't know what a macrobloc is, so they're not looking for it, and therefore, don't see it (except for strong cases).
You find mp3 transparent at ~160 kbps and vorbis at -q4: it's normal. Most people are happy with mp3 at 128 kbps and vorbis at 64...
But you don't know what wil happen in some months. Vorbis hiss/coarse sound or mp3 distortions may be more audible than today. So encode with care...

[Garf]

But you don't know what wil happen in some months. Vorbis hiss/coarse sound or mp3 distortions may be more audible than today. So encode with care...

That risk can be greatly reduced by not reading HA. QUICK! LEAVE!

[hankwang]

For mp3, use HA's recommended LAME build of 3.90.3 with the switch "--alt-preset standard" and try again.

(But... good luck with harpsichord. Maybe use "--alt preset insane" for that difficult to encode instrument.) 

Well, I was more interested in the big picture than in the subtle differences between different builds. If I one day start to rip all my cds, I will do that. My main issue is that things become transparent for my ears at very low bitrates, much lower than --preset standard, even for the harpsichord, would give. How do the people here at HA manage to hear differences at 200+ bitrates when I don't hear the difference with the original at 150 kbps in the worst case??

[guruboolez]

I did two tests a ~130 kbps (including vorbis -q 4) with various classical samples:

http://www.hydrogenaudio.org/forums/index....c=14091&hl=test
http://www.hydrogenaudio.org/forums/index....c=16395&hl=test

I strongly suggest you to compare vorbis with QuickTime AAC, newest Nero AAC, or the amazing (with classical music) WMA9PRO.


Note that results are mine, and based on my own subjectivity. In these conditions, vorbis performs poorly compared to other lossy solutions.

[hankwang]

I did two tests a ~130 kbps (including vorbis -q 4) with various classical samples [...] I strongly suggest you to compare vorbis with QuickTime AAC, newest Nero AAC, or the amazing (with classical music) WMA9PRO.

Thanks for the tip. I browsed a few old discussions on this forum, but I didn't go back that far. Other formats than Vorbis and MP3 are not an option for me since I run Linux. Moreover, I'm thinking of getting myself a hardware player, both portable and disc-based on the stereo. After all, I don't want to switch on my noisy computer just for listening to music. AAC is not supported widely on dedicated hardware players. Neither is Vorbis, unfortunately, but I expect that to change soon.

[guruboolez]

Isn't AAC playable on Linux systems? Faad2 is open-source AKAIK, and xmms or other player might have their plug-in. I don't know, but the contrary would surprise me.


EDIT: I suppose that you need encoder working on linux too... so forget my answer (or take a look at faac AAC encoder, not as good as iTunes/QuickTime, but not bad, and sometimes really good.

This post has been edited by guruboolez: Jan 18 2004, 22:04

[rjamorim]

Isn't AAC playable on Linux systems? Faad2 is open-source AKAIK, and xmms or other player might have their plug-in. I don't know, but the contrary would surprise me.

yes, there is FAAD2 for linux, both standalone decoder and AAC/MP4 XMMS plugins. Check the "Debian" part of RareWares, maintained by xmixahlx.

EDIT: I suppose that you need encoder working on linux too... so forget my answer (or take a look at faac AAC encoder, not as good as iTunes/QuickTime, but not bad, and sometimes really good.

yes, FAAC is also compilable in linux.

You can run Psytel AACenc in linux too, using Wine.

[rjamorim]

AAC is not supported widely on dedicated hardware players.

It is, if you consider the iPod has 30%+ of the compressed audio player market share.

[maikmerten]

yes, FAAC is also compilable in linux.

You can run Psytel AACenc in linux too, using Wine.

I´m not sure if it is a good idea to switch from MP3/Lame to AAC/FAAC if quality matters... or has FAAC improved a lot?

And running AACenc in Wine - argh...

(I think it is a horrible idea to put the "ugly" Win32-API on top of a woooonderful Unix-like API )

Maik

[rjamorim]

I´m not sure if it is a good idea to switch from MP3/Lame to AAC/FAAC if quality matters... or has FAAC improved a lot?

According to the developer (knik), yes.

It'll be shown at the AAC test I'm going to conduce in February.

And running AACenc in Wine - argh... 

Well, worked perfectly here, and for other people as well (Robert Hegemann, xmixahlx). Speed didn't seem to suffer from the emulation either.

[bidz]

Speed didn't seem to suffer from the emulation either.

Wine Is Not an Emulator

[hankwang]

AAC is not supported widely on dedicated hardware players.

It is, if you consider the iPod has 30%+ of the compressed audio player market share.

Well, here in Sweden, I hardly ever see iPods in the electronics stores. It's all MP3 players. We're probably a bit behind.

[rjamorim]

Well, here in Sweden, I hardly ever see iPods in the electronics stores. It's all MP3 players. We're probably a bit behind.

Well, you surely can find an Apple Store somewhere. We have them even here in the tropics! :B

[Althalus]

This apple store has open dygnet runt
http://store.apple.com/Apple/WebObjects/sw...ore?family=iPod

[2Bdecided]

But you don't know what wil happen in some months. Vorbis hiss/coarse sound or mp3 distortions may be more audible than today. So encode with care...

That risk can be greatly reduced by not reading HA. QUICK! LEAVE!

LOL!!!!!

I think HydrogenAudio should come with a warning!