MP3 vs. MPC vs. Ogg: Low volume test Options

[tigre]

Hi there!

Inspired by Using DirectSound SSRC Plugin thread I wanted to know what's the difference between outputs of foobar 2000 and winamp/other players. I figured out following test:

1. Take a sample (I did it with 30 seconds of music, could be done with test signals too), convert it to 32 bit resolution in Cool Edit Pro.

2. Apply logarithmic fadeout to the whole sample: 0 to -150 db (using CEP).

3. Save or dither to 24bit -> save, depending on the needs of lossy encoders (Step 4).

4. Encode with lame 3.90.2 -api, mpc 1.15i braindead, vorbis 1.0 -q 10 (this should not be the weak link).

5. Playback encoded files in tested players and capture output to wav file, resolution 24bit/96kHz if possible. The problem here is that I don't know any tool for this, I asked for advice in this thread. Because of this I've only tested this on foobar2000 0.5 beta so far, as its diskwriting feature is capable of 24bit/96kHz output. So until now it's not a player output comparison but a encoder comparison as you'll see soon.

6. Aply logarithmic fadein 0 -> +150db to the whole sample with CEP to bring the volume back to original level everywhere.

7. Listening to the resulting files.

The reason for this proceeding was to "exaggerate" the differences that occur at very low volume and make them audible.

My observations: All files started to contain aditional noise compared to the original after ~1/2 of playing time. At this point started what surprised me: The mp3 sounded like the original + increasing noise till the end of the sample while MPC and Vorbis started with slight artifacts and sounded more and more awfull: ogg somehow like underwater, mpc like NUMLock's lame audiophile preset and worse.

So is mp3 superior at low volumes? Not really. I gave mp3 an advantage by using CBR. A quick test using lame ape (=VBR) showed the same awfulness as Vorbis and mpc. Besides, I noticed that with decreasing volume the bitrate of all 3 codes decreased too.

Conclusion? I'm not sure, but maybe there's something that could be improved (adaptive ath model for high resolution signals, or ...). Imagine, you have a high resolution source (DVD audio ...) and music that uses its huge dynamic range (not completely of course) - and equipment with really high SNR ... this could be an issue.

I hope I haven't just wased my time and space on this forum . Thoughts welcome.

[tigre]

@Gabriel: Here's the requested MP3 test:
Original test sample: "real" music (salsa): o.wav
length 42.667 sec.
peak amplitude -4.54dB = 19248 max sample value
max RMS Power -10.48dB
min RMS Power -48.4dB
Average RMS Power -21.85dB
Total RMS Power -21.0dB
(Values taken from CEP Waveform Statistics)

1. Creating amplified "original" sample o_a_4b.wav:
- converting to 32bit
- logarithmic fadeout 0 -> -150dB
- save as 4 byte PCM (type 1, 32bit), as Lame can't handle default CEP format "32 bit Normalized float (type 3)" properly.2.

2. For comparing noise created empty file, same samplerate, same bit-depth, same length: n_a_4b.wav:

3. Encoding
o_a_4b.wav:
- lame 3.90.2 --alt-preset extreme
- lame 3.90.2 --alt-preset insane
- lame 3.94a11 --preset extreme
- lame 3.94a11 --preset extreme -F
n_a_4b.wav:
- lame 3.90.2 --alt-preset insane

4. Decoding with foobar2000 0.5beta16 Diskwriter, Output WAV (PCM 24bit dithered), DSP used, Resampling to 96kHz (Fast mode disabled)
- all encoded files
- o_a_4b.wav
- n_a_4b.wav

5. Processing all files made in step 4 like this:
- apply logarithmic fadein 0 -> +150dB
- downsample to 16bit/48kHz (dithered) and save to xxx_fadein.wav

6. Having a look at the files from step 4 with Encspot

7. Listening using WinABX

Results/Conclusions

Step 6: AFAIK --alt-preset extreme (3.94a11 --preset extreme) only uses 32kbps frames when it "notices" silence. So for the mp3 files encoded with "extreme" it could be an interesting information when the first 32kbps frame and when the last 128kbps frame occurs.

lame 3.90.2 --alt-preset extreme:
- first 32kbps frame: 22.7 sec. -> Amplification at this position: -80dB; RMS Power at this position of o.wav: 25dB -> power of encoded signal: -105dB
- last 128kpbs frame: 26.5 sec. -> Amplification: -90dB; RMS Power of o.wav: -17dB -> power of encoded signal: -107dB

lame 3.94a11 --preset extreme:
- first 32kbps frame: 23.1 sec. -> Amplification: -81dB; RMS Power of o.wav: -30dB -> power of encoded signal: -111dB
- last 128kpbs frame: 27.2 sec. -> Amplification: -95dB; RMS Power of o.wav: -19dB -> power of encoded signal: -114dB

Maybe the maths I'm doing here are a sign of faulty resoning, but as I did the same it's comparable. So it seems that lame 3.94 keeps encoding low volume signals where lame 3.90.2 already stopped and the difference is somewhere around 6-7dB.

Step 7: ABX tests (The volume of my system (AC'97 onboard sound, amp, HD 530 headphones) is set to a level I use normally for ABXing music samples = "hearing-damage proof", all ABX tests 4/4 or till guessing probability < 5%):

- n_a_4b.wav vs. n_a_4b_fadein.wav to find out where I start noticing noise ("equipment/ears test"). -> 21 sec. = -83dB RMS power

- o.wav vs. o_a_4b_fadein.wav (Where starts noise added to music to be noticable?) -> 24.0-25.0 range; min RMS power of that range: -33dB; RMS power of n_a_4b_fadein.wav (=noise) at this point: -71dB => "SNR"=38dB

- o_a_4b_fadein.wav vs. o_a_4b_3.90.2_insane_fadein.wav: The *music* sounds exactly the same to me. Increasing noise starting at 24 sec., starting to sound distorted (clipping) at 39 sec. If I focus on the nois I can ABX at some points (34-36 sec. = noise RMS -35 to -25dB), but it's questionable if the difference is caused by lame or by massive amplification of dither noise.

- o_a_4b_fadein.wav vs. o_a_4b_3.90.2_extreme_fadein.wav: ABXable differences start at 13:0-13:9 sec. (ringing, hashness of "s", later "underwater sound") -> Amplification -47dB, RMS power -21dB => RMS power of signal passed to lame -68dB

- o_a_4b_fadein.wav vs. o_a_4b_3.94a11_extreme_fadein.wav: 13:0-13:9 sec. (Same as 3.90.2.; 3.90.2 vs. 3.94 isn't ABXable for me with this sample - well, I didn't bother finding differences at 20 ... 30 seconds where music is totally replaced by artifacts

- 3.94 --preset extreme -F: The same. It seems like the "guilty" is vbr mode, not minimum bitrate of 32kbps, since 3.90.2 api sounds fine the whole range.