I couldn't resist, and after some hesitations, I started the 96 kbps with the first pool dedicated to MP3. I did it on a different computer with poorer components (AC'97). Notation will therefore be less severe (poorer hardware -> less audible problems).
The choice of settings for each encoder was difficult. I tried to obtain an average bitrate comprise between 96 kbps (CBR) and 100 kbps. The tolerence is much restrictive than the ±1O% fixed for my 80 kbps test. iTunes and Fh.IIS 'Audition' are nevertheless out of range for the second group of samples (104 kbps for iTunes / 102 for Fh.IIS), but the deviation is still inferior than 10% of the targeted bitrate (96 kbps).
tested encoders•
Fraunhofer, in
Adobe Audition v1.5group1 (classical) - VBR Q20 - Current: best quality - default: Joint Stereo + Intensity Stereo + 14440 Hz lowpass
group2 (various) - VBR Q30 - Current: best quality - default: Joint Stereo + Intensity Stereo + 14780 Hz lowpass
•
Fraunhofer, in
Windows Media Player 10 (ACM PRO version, 3.3.2.44) © 2004 Fraunhofer IIS
CBR: -b128
•
Apple iTunes 4.9.0.19VBR: 96 kbps Highest (default settings)
•
LAME 3.97 alpha 11ABR: --abr 101
•
Real mp3enc V5.0 rev.6VBR: -V20
calculated bitrate (short samples library)CODE
• Fh.IIS 'Audition'
- classical (185 samples) = 96 kbps
- various (35 samples) = 102 kbps
• Fh.IIS 'ACM encoder'
- classical (185 samples) = 96 kbps
- various (35 samples) = 96 kbps
• iTunes
- classical (185 samples) = 100 kbps
- various (35 samples) = 104 kbps
• LAME
- classical (185 samples) = 98 kbps
- various (35 samples) = 100 kbps
• REAL
- classical (185 samples) = 100 kbps
- various (35 samples) = 100 kbps
hardware and software settings• Compaq Presario 2100 series; AC'97 'soundcard'; poor line_out
• Philips SBC HP910 headphones
• ABC/HR software (ff123)
• files decoded with foobar2000: resampling at 48 KHz & ReplayGain track mode enabled; offset corrected for LAME and Fh.IIS ACM encodings.
tested samples• 40 samples, including
- 15 samples of 'various music'
- 25 samples of 'classical music'
(the selection is exactly the same than for my 80 kbps listening test pools.
RESULTSCODE
Fh.IIS Fh.IIS iTunes LAME Real
Audit. ACM PRO v.49017 3.97a11 5.0 rev.6
A02_metamorphose 3.5 2.0 1.0 3.2 2.7
E06_MODERN_CHAMB 4.2 2.0 2.3 4.2 2.5
E15_MODERN_CHAMB 2.5 2.0 4.0 5.0 2.3
E22_MODERN_ORCHE 3.4 2.0 3.8 4.2 2.8
E26_MODERN_ORCHE 2.0 1.5 3.5 4.0 3.0
E31_PERIOD_CHAMB 4.5 1.3 3.0 4.0 2.0
E40_PERIOD_CHAMB 1.8 3.0 1.5 4.0 3.5
E51_PERIOD_ORCHE 3.0 2.5 1.0 4.2 1.5
E53_PERIOD_ORCHE 2.7 2.0 1.7 3.2 2.9
S03_BOW_Cello_C 2.3 2.5 2.0 4.0 3.0
S08_BOW_Violin_B 3.5 2.0 3.0 4.2 3.5
S12_KEYBOARD_Har 2.8 2.0 2.2 2.5 1.5
S17_KEYBOARD_Org 2.0 1.5 2.5 3.7 2.5
S27_KEYBOARD_Pia 2.5 2.0 3.0 4.5 2.0
S38_PINCH_Guitar 2.0 2.5 4.0 4.3 3.5
S50_WIND_Flute_B 2.5 2.0 3.5 3.0 4.5
S54_WIND_Trombon 3.3 1.8 1.5 3.6 2.5
V02_CHORUS_Child 1.2 1.7 2.0 3.5 2.5
V07_CHORUS_Mixed 2.8 2.0 1.7 4.2 1.5
V10_DUET_Males_A 2.0 2.4 2.8 3.5 3.2
V15_PLAINCHANT_M 2.0 2.5 3.0 3.4 2.3
V19_SOLOIST_Fema 2.0 1.8 3.1 4.5 2.6
V20_SOLOIST_Fema 3.2 2.0 1.5 3.5 2.7
V24_SOLOIST_Male 3.0 2.5 2.0 3.0 2.8
V27_SOLOIST_Male 2.8 2.5 2.5 3.5 2.3
25 CLASSICAL: MEAN 2.70 2.08 2.48 3.80 2.64
41_30sec 2.3 2.0 1.0 3.0 3.0
ATrain 2.5 2.0 1.3 3.5 2.5
DaFunk 3.2 2.0 1.8 3.5 1.5
death2 2.8 2.0 1.0 2.5 2.7
EnolaGay 2.8 2.2 2.6 2.8 2.4
experiencia 3.2 2.2 3.4 3.6 2.7
getiton 2.8 1.5 2.7 3.5 2.0
kraftwerk 1.5 1.5 2.5 1.5 3.5
LifeShatters 3.5 2.5 1.5 4.0 2.0
NewYorkCity 3.5 2.2 1.5 3.5 3.0
OrdinaryWorld 3.7 2.5 2.5 4.0 1.5
Quizas 2.4 2.0 1.3 3.8 2.6
rosemary 2.5 2.0 1.5 4.0 3.5
SinceAlways 2.5 2.2 3.0 2.7 2.0
trust 3.5 2.0 1.0 4.2 1.5
15 VARIOUS SAMPLE: MEAN 2.85 2.05 1.91 3.34 2.43
40 SAMPLES: MEAN 2.76 2.07 2.27 3.63 2.56
Fh.IIS Fh.IIS iTunes LAME Real
Audit. ACM PRO v.49017 3.97a11 5.0 rev.6
CODE
FRIEDMAN version 1.24 (Jan 17, 2002)
http://ff123.net/Tukey HSD analysis
Number of listeners: 40
Critical significance: 0.05
Tukey's HSD: 0.423
Means:
LAME Fh.Aud Real iTunes Fh.acm
3.62 2.76 2.56 2.27 2.07
-------------------------- Difference Matrix --------------------------
Fh.Aud Real iTunes Fh.acm
LAME 0.870* 1.062* 1.357* 1.555*
Fh.Aud 0.193 0.488* 0.685*
Real 0.295 0.492*
iTunes 0.197
-----------------------------------------------------------------------
LAME is better than Fh.Aud, Real, iTunes, Fh.acm
Fh.Aud is better than iTunes, Fh.acm
Real is better than Fh.acm
<<< PLOTS >>>CONCLUSIONS• Fh.IIS 'ACM': this encoding suffers from a severe lowpass (~12KHz), the worse from all encodings tested here. For that reason, I hesitated to feature this encoder. It's very hard for me to compare different encodings when such difference in lowpass exists. This encoder is therefore the easiest to detect; comparison with reference and even with other encodings is immediately shoking. Lowpass could bring one advantage: it often limits the amount of audible distortion. But here, the ACM encoder is really far from being free of artifacts and distortions. The encoder was convincing once or twice (E40, beginning of Death2), but disappointing most of time. Once the test finished, I noticed that this encoder obtained with regularity 2.0 as notation. In one word, this encoder was maybe handicap by the excessive lowpass, and may produce better results with ~14...~15 KHz lowpass (similar value than other encoders tested here).
• Fh.IIS 'Audition': As tester, I'd say that this encoder had the most annoying VBR mode. It was impossible for me to find a unique setting in order to obtain 96...100 kbps for both groups. CBR was not a solution: lowpass would be a big handicap (~11KHz vs ~14,5KHz). That's why I decided to use two different settings: Q20 for 'various' and Q30 for 'classical'.
Average results are similar for both groups, but it's important to note the variations within the classical group. VBR encoding is often difficult at low bitrate, and rarely provide constant quality (purpose of variable bitrate). Illustration here... With the second group, results are more constant and this encoder clearly appears as one of the best MP3 encoder (with LAME).
• iTunes: highest bitrate and poorer quality... at least with the second group of sample. Quality is unlistenable to my ears, with very annoying distortions. The poor quality is maybe a consequence of the generous lowpass, probably excessive for this bitrate. Result are better with classical (encoding difficulty is also lower...).
• LAME 3.97a11: The best for both groups, and high results with classical music. It simply means that quality reached by LAME at 96 kbps is suitable on poor/average listening conditions (with some exceptions of course: bad notations on kraftwerk, SinceAlways or harpsichord), especially with classical. Another point: LAME is the only encoder which automatically resample to 32 KHz. The choice is a very pertinent one in my opinion.
• Real mp3enc V5.0 rev.6: mixed feelings for this encoder. Quality is not comparable to LAME's performance, similar but slightly inferior to Fh.IIS 'Audition' (in any case prohibitive) and also superior to iTunes of Fh.IIS 'ACM'. But if we take into consideration the encoding speed, the performances are much more enjoying. This encoder is 4 or 5 time faster than LAME, and I'm pretty sure that the quality could easily be improved by resampling the output to 32 KHz (any user could do it with foobar2000 for instance, but I recall that my test consists of testing various MP3 encoding solutions with default settings).
This MP3 pool was interesting.
- First, it reveals that Fraunhofer encoders are far from being superior to LAME at bitrate < 100 kbps. In fact, LAME is obviously better with most samples I've tested here.
- Second: iTunes reveals another time severe flaws. I know that Roberto still regrets bad choice made for his MP3 test (iTunes was tested with lower bitrate than other contenders). But here, even with few additional kbps iTunes MP3 appears as a poor encoding solution, especially with 'various music'. Obviously, iTunes MP3 doesn't need to be handicapped by wrong setting to finish last...
- Third: ultra-fast encodings doesn't necessary ruin the encoding quality. REAL (ex-XING) illustrate it. Acceptable quality is possible, even with VBR at low bitrate, even with Turbo enabled.
EDIT: many thanks to Enig123 for his work