HE-AAC v.1 & v.2 comparison, Winamp vs Helix vs Nero Digital Options

[guruboolez]

The very new Winamp “surround edition” (5.1) and its own encoding solution for low-bitrate is a good occasion to make a comparison between different HE-AAC encoders. Preliminary comments posted on this board were clearly enthusiastic, especially for aacPlus v.2 quality. I’d like to contribute to the evaluation of this new encoder and as usual to post a complete report of my personal listening test here.

From now I will be brief.



Samples:

I used 2 series of samples I already used in the past:
• 40 samples mixing “classical” (25 samples) and “various music” (15 samples) (serie used here for all pools.
• 6 samples coming from DVD ripping (48000 Hz files transcoded from original AC3 files) (full serie already used here.

Encoders:

• Coding Technologies “aacPlus v.2” : 48 kbps with SBR and “Parametric Stereo”
• Coding Technologies “aacPlus v.2” : 64 kbps with SBR and “Stereo” (not parametric)
• Helix “Producer” v. 11.0.0.1897 HE-AAC: 64 kbps with SBR
• Nero Digital aacenc32 v.3.2.0.20 HE-AAC: 64 kbps with SBR

Software & Hardware setting:

• Creative Audigy 2
• ABC/HR for Java 0.5 alpha 5
• faad 2.1 MPEG-4 AAC decoder

Important to note:

• the test consists on pure ABCHR evaluation
• offsets were removed and gain was always corrected (difference could reach 1.5 dB!)



OVERALL RESULTS

CODE

WINAMP WINAMP HELIX NERO
48 kbps 64 kbps 64 kbps 64 kbps

A02 3.3 3.5 3.5 2.5
E06 3.0 4.0 4.0 1.8
E15 3.0 5.0 4.8 4.3
E22 3.5 4.0 4.0 3.0
E26 2.7 3.0 3.0 2.0
E31 3.0 3.7 3.7 1.0
E40 3.0 3.5 3.5 2.5
E51 2.5 3.5 3.5 2.5
E53 1.5 2.5 2.5 1.8
S03 4.0 4.5 4.5 3.5
S08 2.5 3.5 3.5 1.5
S12 1.5 1.8 1.8 1.0
S17 3.0 2.5 2.5 1.0
S27 1.2 2.0 2.0 1.7
S38 3.0 3.5 3.5 1.5
S50 2.0 3.5 3.5 1.0
S54 2.5 3.0 3.0 2.0
V02 3.8 3.8 3.8 4.0
V07 3.9 3.7 3.7 3.0
V10 3.5 3.5 3.5 3.5
V15 2.0 3.0 3.0 2.0
V19 3.7 4.2 4.2 4.5
V20 3.0 3.5 3.5 1.0
V24 1.5 4.0 4.5 1.5
V27 3.5 2.0 2.0 1.0
---------------------------------------------
CLASSIC (MEAN) 2.80 3.39 3.40 2.20
---------------------------------------------

41_30sec 4.0 2.5 2.5 1.5
ATrain 3.0 3.5 3.5 1.5
DaFunk 3.5 4.2 4.0 1.5
death2 2.5 2.5 2.5 1.5
EnolaGay 2.3 3.0 3.0 1.0
experiencia 3.5 4.0 4.0 1.5
getiton 3.5 4.3 4.3 2.0
kraftwerk 2.0 1.8 2.5 1.0
LifeShatters 2.8 3.0 3.0 2.0
NewYorkCity 3.0 3.5 3.5 2.5
OrdinaryWorld 4.0 2.0 2.0 1.0
Quizas 3.5 4.0 4.0 1.5
rosemary 3.0 4.0 4.0 1.5
SinceAlways 4.0 2.5 2.5 1.0
trust 1.0 2.0 2.0 1.5
---------------------------------------------
VARIOUS (MEAN) 3.04 3.12 3.15 1.50
---------------------------------------------
Alien 4 3.5 4.0 4.0 1.5
Farinelli 4.5 4.8 4.8 2.0
Come Drink... 5.0 5.0 5.0 5.0
Ran 3.5 4.0 4.0 3.7
BlackAdder 3.0 3.2 3.2 1.0
Pulp Fiction 3.5 3.3 3.3 2.7
---------------------------------------------
DVD (MEAN) 3.83 4.05 4.05 2.65
---------------------------------------------

=============================================
46 SAMPLES 2.89 3.29 3.31 1.94
=============================================

ABCHR logs are available here; I stopped to comment the notation after ~20 tests.



• Helix & Winamp 64 kbps are highly similar. It might be possible to differentiate them with an ultra-careful listening, but my notation is only based on an attentive but fast evaluation. I suppose that Helix HE-AAC encoder is also coming from Coding Technologies. Apart quality, both have the same offset and a very similar gain (volume) deviation.

• 48 kbps HE-AAC with Parametric Stereo is inferior but not very far from 64 kbps HE-AAC with “regular” stereo coding (I guess it’s joint-stereo). It’s a very good performance I would say. At such low bitrate every bit is precious and by removing 16 kbps to a 64 kbps we could expect a huge drop in quality. But here, the quality stays close from the 64 kbps encodings (and is sometimes clearly better – see below).
I quickly noticed unusual issues with PS encodings. I can distinguish three different variants:
- weird and high-pitched noise
- stereo distortion: ping-pong effect (rare) and fake-stereo effect (reverberation, echo….)
- “equalizing issue” (unbalanced sound, often hollowed, lack of bass, flattened image, etc…), which is the most common problem I noticed.

• Nero Digital AAC was the first publicly available HE-AAC encoder (released in summer 2003), but the latest version is now obviously outdated and not competitive anymore. On 46 samples this encoder finished last 36 times and first 2 times only! The quality is often really poor compared to all other contenders, including the 48 kbps encodings made with Coding Technologies/Winamp encoder.
Nero Digital usually suffers a lot from what I’m used to call “SBR artefacts”: a grainy sound, smeared impulse and also a lot of small but highly irritating noise packet/aggregation. I heard these issues for the first time with mp3PRO and found them again with the first (and only tested) HE-AAC (Nero Digital). Problems are audible even at the highest available bitrate: 96 kbps and also with forced 128 kbps. That’s why I always thought that all these issues were consubstantial to SBR technology. Now, by testing other HE-AAC implementation, I must completely change my mind. What I called “SBR defect” is rather a specific Nero Digital flaw. I don’t mean that Coding Technologies & Helix encoders are free of these problems: they’re all more or less perceptible (it really depends on the sample). In other words, there are still “SBR artefacts”. But both encoders have apparently a better control of the side effect of SBR. Pre-echo on strong impulse is also much stronger with Nero Digital (see Appendix) when compared to Helix/CT.
I’m now thinking to my 80 kbps listening test I made on the beginning of the summer. I tested 185 samples with a HE-AAC implementation which appeared to be far from Vorbis aoTuV quality, and surprisingly similar to Apple LC-AAC and not that better than Lame MP3. I’m now pretty sure that Coding Technologies “aacPlus” would perform much better and could maybe compete with aoTuV at 80 kbps (for 64 & 48 kbps, I fear that it would be hard for aoTuV to stay competitive against the SBR beast)… I read somewhere that Nero 7 is announced for October. I guess that the new generation of Nero Digital HE-AAC will be released with the new Nero package and I’m sure that Ivan had built a competitive encoder.




DETAILED RESULTS




• classical group : parametric stereo doesn’t work very well with several samples tested here. The 64 kbps are clearly more enjoying (but are of course bigger). Nero Digital didn’t perform very well, but at least it’s not so far from 48 kbps PS encodings. The Winamp/Helix duo gets a better note with “classical” samples than with “various music” ones. This is another surprise for me: I really expected from SBR to work (much) better with louder music.

• various music group : Parametric Stereo is working very well here, and at 48 kbps the results are very close to 64 kbps encodings without PS. But a closer analysis reveals that these good results are contrasted. Most often 48-PS encodings are worse than 64-JS; not too worse but worse. But with three samples 48-PS encodings are better than 64-JS: not only better but much better (to my ears – see Appendix for illustration). The incriminated samples are 41-30, OrdinaryWorld & SinceAlways. What happens? The Parametric Stereo encodings are free of nasty artefacts audible with 64-JS, located on cymbals mostly, and which consists on aggressive and sandy noise (except for 41-30 which has another problem).
Second surprise: Nero Digital results, worse again compared to competitors, but also worse with “various music”. Nero Digital was used to have problems with classical music; here it’s the opposite. Most often encodings produced by latest Nero Digital encoder are simply unlistenable. These unbalanced results in favour of “classical” samples are confirming the results I got after the 80 kbps LT I made this summer.

• DVD transcoding : there are too few samples to make any strong conclusions. First comment: notations are higher (for all four encoders) with this group of sample. This could be partially explained by the presence of one mono-encoding which sounded transparent with all encoders (which obtained as consequence 5 points for this sample). Helix & Winamp quality at 64 kbps are really excellent. I’m not fond of DVD ripping but I think I will consider HE-AAC again (I was very disappointed by my previous tests, all made with Nero…). Nero Digital performs less badly than with music encoding, but is still far from all other competitors, including the 48 kbps “aacPlus” encoder. It confirms my previous experience with HE-AAC and DVD ripping: poor. The usual artefacts of Nero are also audible, altering voice as well as music.



APPENDIX

• three different statistical analysis of the complete results are available here:
http://audiotests.free.fr/tests/2005.09...ots_mixedx3.png


• Illustration of Nero Digital ultra-smearing issues with Kraftwerk.wav sample:

http://audiotests.free.fr/tests/2005.09...1/kraftwerk.gif
http://audiotests.free.fr/tests/2005.09...ftwerk_Helix.wv
http://audiotests.free.fr/tests/2005.09..._NeroDigital.wv


• Illustration of Nero Digital grainy sound with S50.wav sample:

http://audiotests.free.fr/tests/2005.09/AACHE/01/S50.gif
http://audiotests.free.fr/tests/2005.09...01/S50_Helix.wv
http://audiotests.free.fr/tests/2005.09..._NeroDigital.wv

• Illustration of Nero Digital grainy sound (other samples):

http://audiotests.free.fr/tests/2005.09...01/E06_Helix.wv
http://audiotests.free.fr/tests/2005.09..._NeroDigital.wv
http://audiotests.free.fr/tests/2005.09...semary_Helix.wv
http://audiotests.free.fr/tests/2005.09..._NeroDigital.wv


• Illustration of Parametric Stereo distortions (three samples):

http://audiotests.free.fr/tests/2005.09...ereo_issues.zip
Try also rosemary.wav (beginning)


• Illustration of SBR “sand” reduction with PS encodings:

http://audiotests.free.fr/tests/2005.09...ld_Winamp_48.wv
http://audiotests.free.fr/tests/2005.09...ld_Winamp_64.wv
http://audiotests.free.fr/tests/2005.09...ys_Winamp_48.wv
http://audiotests.free.fr/tests/2005.09...ys_Winamp_64.wv
http://audiotests.free.fr/tests/2005.09...27_Winamp_48.wv
http://audiotests.free.fr/tests/2005.09...27_Winamp_48.wv


• Illustration of a minor artefact audible with Winamp and not with Helix:


http://audiotests.free.fr/tests/2005.09..._blip_Winamp.wv
http://audiotests.free.fr/tests/2005.09...k_blip_Helix.wv


P.S. All uploaded samples are already decoded (in order to any avoid biased evaluation due to possible playback with non-SBR compatible AAC decoders). They’re consequently very short – maybe too short to be convincing. Try if possible to download the full samples and to encode them with the required settings.

This post has been edited by guruboolez: Dec 29 2005, 22:40