After reading about the listening test here on HA praising 128kpbs iTunes AAC and such I decided to do my own listening test with real music I listen to.

I'm not very experienced with this, so I started at 64kbps to spot problem regions under the assumption that these will remain problem regions with similar artefacts at higher bitrates. Besides lots of other things I noticed that the stereo image was not correct, but thought this was normal for 64kbps (maybe its using is, or whatever).
However, to my surprise the stereo image problems remained up to 128kbps "VBR" (I was going in steps) and even at 192kpbs "VBR". I can't imagine no one noticed this untill now. Even 192kpbs is a piece of cake to ABX cause of this.

To make sure it's not the decoder I also decoded using iTunes with the same results.

Original sample can be downloaded here.
(Since I'm new to this: should I have uploaded the encoded version, too?)

The thunder right at the beginning is not correctly positioned. Also the second, higher frequency synth sound (the one that is percusion-like used and to the right, first used after the main synth "melody" begins) sounded strange at 128kpbs (and is also not correctly positioned) -- didn't bother to check at 192kbps.

Granted, the problem is not an anoying one -- I probably couldn't tell which is which without direct comparison, but anyways I'm less than impressed by iTune's performance.
I wouldn't have thought that AAC is so easy to ABX even at 192kbps VBR:


iTunes version used: 6.0.1.3

Can anyone confirm my findings? Is this a known problem?

Edit: wording, last sentence

What hardware were you listening on? Headphones, loudspeakers, soundcard...?

I was listening using onboard sound (NForce4 w/ ALC655) connected to an amplifier and Beyerdynamic DT801 headphones.

Edit:
Out of curiosity just did a test with 320kbps AAC: 10/10.. This can't be an encoding artefact, can it? I guess it's a strange bug or something (or the thunder is a killer sample, don't know)..
However the stereo image on the synth sound is correct at 320kbps (or I'm tired). It may also be some encoding artefact that I perceive as a changed stereo image. I think it was more subtle at 320kbps, but still no problem.

Edit2:
I just tested LAME 3.97b2 -V5 --vbrnew (112kbps according to foobar): It was harder for me to ABX. Stereo is correct, but the synth sounds sound different (stopped at 6/6 because I'm tired).

Edit3:
It was -V5 --vbr-new, of course.

How about trying another AAC encoder like Nero?

Reading that makes me wonder if maybe you are doing something wrong during encoding to iTunes AAC, maybe transcoding? something else?...

I started with a wav I ripped from the originial CD for this listening test. I used foobar with itunesencode to encode this wav to AAC.
Later I cut the wav to 30s and compressed it with flac to upload it in the upload forum.

I did not transcode from a lossy source or whatever. Also in foobar's diskwriter prefs nothing is enabled that would change the sound (dsp, rg, dither etc.).
Besides I used the same procedure with the other encodes.

I just tested with Nero AAC (some old version I happen to have installed):
internet profile - stereo image is ok. I didn't try to ABX as I'm really tired now. That's also why I will retry the Nero test tomorrow.

It would be nice if someone with experience (no pun intended *) and good ears would try the sample I uploaded. At 128k it's absolutely apparent (to me).

*The sample happens to be from the album "Experience"

Edit:
Just retested iTunes and Nero (despite being quite tired ):

Nero:
Even at internet profile didn't have any stereo problems. I didn't listen for other artefacts.

iTunes:
At 128k VBR very very obvious stereo positioning problems.
At 192k VBR still obvious stereo positioning problems, easy to ABX, but less pronounced than at 128k.
At 320k subtle stereo positioning problems. 100% ABXable, but not that easy.

Just noticed there is a newer iTunes version out. Updated and retried with iTunes v6.0.2.23 (QuickTime 7.0.4): exactly the same.

An update:

Because no one answered wether they hear the same problems with iTunes encodes, I asked my brother to do an ABX test. I wanted to know if I'm the only one noticing this.

This was the first time he did such a test an he's not really interested in this stuff (nor does he listen to music with headphones regularly) -- read: he's absolutely unexperienced. He did pass the 128k VBR test with 10/10 and confirmed the wrong stereo positioning.

Since the last post I also tested this with lots and lots of other samples and this problem was easily heard with all of them.

Just to show you how easy it is for me to ABX at 128k VBR:


.. 1-2 secs per vote. Just need to listen to X to immediatly decide.

This is a serious bug in the iTunes encoder. Although the effect gets less pronounced with increasing bitrate it is still there at 320k.

I will upload a 128k encode here so it's easier for others to test.

I can confirm the stereo positioning problem, but not the artifact on the synth sound.

Thanks for testing. I'll upload a 192k encode, just in case you want to test that one, too.

The synth sound sounded strange to me at a first glance, but I can't be sure of this (it was just meant as a side note). Because the positioning is so obvious, I'm not under DBT-condition wrt the synth sound anymore so it's pointless for me to really investigate.

I actually did my own encode and tested both yours and mine, they were the same, sorry for doubting you earlier.

I also can confirm strange behavior of iTunes (6.0.3.5) aac encoder. The killer sample is glockenspiel. Even at 320cbr. There are two distinct ringing artifacts in the right channel. The sample is uploaded here.

Did you also try CBR mode in iTunes? Something tells me that CBR mode is alot more optimized than VBR encoding when it comes to the Quicktime/iTunes AAC encoder.

Downloaded the samples and will start ABX'ing in a few hours (kinda busy now).

I didn't try this sample but I can say I did an abx 8/8 on a song Hush by Tool about a mnoth ago.
The comparison was between MPC q5 and iTunes AAC 192 VBR.
In my opinion AAC was worse on handling the snares, chrashes and similar drum kit parts.

It is 320 CBR.

You don't need to ABX them - artefacts are very very audible. Just listen them.

drezon,
I could try this with iTunes5 if you'd like. If so, please upload the original same in wav format.

This might seem like a bad question, as I am new to AAC... but in iTunes, under the AAC custom settings --> channels, are you forcing stereo or using the "Auto"? Does this make a difference?

Just go here, and download the FLAC file. Decode it then use the WAV

The 'auto' option will just choose whether to encode the file as mono or stereo depending on the input file. It shouldn't make a difference on the encoding (unless you encode a mono file as stero and vice-versa).


[ EDIT : Spelling ]

Decode it with what? I don't believe iTunes or EAC will do it, and I'm not interested in installing software to read/convert FLAC for just this one-time listen.

You can download it from here.

I just tried this test using the thunder sample:
WAV
AAC 128kbps VBR (iTunes 6.0.3)
MP3 LAME -V 2

It appears that the AAC encoding is rendering the low frequencies of the "thunder" sound with less channel separation - more mono information. This seems to be much more of the effect, as opposed to shifts in the frequency domain.

I am sure that this is only detectable with headphones, as such low frequencies would always sound essentially mono over loudspeakers for obvious reasons.

LAME -V 2 does not appear to have an issue in my very short tests.

The artifact is not unmusical, and I have never noticed anything "wrong" with low-frequency AAC soundstages on typical material - even older recordings with "one channel bass" using headphones.

Interesting!

It's hard to perform any ABX test with the file (two samples merged into one file) you uploaded
But you're right, the artefact isn't very hard to spot. I could clearly hear one after the second glockenspiel note. It's a kind of "blip" occuring after the attack. You can also "see" the visual representation of the problem:

Nice movie, guru.
Glockenspiel turned out to be a problem sample for both CBR and VBR modes (all bitrates) of iTunes (6.0.3.5) aac encoder. BTW, this is the main reason why aac (iTunes 6.0) is just a little bit better than mp3 (Lame 3.97b2) in SoundExpert Multiformat Listening Test @128 kbps. Also it’s worth mention that wma 9.1 pro produced very annoying artifacts with glockenspiel sample in the test as well.

Is that all this is about? And to think it had me worried. I agree this isn't a problem unless you listen to music with headphones. It is generally accepted that frequencies below 80hz are non directional -- that is the listener cannot discern from what direction they came. I only use headpones when I absolutely have to so this is a non issue for me.

I don't think it has to do with low frequencies as I'm equally able to detect the wrong stereo image with most synth sounds (in the posted sample as well as other music I tried).

I also tried classical music and instrumental and there I was not able to hear wrong stereo image (with the small number of songs I tried at least).



Indeed it is not that bad, but a serious bug nonetheless. Esp. since it is even hearable at 320kb. Also at 128k it is so bad that I can identify the AAC encode without any reference (I just listen to a random permutation of orginial and AAC and at each stage say which is which).



I guess you have to be worried. I just ABXed the 128k encode with speakers -- the first time I ABXed with speakers btw.:


Edit: Same score for 192k w/ speakers.

Yes, but a bug is a bug and it has to be fixed. According to the OP, the problem does not seem to be apparent using Nero AAC. So, the problem seems to be with Apples' AAC encoder and it does need to be fixed.

Didn't Apple update the encoder between February and now? Is the bug still present in iTunes 6.05/Quicktime 7.1.2?

Just redid the test with latest iTunes (6.0.5.20). As expexted (at least by me) still exactly the same.

I wouldn't hold my breath for a fix. Apple still didn't fix the MP3 VBR skipping on iPod Mini/Mini 2ndGen/4G. It seems Apple sucks even more than Microsoft when it comes to fixing bugs (which is quite an achievement!).

OT:
Since Februrary I rockboxed my two iPods (Mini 2nd Gen and 30gig Video) and am using Ogg Vorbis -q5 now. I did some tests with latest Nero encoder and found that for the limited material I tested with Ogg Vorbis (lancer based on AoTuV 4.51) reached transparency at -q5 (bout 150kbps) for me while Nero AAC needed ~170kbps.

Also the new Nero encoder (free standalone version) did have some stereo positioning problems below ~150kbps. Not anywhere near as extreme as iTunes and in contrast to iTunes completely going away at ~150kbps, but the old Nero encoder didn't have that problem.

Bottom line: I don't care about AAC anymore. Ogg Vorbis reaches transparency at lower bitrates (at least for me, with the limited material I tested with). It didn't have any stereo problems whatsoever (even down to -q1), and even -q1 (~80kbps) produced very few artefacts. So few indeed, that I think I won't notice them under normal listening conditions.

To my best knowledge, the fix will be shipped along with Mac OS X 10.5, and its previewed version will be available in WWDC 2006.

You wouldn't happen to know other fixes/extras in store for that version (gapless playback for instance), would you?

Good to know the bug report has been read (I wasn't actually sure about that) and a fix has been developed. This lets me hope that Apple isn't as bad as I pictured above (I was actually a bit exaggerating , though the MP3 issue still holds, as does the distorting eq, no gapless playback etc..).

Any chance for you to tell us when there will be a fix for Windows Quicktime / iTunes?

excuse me, but what is this ABX?

Sticky (in General Audio): What is a blind ABX test?

I kept havng the same miserable results until today, actually. I updated to OS 10.4.8 and iTunes 7.0.1. Just tried encoding the "glockenspiel" file at 160 and 256 kbps VBR AAC, fully expecting the results to be the same, but, amazingly, the bug appears to have been fixed. Has anyone else noticed this?

No, I didn’t notice the “glockenspiel bug” disappeared (I used iTunes 7.0.1.8 on my Windows XP SP2 system), it still persists.

You're right. I took the time today to compare a bit more carefully. It looks like I got a little too ahead of myself. The artifacts are still there, but not as pronounced. However, there is a very noticeable difference since the 10.4.8 update. It appears that someone at Apple attempted some sort of fix to the encoder, but still didn't quite succeed. I guess it doesn't matter because broke is still broke, so I guess I'll just have to continue using LAME 3.97 mp3.

As a fellow iTunes user, I would like to extend my appreciation for your finding, reporting and pursuing this quality issue. However I am curious, as quality appears to be such an issue with you, why don't you simply use a lossless format like Apple Lossless? Doing so reportedly also apparently also improves gapless playback as well.

John,

The point behind lossy formats like the iTunes AAC format is so you can carry your music around with you on your protable without eating a hole in the battery. So yes, quality is a issue but not the point of only putting 200 songs on a 60GB iPod just to retain that "true CD quality." Additionally, all updated 5G iPods, all 5.5G iPods, and iTunes 7 now support gapless playback of mp3 (both iTunes and Lame), iTunes AAC, apple lossless, wav, etc. files so there really is no need for lossless unless you want to have a digital backup of your CD's.

Just to let you know, I tested thunder.wav again with itunes 7 and I didn't notice any change. Stereo positioning still borked as usual..

I hope Quicktime 7.1.3 isn't the "fixed" version skuo was talking about.

But even if the fix finally comes with Leopard (due to somewhen in early 2007): one year to fix a (IMHO serious) bug in probably the most used AAC encoder is way too long. Especially considering that all iTunes-Store songs are encoded with a buggy encoder (with no way for the end-user to reencode) -- another reason not to buy lossy music.

As time went by, Quicktime 7.2 is out.

Any news on this front?

I'm still waiting to start converting a TB of flac files to AAC. This bug discovered by drezon really saved my day when he discovered it.

Quicktime 7.2 should be the version skuo was talking about - maybe prereleased due to the delay of the whole Leopard package.

Can anybody confirm that drevons discovery has been fixed? Apple had more than enough time to fix such a serious issue.

Damn, it is not fixed!

I just did a blind test using Westone UM2 directly plugged into my Mac Book. I instantly get 10/10 up to 256kbs AAC. The original thunder just cuts "sharper" through the sound stage.

That's really a shame. Skuo, can we expect anything different for Leopard, yet? For my ears that's unusable.

Apple is really risking very bad press about their new 256kbs "'Premium" tracks. Any audiophile journalist who picks this up could build a catchy story on top of simple, provable facts. That's not premium at all.

You are worried about one killer sample that isn't really music? Every encoder has its killer sample.

I was kinda thinking the same thing. Every encoder has killer samples. Even Lame has some samples that people can effectively ABX at 320kbps. I guess I don't see what the big deal is. It might be a bug but this bug is being heard on this sample (and probably a couple of samples). If Lame is allowed some killer samples, then can't iTunes AAC be allowed some as well? It is not like I use iTunes AAC (I use Nero AAC for their better VBR implementation and now the iPod/iTunes support gapless playback of Nero AAC files) but I think some people should cut it some slack instead of focusing on this one sample.

As previously mentioned, it is supposed to be fixed in the major encoder update that will come out together with Leopard. Since that upgrade is still a month or two away you shouldn't be surprised it is not fixed yet.

You are right.

But I hope that Itunes vs Nero at some bitrate commonly used on portable players will be tested more widely asap.

We at Nero are always willing to improve quality of our encoder, so whenever killer sample for Nero AAC Encoder is found please report it here at forum and we will check out what can be done about it.

[quote name='drezon' date='Feb 8 2006, 16:03' post='362873']
After reading about the listening test here on HA praising 128kpbs iTunes AAC and such I decided to do my own listening test with real music I listen to.

I'm not very experienced with this, so I started at 64kbps to spot problem regions under the assumption that these will remain problem regions with similar artefacts at higher bitrates. Besides lots of other things I noticed that the stereo image was not correct, but thought this was normal for 64kbps (maybe its using is, or whatever).
However, to my surprise the stereo image problems remained up to 128kbps "VBR" (I was going in steps) and even at 192kpbs "VBR". I can't imagine no one noticed this untill now. Even 192kpbs is a piece of cake to ABX cause of this.

To make sure it's not the decoder I also decoded using iTunes with the same results.

Original sample can be downloaded here.
(Since I'm new to this: should I have uploaded the encoded version, too?)

The thunder right at the beginning is not correctly positioned. Also the second, higher frequency synth sound (the one that is percusion-like used and to the right, first used after the main synth "melody" begins) sounded strange at 128kpbs (and is also not correctly positioned) -- didn't bother to check at 192kbps.

Granted, the problem is not an anoying one -- I probably couldn't tell which is which without direct comparison, but anyways I'm less than impressed by iTune's performance.
I wouldn't have thought that AAC is so easy to ABX even at 192kbps VBR:


iTunes version used: 6.0.1.3

Can anyone confirm my findings? Is this a known problem.

Don't you have to use more samples for each test? I've noticed that I can get first 4 or 5 right because the abx program does not switch between the two samples (x, y) immediately. If your testing withing the first 15 seconds, one after another, they will all be the same. I think you should try about 20 samples over a 3 minute song to really just the accuracy of an ABX test.

This is not a "killer sample" but rather, in my opinion of course, a bug in iTunes encoder - and I think a very old one.

Same problem with the same encoder happens with usual MPEG test clip called "glockenspiel" (forgot the exact name of the file, but it is easy to find it) at the end when bell sound goes into slow fade-out, the noise spectrum starts jumping up and down without apparent reason - it seems to be a bug in estimation of the AAC scaling factors, where too much noise is modulated almost few dBs below the signal energy.

I remember that one from long time ago (during time we were tuning our SF estimation ) and it is a bit strange people @Apple did not fix it yet.

Exactly. This is no strange synthesized killer sample, but a stereo recording of a natural phenomenon.

Stereo recordings of thunders have very complex stereo images extending several kilometers. That's a perfect benchmark to separate men from boys regarding the preservation of spatial information. I bet that any natural recording of a thunder which has been recorded as close to the source as this one would make Quicktime fail at this point.

You may ask now why in hell would we need the capability to reproduce kilometer wide soundscapes, if we just want to listen to ordinary music? Well, a decoder which can handle a thunder just indicates very good algorithmic solutions for spatial representation. It scales from tiny space to landscapes and is able to represent images containing both without trade-off. It hints an elegant solution instead of a patchwork of special case optimizations.

And what about the sound of a great opera hall or plain reverbation in pop music? Artist like Björk work for weeks on the perfect sound of single samples. If you buy it on iTunes you may never know what she really had on her mind what it should sound like. Take this thunder sample. It brutally cuts through the scene in the original. The 256kbs Quicktime sample sounds completly "thundery" but severely misses the sharp cut through the scene. If a musician wants you to hear and feel that cut, he can't get this message to your ears using the iTunes AAC platform right now.

If Nero's encoder is able to handle this, I congratulate sincerely. I also appretiate muaddib's message about Nero's attitude towards problem samples. That encoder seems really worth that I give it a try.

I haven't followed the compatibility issues for a long time now. Can they be considered solved? I'm talking about iPod compatibility, iTunes tags and gapless playback on both.

As far as I know there are no known problems. iTunes/iPod handles the Nero gapless info (not the other way around, but in your case that doesn't matter I think). And the Nero tools write all the tags in iTunes and NeroDigital format by default.
Just have to make sure your files are called .m4a and not .mp4