But how come I had different results while testing on the same machine - an FSC Amilo Xi 2550 notebook with a dual core CPU with XP and Vista? I am really confused.

Edit: Found the problem. XP was really set to /numproc=1 /noguiboot.

Tested the new 8.0.1 version and I get the same results on my Vista machine.

Oh, is there a new version of iTunes?

Have you already tried the effect of the quality settings on XP and Vista? It would be useful to know if those settings are ineffective when the encoder appears to be creating only ABR like MP3 files.

I only have a Vista machine at home and today is Unification Day in Germany so I am not at work. I can try on a VM with Vista and XP, though (which I am going to do now).

Edit: And yeah, I turned on my PC today and the Apple Updater popped up.

I'd guess the settings have a clear effect whenever the files appear to be real VBR, like in my case.
I can't test the other case because none of the PCs I have access at the moment produce those ABR like results.

BTW, do you have a multicore machine or a singlecore one?

Single.

My "HTPC" would have HyperThreading, but it is having strange problems and I have disassembled it for testing the individual components. I may need to replace the motherboard. It is the machine that produced the lower iTunes VBR bitrates a year ago (on XP).

It seems iTunes encoder has some problem on multi core machines. How does it fit with nao's result on a MAC, is the G5 a multi core machine too?

I don't have iTunes, does it feature some switch/check-box to en-/dis-able multi core usage?

So, I just did some tests with fatboy on my dualcore machine running Vista. The lowest VBR quality setting produces 118 kbps, the mid one 120 kbps and the highest one 122 kbps. The two options Smart Encoding and Filter 10 Hz also have an impact, but a very small one (bitrate difference was only 1 kbps).

I am afraid iTunes does not have an option to enable or disable multicore usage.

While at it - Alex B, could you please encode fatboy to 128 kbps VBR AAC using iTunes? Wondering if the AAC encoder has the same problem. Please tell me what bitrate fb2k reports.

My G5 machine has a dual-core CPU. Only G4 machine has a single-core CPU in my test. So, probably the insight that iTunes mp3 encoder has a problem on multi-CPU environment is correct. Unfortunately, there is no option in iTunes to turn-off multithreading.

OK, that convinced me. I filed a bug report to Apple. Hopefully they will reply.

BTW, regarding the AAC. fb2k reports 131 kbps and iTunes 128 kbps.

So the quality settings have some very slight effect. Apparently the selected setting is memorized and used, but it does not work properly (I suppose we can assume that the setting is designed to be able to adjust the amount of bitrate fluctuation.)


From foobar's file properties window:

File Name : fatboy_30sec.m4a
File Size : 478KB (489 618 bytes)
Duration : 0:29.207 (1288011 samples)
Sample Rate : 44100 Hz
Channels : 2
Bitrate : 131 kbps
Codec : AAC
Codec Profile : AAC LC
Encoding : lossy
Tool : iTunes 8.0.0.35, QuickTime 7.5.5

iTunes AAC encoder does not support multi-threaded encoding, so probably it is not affected by the problem.

Sebastian, please upload the 122 kbps "VBR 112" sample. I'd like to compare the files in a listening test.

Actually a "dual core" VBR 128 encoded sample would be better. It would have been the selected setting if the bitrates were tested only on machines that trigger this problem.

Here it is: http://www.hydrogenaudio.org/forums/index....showtopic=66253

Did you notice the following text just below encoder settings:
for dualcore and the same text without "using MP" - for singlecore?

I encoded one album (~1 hour long) with MP3 CBR 160 kbps:
1 core: 3m 20s (= 200s)
2 cores: 1m 57s (= 117s)

The files have different comment tag, but identical audio content. With VBR, this isn't so:

VBR 112 kbps:
1 core: 136 kbps 3m 37s (= 217s)
2 cores: 118 kbps 2m 12s (= 132s)

So multiprocessing support is really good but gives incorrect result with VBR enabled.

A listening test didn't bring up any further surprises.


As expected, both samples were seriously distorted. I did the ABX test only for checking that I can reliably distinguish the MP3 samples from each other.

Both were bad, but the single core VBR112 version was clearly better.

Does anybody else think we should consider dropping iTunes as a contender as a result of all this? In my opinion, we shouldn't have buggy encoders in the test.

But Apple can fix this issue easily just by disabling multiprocessor support when VBR mode is on .

That's exactly the reason I wanted to compare the two files. I wanted to know if the "single core" version produces expected (i.e. better) quality when the encoded file has a higher bitrate and thus the single core version could be considered as normal.

I doubt Apple will fix anything quickly so we have only two practical options: either we can drop iTunes completely or we can test samples that are created with a single core machine. We can test them now and Sebastian can decide later what to do. In the past Nero's AAC encoder has been disqualified from the final results because it wasn't working correctly. However, its results were published separately because that was considered to be fair for the high quality Nero encoder and because the fixes Nero did after the test wouldn't have changed the test results considerably.

EDIT

It might be good to test a few other samples before making the decision. The fatboy sample is a real killer so maybe a couple of easier samples should be tested.

Yes, this is common practice for iTunes and has been for sometime now with both their mp3 and AAC encoders. It wasn't until recently where Apple changed what iTunes would display for iTunes encoded VBR AAC files. It would display the overall average bitrate and that only confused people. They would encode a 128kbps VBR AAC files and iTunes would show a bitrate of 130kbps+. Then Apple changed things again so that iTunes would display the setting used. For a 128kbps VBR AAC file, it would display 128kbps (VBR) for the bitrate thus confusing less people (I guess). It is a shame as I would rather have iTunes display the overall average bitrate of songs (it does this for Lame encoded mp3 files) and then put (VBR) in parenthesis (again, it does this for Lame and FhG encoded mp3 files).

It is true that Apple can fix display/encoding issues with iTunes rather quickly but they don't. Apple just released a new version of iTunes and it doesn't look like they are going to update it anytime soon (at least the mp3 encoding part). The iTunes mp3 encoder receives updates even less frequently than either iTunes or its AAC encoder. So don't hold your breath for an mp3 encoder update even if it is a reported bug. Everyone on hydrogenaudio could report this bug and it still wouldn't speed Apple up. Alex B is right though, some more tests should be conducted before ruling out iTunes.

I also agree that other samples should be tested for differences. It could provide some sense of the bug's scope. But I'm still sceptical about including iTunes, even after just this one result. IIRC, the bug in the old NeroAAC contender was only discovered after the test had begun. So we had the results of that contender already. Since we know of this bug before the test, it doesn't make sense to me to ask people to listen and rate it.

I installed several iTunes versions and tested them with fatboy (5-sec long) sample. Encoding settings are the same: 112kbps (VBR, max quality).

iTunes 5.0.1.4 was released at september'2005. No-one noticed this bug before?

I guess that is because a small community uses the iTunes mp3 encoder, all others look towards Lame and the various programs using the FhG encoder (Windows Media Player, the Zune PC software, etc.).

Thank you for your findings lvqcl.

Do you guys think I should wait x days for Apple to respond to my mail or should I continue with the test right away? I have a singlecore Celeron notebook, so it wouldn't be a problem to encode the samples and start the test.

It's one thing that you have a single core machine to make the encodes, but many people don't have a single core computer. So, in the event that iTunes makes a strong showing in the results, and therefore people are more comfortable using it, they won't get the same performance from *their* encodes. The iTunes contender samples would not be representative of real world use.

Of course, it may be that Apple fixes the problem, but I wouldn't hold my breath.

Of course, I would include a big "WARNING" or something like that on the listening test page.

I still think it should be disqualified. These tests are time consuming and require a lot of unbroken concentration. I was planning on participating but I don't think I will if this broken encoder is included.

Seriously, what is the point in having a rank on a contender when its samples can only be produced on single core machines?

I could also do the same thing I did with the Nero AAC encoder after Guru noticed the bug with the bitrate allocation, although that was a slightly different story.

It would be good if Apple would acknowledge that the encoder has a problem with multi-core machines and that the encoder behaves as intended on a single-core machine. Then we could assume that Apple will eventually fix the problem and the test results would be usable for all iTunes users.

BTW, has anyone tried what happens when the machine has more than two cores?

As Sebastian have said, in general it would be very interesting to have up-to-date test results of the iTunes MP3 encoder. It is probably the dominant MP3 encoder on Macs and many professional content creators work on Macs.

EDIT

After finding out that iTunes isn't faster than LAME I was against testing it because I didn't see the usefulness of the results in a wider perspective beyond my personal needs, but I have changed my mind.

I could test on a 8 core machine... (4 real, 4 synthetical cores)

Just did a quick test and encoded at 112 and 128 kbps. The results are just like with two cores. 112 kbps file comes out at 122 kbps, 128 kbps file comes out at 141 kbps.

I am trying to get in touch with someone from Apple directly. If I don't get any feedback soon or if the fix is supposed to come in a version that will be released too late (let's say in a month or so), I guess the best thing I can do is to 1. display a big warning on the presentation page and 2. display the iTunes ranking detached from the rest like I did for Nero. As Alex B pointed out, iTunes is a really popular MP3 encoder at least on Macs and I would like to avoid disqualifying it entirely.

Could the mysterious Apple developer mentioned in the following quote help contacting the right person(s) at Apple?


I have no idea who this person could be.

I do...

You have to be a HA oldtimer to know that But he posted pretty publically on this forum in the past.

Does anyone have an idea of what Apple could have done wrong to cause an unlikely problem like this?

Could Apple actually have included two separate MP3 encoder components and made a mistake when configuring the internal options?

If we would need to "reverse engineer" the situation, what would a developer use for intentionally making the encoder behave differently on different machines?

Some digging: http://developer.apple.com/quicktime/whatsnew.htm

- "Multiprocessor (MP) Support" was added to QuickTime version 5
- "VBR Sound Compression Support" was added in QuickTime version 6

It's bad, that there seems to be no way to control the MP feature in QuickTime.

Alex B
I think the answer is quite simple. If you want to use full power of a multicore CPU, then you need to split the computing into more threads. This means you have to significantly change your algorithm. I heard before that you will get different results with multithreaded versions of libavcodec, x264 or XviD codecs.

Multicore CPU is IMHO a very bad idea and direction. But CPU manufacturers had probably no other cheap solution on how to further increase power of their processors. If you want to use all cores at the same time in your encoder, you have to write it differently (much more complicated) compared to single thread encoder.

I hope this explains why you get different results on singlecore and multicore CPUs.

Multicore is not a bad idea. People normally have more than one program running. So each core can take care of each program.


Well, somewhere the split information processed on each core must be merged. You can have an issue there.

By the way, are CBR results binary identical on single and multi core machines? Do they have a problem there too?

Just wanted to let you know that someone from Apple is investigating this.

Let me quote myself:


iTunes 4.9.0.17 uses QuickTime 6.5.2 and iTunes 5.0.1.4 uses QT 7.0.2. So I doubt that this particular bug was introduced in QT 5 or 6.

I also tried to install iTunes 4.9 and then upgrade QT to 7.5. But this didn't affect MP3 coder at all: mp3 files created before and after upgrade were bit-identical. (AAC encoding was changed, however: when encoding to AAC, iTunes crashes )

Probably iTunes mp3 encoder doesn't depend on QuickTime, because QT API has no interface to produce mp3.

Good news! I just received a mail from one of the persons in charge and the bug was indeed identified and should be fixed in the next iTunes version. Thanks to the Apple developers for reacting so quickly and also thanks to the HA members who supported in identifying the problem.

Edit: BTW, it seems that in some cases, CBR encoding can be affected by this bug as well.

Did that person say anything about which behavior is correct? I wouldn't be surprised if they would consider the single-core version broken and limit the amount of the bitrate variation.

BTW, I'd like to see bitrates of all 14 test samples when they encoded with iTunes on single-core CPU (just out of curiosity)

Single core behavior is correct.