From reading this thread:

http://discussions.info.apple.com/WebX?128...Hk.18@.3bc2a4c8

...it appears that iTunes 4.0.1 uses the "BETTER" QT quality (not "BEST") -- it sacrifices quality for higher encoding speed.

-Jonathan

i for one have already sent apple feedback asking for the qt "quality" settings to be brought into itunes so the user can choose to sacrifice speed for quality -- like i want to -- and i linked this thread to them to show how their own slight quality "win" (quicktime) can be easily discounted because their consumer encoder (itunes) does not carry the same quality. others concerned should provide feedback to apple, too. sometimes, they listen.

A fairly knowledgable (they are hit and miss) Applecare representative told me (after confirming with an Apple programmer) that iTunes uses "Good" quality for its encoding, and iTunes encoding did not change between 4.0 and 4.0.1. The main quality/encoding difference since iTunes 4.0's release is the upgrade of the QT encoder used (from 6.2 to 6.3).

I cannot verify this information quantitatively since 4.0 and 4.0.1 encoded files are different post the installation of QT 6.3. (I have both on my system.)

Strange, my encoding speed dropped dramatically with the 4.01 upgrade. I wonder if Good is like q3 for lame, better is like q2 and best is like q0.

I've been having a very fruitful e-mail swap with Apple's head AAC developer, Mr. Stanley Kuo. Here's his clarification on this specific subject:



Regards;

Roberto.

Roberto, do you still plan on adding the individual user comments and sample bitrates?

I do. Will probably do that this weekend.

I'm really sorry about the delay. This week has been terrible. Finals, listening test (and dealing with lots of people claiming my test is useless), parents came visit me (fortunately they were in route to another town),...

Some say a monument will never be erected for a critic.

I now erect ( uhhh, pile up ) one. It looks, and smells like cow feces.

I for one appreciate your hard work. I have been wondering for some time how the AAC codecs stand against each other.

Thank you

Roberto, thanks for clearing that up; it's been a subject of so many posts on many boards.

(So much for the "knowledgable Applecare". ) But, where is iTunes' option to "give the user the ability to change the encoder's mode"? I can't find it...

I am not a Mac user, so I am unable to discuss that subject further with Mr. Kuo, sorry.


The test comments have been uploaded.
http://rarewares.hydrogenaudio.org/test/aa...t/comments.html

Sorry for the sucky way I made them available. I'll try to make it better soon, it's 4:45 AM here...

Regards;

Roberto.

2 points about the results:

1 - If you submitted the results and don't see them there, it's because you didn't mention in the e-mail that you allowed me to publish them. If you actually forgot to mention and what them to be published (either with your real name/nickname or anonimously), please mail me at the same address.

2 - I would like to request people to avoid making fun of any "dumb" comments they might find. Remember there was a time you weren't experts.

Funny that everybody hears different things. If we add together all flaws that people heard there won't be much left of the original music.

One of the codecs got these comments but was still ranked second for this sample... the others must sound horrible then.

"Bad pre-echo, clicks, cut-off, warble, 'discrete' freq sweep, dirtier, noisy, tones merging, distortion or clipping, noisy attacks, rumbling, muffled, post echo tones, smeared attacks, faser sound, gurgling, noisy hi-hats"

I think this is normal, these results might not respect the personal preferences (and i'm not talking about zealots ).

According to my personal preferences, for example, Sorenson scored the same as FAAC
I'm quite surprised, for this. I remember a previous thread where Ivan Dimkovic stated this codec (AACdemo) "as the state of art" around 128 kbps. But i'm not here to start a flame, i respect the other's impressions and the tests conclusions.
Nevertheless, If you're speaking about Flooressence, i think that this sample should be considered a real problematic sample for Sorenson: very annoying, expecially when an obvious "scratch" occur at 2.2 - 3.3.

IMHO this abundance of artifacts is due to the different way people DESCRIBE problems. Here there are a lot of people for whom English is not the native language and there are objective difficulties with comments, expecially for newbies/untrained people (myself included ). If you read the comments in a "critical" manner you should see that some people named the same artifacts with different terms.

@Roberto: What about to publish anymously the scores for those people who don't want to be named ?
I think that the raw scores and the comments (with or without names) should be published. IIRC this was the method adopted by ff123 for the 64 kbps test.

IMHO, Sorenson and AACDemo make different streams - similar to some extent, but different - I can ABX these two on many impulse clips.

It might seem that encoder used in Sorenson has TNS artifacts (especially audible on clips like Fatboy) that are not present in similar products sharing the FhG's code (FhG's professional AAC, Dolby's professional AAC, etc..)

Also, what is more important - some versions of the FhG's AAC codec use low-pass which is approx. 0.5 kHz lower - making sound a little bit dull at 128 kbps. Sorenson is one of those.

I see a couple of results that confuse me:

1)

http://rarewares.hydrogenaudio.org/test/aa...Z-results03.txt

The rating is 5.0. How is this possible? ABC/HR should not have recorded a value of 5.0, or is there some sort of bug somewhere?


2)

http://rarewares.hydrogenaudio.org/test/aa...alf_results.txt
http://rarewares.hydrogenaudio.org/test/aa...alf_results.txt

Hopefully, the two separate ratings were combined afterwards by the listener to yield a composite rating.

ff123

BUSTED!!!!



I merged the results together (sum/2). That should produce a fair overall result for that test.

Hrm.. the problem is the way I put it in the readme.



Next test, I'll make the publish thing differently, saying that comments will be published anyway and if you don't want to be anonymous, say so.

BTW, added the results of JohnV and ff123.

I think the next test should mention something about artifact training (not just training
about how to use ABC/HR). It looks like artifact training helped at least one person.

Experienced listeners should be able to skip over the training. It probably wasn't a big
issue for this test, because it looks like most people who participated probably had some
experience with listening for artifacts.

ff123

Man, Gecko tossed all the codecs into the rubbish bin! I wish I had spread my scores out, to better differentiate the scores and make my results a bit more statistically significant.

I'm glad that some comments were pretty extensive. I'll go back to the samples and listen for the mentioned artifacts. IMO, after ff123's "artifact basic training", listening for artifacts in the 64kbps and 128kbps tests is the obvious next step... like real-world artifact combat, I guess.

It's possible to come up with a crude "listener sensitivity" index, just by adding up the total scores on all samples. In general, the lower a listener scored, the more sensitive he must be to artifacting. Of course, people don't necessarily use the same part of the rating scale -- that's why I said the index is crude.

I've noticed that I'm pretty much "middle of the road" as far as hearing artifacts is concerned -- I hear some things, but I know I don't hear as much as others do.

For small listening tests, it would be useful to call on those people who are especially sensitive.

Here's several more discrepant results:

http://rarewares.hydrogenaudio.org/test/aa...Z-results02.txt
http://rarewares.hydrogenaudio.org/test/aa...Z-results04.txt
http://rarewares.hydrogenaudio.org/test/aa...Z-results07.txt
http://rarewares.hydrogenaudio.org/test/aa...Z-results08.txt
http://rarewares.hydrogenaudio.org/test/aa...Z-results09.txt

Obviously, the results files should not be edited after they have been saved, which is what it looks like happened here.

ff123

Edit: Oops. Looks like the test proctor did the editing.

Ah. Duh. That one was me. So that your process program would add all the 5.0 results to the list and not output errors to the listing file.

I will revert the mess uploading the original results files (empty) that he sent me.

Ah, that explains everything. Thanks. The process program can be easily modified to not output warnings for results files with all 5.0's

Here are my own results :
http://membres.lycos.fr/guruboolez/AUDIO/a...128/tableau.txt



I'm totally agree with you. Exemple : beautylept.
Proxima, JohnV and Gecko are severe (as me), and others people seem to appreciate a lot the results of the different encoder on this harpsichord sample.
We can't consider garf as an inexperienced listener, but his rating was really high (3.8 - 4.3 & others > 4.5....). For comparison, Gecko is the most disapointed (average : ~2.0 / max = 2.5)

Gecko
Garf

There was another case in which I'd considered all codecs unlistenable (2.0 highest score), but I saw other people giving high grades. All a matter of preference.

Lifeshatters I suppose. I have at contrairy some difficulties to ABX similar samples : they are deafening and boring me at the same time. I can understand, now, some reaction with harpsichord music

I didn't know I was such a "rigorous" judge. I find it difficult to give consistent ratings. It's all very subjective and possibly different from sample to sample. If I'm grudgy, I might give lower scores etc. (Allthough I don't remember being in any extreme mood when I did these tests). I try hard to use the verbal scale for rating. Notice that 3.0 is still only slightly annoying. I quickly skimmed my results and found that I did rate LifeShatters_2.wav a solid 4.5. That's better than "perceptible but not annoying"! Maybe more emphasis should be put on the verbal scale rather than the pure numbers. On a scale from 1 to 5 a three is only mediocre, while the verbal description is better than that imo. Maybe that confuses people.

I didn't give the encoders a "128k bonus". What I want from a codec/format is transparency. I guess (at 128k) that is also part of the design goal and for some listeners it has been met on some samples, so I'm asking much, but not too much, imo. Also, I don't have any other reference than the original, so what else should I rate against? I would have given higher scores if the question would have been: considering the bitrate, how do you think these encoders perform? I do admit that "annoying" is also a very subjective rating and personally, I'm annoyed pretty easily by artifacts. Basically, if it doesn't sound like the original, then it's not acceptable for me (allthough some things like a slight lowpass can be OK).

Two more things to consider: My equipment is mediocre at best. (I recently upgraded to an Aureon Sky though. ). I'm pretty familiar with the test samples from previous testing.