They are the same encoder. The difference is that in QuickTime you have the option to encode in three quality modes: Good, Better and Best. iTunes is hard-coded at Better. Best supposedly is the same as Better in 16bit streams, and only shows any improvement in 24bit streams.

I doubt LAME at 128kbps would surpass AAC, it was the lowest in the previous 128kbps multi-format test (besides Blade). What has changed at 128kbps since then that woukld put it over AAC now? LAME is a good choice as anchor.

In the multi-format is was worse than QuickTime. But I can't predict how it will behave against Compaact or Faac.

Besides, as I recently posted, Lame has been featured in too many tests. Enough is enough, IMO.

So, if Roberto doesn't want to include LAME here then how can I know whether LAME is better or worse than FAAC?

IMHO there is no point testing LAME against AAC winner again since it already lost badly in such test and latest AAC encoders can only be better.

@rjamorim:
Maybe you should start a poll to decide what will the anchor be, reminding users that LAME as already been tested in the previous listening tests and will be tested again in the multiformat listening test, after this one.

I would like to see these in the test:

• Nero
• Apple
• Real
• FAAC
• Compaact

For the anchor, i vote for a MPEG-4 codec. Maybe an early version of FAAC or Psytel.

I re-read some of the posts, and I have changed my mind. The next multi-format will pit LAME against AAC, so let's keep this one an all AAC test, including anchor.



I would wait till after this test, as once we see the results, it might not even matter any more to you how it compares to LAME. These tests have definately had interesting results. Yours might win


Edit: BTW, knik, was that entire quote yours, or were you answering someone elses question? It seems as if you are FOR LAME in the first sentence, but then you say there is no need for it in the second sentence...

I'm sure the results will be very interesting with or without LAME but including LAME would make it even more interesting.

Gotcha, you want to know how LAME does in comparison to the OTHER AAC codecs, not just the winner. I can understand that.

I second that! The AAC winner might not be useable for all situations, so it's quite interesting to compare every AAC competitor with the best MP3 solution available.

Besides, what's wrong with using Lame in many tests? It allows (to an admittedly small amount) inter-test comparability (Another meaning of the word anchor?), while FhG l3enc 1.0 will only make the used rating range smaller.

The whole point is that including LAME here is at least as useful as including it in the multiformat test.

Agreed. I really want to know how the latest version of LAME stacks up against most AAC encoders, not just the winner.

Using L3enc as anchor would be "funny", but would it be of any use? i don't think so, but having LAME as anchor would be of use for many people. This way they can see how all AAC encoders, not just the winner (currently Quicktime/iTunes) performs against the best MP3 solution.

IMO, this is a test to determine the best AAC codec, not compare MP3 and AAC. if you really want to know how LAME (a free mp3 encoder) fairs against FAAC (a free AAC encoder) run asimple test. with 2 encoders it is much easier than with 6 and LAME included.

if early FAAC is out of the question than the early MP3 encoder sounds best. it will act as a good anchor.

in truth, if we REALLY wanted to be able to compare every codec in relation to each other we would have to run a test including every codec at once. that is far too many to do in a single test (time and patience being the limiting factors).

stick with AAC encoders and leave LAME out. you never really know how the test will turn out anyway.

I also like the idea to use l3enc as an anchor, but may be it will be better to use it's latest version (2.71 or whatever was the last) ?
To compare to the first mp3 codec in it's best, not the buggy first version ?

Well, the anchor rating isn't supposed to be useful to users. It's supposed to be bad (Lame doesn't fit that criteria) and put things into perspective. That's why formal tests would rather use lowpass as anchor

Let's be realistic: If Lame is included, it won't be an anchor. It will just be another competitor. An MP3 competitor in an AAC test. You probably notice the mess here...

What people seem to forget is that the bottom anchor is supposed to fail! It's not there to compare against the actual competitors.

Anyway, that's my point of view. If the majority of users want, I'll feature lame. But I'll be sincere: I don't like the idea of adding a MP3 competitor to an AAC test.

Regards;

Roberto.

roberto, i respect your idea, but i don't share it ... i mean, and even you said it i think, the test are for the community, as such, for me at least, and a poll could be well be used, it is very interesting to know if the "actual champ", is still up to the task, compared to the "title challenger" ... for me including LAME would be very nice for the test ... it would mean something like "should i bother to re-rip into AAC, or can i still keep my LAME files"? ... lets do a poll

PS: i hoppe we can still go next week to picaso's display at san pablo

EDIT: this way, it would also help to confirm, or refute the idea of "AAC is way better than MP3".

Including Lame simply to have a comparison Lame/FAAC is not very convincing to me. There are lots of comparisons you have to leave out in every listening test (why not include Xing to see how it fares against compaact?), and I can't see why Lame/FAAC should be so interesting that it would merit an inclusion of Lame in this test (but I'm open for arguments).


Inlcuding Lame for inter-test comparability could be interesting but unless results are really clear-cut, you couldn't draw many safe conclusions. For example, if Lame finishes last in the AAC test you could claim with some amount of certainty that all the AAC encoders are better than all the mp3 encoders from the mp3 test. But if, say, Lame finishes last in both the AAC test and the multiformat test (excluding anchors, the latter is to be expected), you will only provoke senseless claims like "Vorbis finished 1.2 points ahead of Lame in the multiformat test, but FAAC only managed an advantage of 0.9 in the AAC test, so Vorbis must be better than FAAC".


That's what the upcoming multiformat test is for.

I'm still against the inclusion of Lame. In my opinion, the disadvantage of increasing the difficulty level of the test outweighs any advantages. You have to be realistic about this. There were on average 16 participants in the mp3 test, and I'm convinced the number of participants depends very much on the difficulty, so if we add another serious contender to this test, we can probably expect even fewer participants (unless Roberto includes a bunch of problem samples). So, even if the results get a bit less interesting without Lame, in my opinion significance beats interestingness any day of the week.

The first multiformat test already proved that. The best AAC implementation (QuickTime) was quite better than the best MP3 implementation (Lame)

I second the idea of using Xing (ver 1.5) as an anchor. It seems to be one of the most popular mp3 encoders used by consumers out there. It's kinda outdated, not tuned since 1999, doesn't use block switching, etc. So it's not going to compete with any of the AAC coders, yet represents the consumer standard

right ... i didn't remembered that test ...

Well, I see there is too much discussion about what codecs to support and what anchor to use. I think the best and most fair way to settle the issues is creating a poll.

A poll to select the anchor would be easy, but codec will prove complicated since there's no way to set IPB to allow users to choose 5 options.

So, I will use codec packs as options. Something like this:

1. - iTunes, Nero, Faac, Compaact and Winamp
2. - iTunes, Nero, Faac, Real and NCTU
...

Since there would be several different possibilities involved, I would like to see if everyone agrees on a list of codecs that MUST be present.

1 - iTunes: Because it was the winner of the first AAC test and it's hugely popular
2 - Nero: Because it's Nero (heh) and it's popular.
3 - Faac: Because it's the only open source and really multiplatform alternative, and because I believe it's quite popular as well.

From there, I would create different packs featuring Compaact and/or Winamp and/or NCTU and/or Real.

As an added bonus, I will get rid of accusations of favouring this or that codec.

Do you agree?

Regards;

Roberto.

Why not use Fhg AAC encoder (Sorenson?) for low anchor? Would be interesting to see progress of recent aac development.

I think Compaact is essential as right now it is the only VBR-able competition to Nero. I know it has undergone a number of quality improvements as well, and it would be interesting to see how it stacks up.

- Matt

only question is ... haw -APS would compare agains AAC?

Because it might be better than other codecs.



Ehm... FAAC?





Please, don't tell me you are suggesting I test different bitrates...

Probably the choice of anchor could be a separate poll in itself. But to reiterate a couple of points made by others in this thread, the anchor

1) should be significantly worse than any of the other codecs.

2) should sound bad enough to be readily identifiable. Yes, that will compress the ratings, but it will take a lot of the burden off the listener, who now need concentrate on one less codec. Also, it may encourage more people to participate, who might otherwise be scared off. I think lame and audioactive are too good, but the other mp3 codecs might work.

l3enc might work, but I would probably encode the samples with it to listen to how bad it is. It might be too good as well. In fact, it's probably a good idea to listen to the encodes of any prospective anchor.

Small listening pre-tests are a good idea, in my opinion.

ff123

Yes, absolutely.
I think this would be the best way to settle this issue.

But I think that everybody agrees on the 3 codecs you mention, so why don't you have a list with only the remaining codecs. Let every one vote and select the three with the most votes.

Two, actually
(The anchor would be handled in another poll)

I think that might work too, although that would limit votes to one per person, and the ideal would be two. Ideas, comments?

Hmm?
Two separate polls?
Monday through wednesday, choice of first contender.
Separate poll, thursday through saturday, remaining candidates minus first poll winner.

It's the only thing that comes to my mind right now.

that's a good idea, having a separate poll for the anchor i think it's the best choice right now

after reading this thread i've been convinced that LAME would not be a good anchor, now my vote would go for iTunes since it had the lowest score in the last test

go for the polls separated by time.

please just pick an anchor. it doesn't really matter as long as it sucks. it isn't in there to compare to the other codecs, its there to lose, and keep the other ratings up.

My vote is to use the old FAAC as the anchor. After all, this test is about AAC.

Yes, I will do that.

Now, choosing the codecs:
http://www.hydrogenaudio.org/show.php/showtopic/18523

Some time next week, choosing the anchor.

Please vote.

Regards;

Roberto.

What about the Homeboy AAC encoder as an anchor? Wasn't it really bad?

Regarding Nero AAC - Fast mode shouldn't be used, as it is still not tuned.

I like the idea.
Using a stable high quality encoder as an anchor could indeed allow some inter-test comparison and IMO such kind of anchor would be much more valuable than any low quality old encoder. An ancient encoder is just a waste of listening slot.



How can you know LAME won't lose? IMHO LAME at a last place is a realistic scenario.
I really think comparing the best MP3 to the worst AAC encoder would be fascinating and I can't see any mess here.

EDIT: Winamp problem is a decoder-only problem. Nothing seems wrong with the encoder.

EDIT: Hmm, actually I can't be sure if this is an encoder problem too. I guess some testing should be done whether a Winamp file decoded with Winamp sounds better then with another decoder. The difference between 2 decoded samples (winamp decoder versus another) is quite difficult to hear, but in a test like this it will probably be noticed.

Besides this, I think NCTU-AAC should be added to the test, just to show how much it sucks. As low anchor also the new Adobe Audition filter could be used.

Menno

If everyone knows that NCTU-AAC sucks badly may be it could be used as anchor? Just a thought.

-Eugene

VQF ran at around 80-96kbps. i should know, i was the one that was so happy about it intially (with my 128kbps mp3 vs 96kbps vqf) i spammed the mp3 newsgroups and mp3.d newsgroup about it when i first started playing with it. (i wouldn't even touch it now, altho i think it's in my audio tools collection somewhere X)

can't find it or anything else that's helpful .. seems to have fallen off google

oh, here we go.. here's one at least... http://www.mp3-tech.org/tests/pm/VQF-96k.htm

A question comes to mind: Is NCTU AAC bad enough to be used as low anchor?

Well, such claim is better than pure guess when you can't compare faac to vorbis directly.

@ Menno: Thanks for bringing this issue to our attention.

Discussion here:
http://www.hydrogenaudio.org/forums/index....ndpost&p=182562

About anchors: I'm taking note of all your suggestions, and will later create a poll so that people can vote on what they prefer. Thank-you.

rjamorim: Are you going to use VBR streaming profile with Nero? I have done some encoding with it, and the average bitrate is too high for this "128kbps" test. VBR Internet profile produces closer to 128kbps on average.

I think that more encoders should be included or the anchor should be replaced with some AAC encoder. As i did the MP3 test, the anchor was useless to me, and this time it make out quite well against some other encoders.

Seconded. Same here.

I personally would prefer that fewer codecs be tested rather than more. If an anchor is used it should sound bad, IMO. Otherwise, I think it's better to just drop it. Especially on a test in which all the codecs should approach transparency. 6 codecs was almost too much for me in the mp3 test.

ff123

Evil Psytel as anchor.

OK, now a fucker decided to mess my poll.

My patience is growing too thin, so I'll tell you what I plan to do:

-Compaact and Real are in
-Winamp is out because it seems to be broken, so it would be unfair to test it now.
-NCTU is in as an anchor.
-The test will be encrypted and will require Java ABC-HR.

Flames? Opinions? Comments?

BTW: Everything is discusseable, except encryption. It is definitely going to happen.

Regards;

Roberto.

It may sound a bit unfair to newer members, but to remove any doubts about manipulating the result in favour of compaact!, I suggest to only take hydrogenaudio members into account, that were members *before* the release of compaact!, i.e. end of october 03.

If the participation at the listening test would be without restriction, and compaact! would rate good, nobody would believe it now anyway.

Alexander

*deleted*

How strong this encryption is? What crypto system it uses?

Ph34r my m4d scr33ning skillz!