Choice of anchor/6th codec for AAC test Options

[dev0]

The modified sample set (with fatboy included) seems hard enough to me.
Maybe there should be another call for (problematic) samples.
But I wouldn't want to change much about the set of samples, since keeping it as unchanged as possible adds the possibility of inter-test comparisons.

[guruboolez]

Including Winamp encoding (and highly popular) solution looks necessary for me.

As difficult sample, the "mandolin.wav" I've uploaded last week should be difficult enough and easy to ABX:
http://www.hydrogenaudio.org/forums/index....howtopic=18360&

If some other people could confirm the possible artifacts...

[rjamorim]

That was before I was exactly sure what the problem was. The Winamp encoder is just as safe to use as any other encoder, I already said it before. The Winamp decoder has a problem.

Well, OK, but I still don't understand why the (few) responses I got for that quick test said the FAAD decoded files sound worse. (Well, Tigre said that one of his tests sounded worse with the WA decoder, but in the other test both were tied!)

Including Winamp encoding (and highly popular) solution looks necessary for me.

I'm pretty sure it would be easy for you to test 6, 7, 8... competitors

[rjamorim]

BTW, there seems to be a misunderstanding here: I didn't say the 6th competitor would be Winamp, or Real, or Compaact...

The competitor list hasn't been decided yet. And I don't know when it will be. I will maybe create another poll, but I don't feel much like it, for obvious reasons.

So, don't vote at 6th because you feel Winamp must be present - Winamp might get featured neverthless. Only vote if you believe the anchor is unnecessary.

This post has been edited by rjamorim: Feb 10 2004, 15:17

[bond]

well as you said the number of the used codecs is limited to 6, in fact using an anchor would mean that winamp or compaact or real would have to be dropped


so if you want to have compaact and winamp and real in the test you have to vote for not using an anchor

[guruboolez]

A nice feature for ABC/HR would be an automatic anchor generation (lowpassing with SSRC, or bad encoding with Blade for exemple). And why not, this anchor should be include not as a challenger (lost or hidden among many blind competitors) but as a negative reference. This implies that listener could call anchor as well as the Reference.

For being more clear:


Present ABC/HR software have this look:

CODE

 R    R    R
A B  A B  A B



R = Reference
A & B : hidden files

I suggest something like that:

CODE

 R    R    R
A B  A B  A B
 a    a    a



R = Reference
A & B : hidden files
a = anchor

R will be listened for reference
a will be listened for tempering the notation of each file. Just as a negative reference.

I don't know if I'm clear, or if what I'm saying is stupid.

[ErikS]

I voted other. Blade or a simple lowpass at 7 kHz. Reason: blade was used before and can then serve the purpose of comparing the results of the two tests. They are connected by this common denominator. Same goes for the lowpass at 7kHz, but then you compare to other tests which are conducted under other conditions.

[music_man_mpc]

I voted other. Blade or a simple lowpass at 7 kHz. Reason: blade was used before and can then serve the purpose of comparing the results of the two tests. They are connected by this common denominator. Same goes for the lowpass at 7kHz, but then you compare to other tests which are conducted under other conditions.

If you liked Blade as an anchor, you'll love l3enc 1.0, its the first software mp3 encoder ever! Maxximum L33T Anchorage!!

[MGuti]

all i can say is that if i hope to be able to partake in this test it better have an easy anchor. no way i can sit through LAME 3.95.1 AND 5 other top-notch AAC encoders which (we already know) are better than LAME. why prove it again? if you really feel the need to see how FAAC does against LAME, do a 2 encode test. thats EASY to do, and takes the stress off of the people doing the tedious test (why test any other independantly against LAME? all the rest cost money and if you feel like spending money, buy the best).

just my opinion i want l3enc 1.0

forget lame, it has had the glory enough and will be in the multiformat test anyway (we know it won't win here besides).

[music_man_mpc]

all i can say is that if i hope to be able to partake in this test it better have an easy anchor. no way i can sit through LAME 3.95.1 AND 5 other top-notch AAC encoders which (we already know) are better than LAME. why prove it again?

I'm totally with you on the need of a real anchor! However I'm not so sure that ALL of the 5 "top-notch AAC encoders" are better than LAME 9.95.1. In fact I seriously doubt that,

[rjamorim]

Is it possible to make some samples available to compare so that we can tell if the anchor will be bad enough?

I made l3enc and Lame encoded samples available, since it seems one of these will win.

Keep in mind this is the exact same sample set as the one used in the MP3 test. I didn't replace samples yet (I'm waiting for comments )

Files encoded with FhG l3enc 1.0 CBR 128kbps
http://rarewares.hydrogenaudio.org/samples/samples_l3enc.zip (4.5Mb)
I'm positively amazed these files are 100% compliant :B

BTW, there's a small clip at the beginning of each sample. It might be related to the fact that my waves are WAVEFORMATEX, which was not yet developed back then :B
I will cut the offending part with mp3directcut before uploading the sample packages if l3enc is chosen.

Files encoded with Lame 3.95 with --preset 128
http://rarewares.hydrogenaudio.org/samples/samples_lame.zip (4.5Mb)

The original samples, in case you want to check them out:
http://rarewares.hydrogenaudio.org/samples/samples.rar (36Mb)

Please post your comments about the encoded samples.

Regards;

Roberto.

This post has been edited by rjamorim: Feb 11 2004, 05:16

[ff123]

Under noisy conditions (kids, tv), I didn't find l3enc too annoying except on mybloodrusts and waiting. But when I compared lame on waiting, I think I preferred l3enc. It's probably a lot better than Blade.

But others may have different opinions.

ff123

[ff123]

Ok, quieter now. Also encoded to xing(old) and faac 1.17. abchr results:

CODE

sample        faac     xingold   lame    l3enc
bigyellow     3.1      1.0       4.3     3.6
dafunk        2.1      3.0       5.0     4.2
enolagay      4.7      2.1       4.7     4.2
experiencia   3.2      4.7       4.7     4.0
gone          4.2      1.8       5.0     3.0
illinois      4.0      3.4       4.3     4.8
mybloodrusts  5.0      1.0       3.7     2.8
newyorkcity   4.5      2.5       4.8     4.0
polonaise     4.7      1.0       4.4     3.9
riteofspring  4.4      1.0       3.7     5.0
scars         5.0      2.1       4.0     4.2
waiting       3.7      2.5       4.0     4.4


Means:

lame     faac     l3enc    xingold  
 4.38     4.05     4.01     2.17  

---------------------------- p-value Matrix ---------------------------

        faac     l3enc    xingold  
lame     0.347    0.291    0.000*  
faac              0.906    0.000*  
l3enc                      0.000*  
-----------------------------------------------------------------------

lame is better than xingold
faac is better than xingold
l3enc is better than xingold

So I'd say that xingold would be too sucky, and based on faac 1.17's performance on the previous aac test and lame's performance on the mutli-format test, any of the three other choices: lame, faac 1.17, or l3enc would be adequate choices as anchor. but since faac and l3enc seem to be marginally worse than lame (for me), I would personally choose either of those two.

ff123

Edit: l3enc appears to narrow the stereo, which is why I downrated it on some samples.

This post has been edited by ff123: Feb 11 2004, 09:04

[knik]

@ff123
Thanks, you did a great job.
l3enc is indeed good enough to be an anchor in this test but I still think lame is better because of other advantages (popularity, inter-test anchorage)

[bond]

now that it seems that the decision has to be made between lame and l3enc rjamorim should maybe start a new poll only featuring those two, as 48 people voted for none of those (which is the majority btw )

[rjamorim]

now that it seems that the decision has to be made between lame and l3enc rjamorim should maybe start a new poll only featuring those two, as 48 people voted for none of those (which is the majority btw )

Hrm, OK... We're running out of time, but I guess there is still time to create another poll.

But, still, people aren't discussing other very important issues: What samples should be replaced, what codec (Winamp, Real or Compaact) will be left out...

@ff123: Thanks a lot for all your help

[bond]

[quote=rjamorim,Feb 11 2004, 03:21 PM] [QUOTE=bond,Feb 11 2004, 07:53 AM] what codec (Winamp, Real or Compaact) will be left out... [/quote]
hm i think the last poll clearly showed that people want winamp (as most voted for winamp and real or winamp and compaact), so you could start another poll only using real and compaact

now thats what i call democracy

[tigre]

My 2 cent:

Does anyone/anything still use l3enc today? If the answer is 'No', using it as anchor would be just a waste. Either use some codec that has been tested in other recent tests (e.g. lame 3.95.1) too, so there's (rough) comparison between tests possible, or just don't use an anchor. Xing hasn't been a real anchor in 128kbps mp3 test - it doesn't seem to be a problem.

I voted for NTCU because I'd like to see their questionable claims being proven (hopefully) wrong, but if l3enc is the hottest competitor for the anchor job, I'd prefer not to have an anchor at all.

[rjamorim]

hm i think the last poll clearly showed that people want winamp (as most voted for winamp and real or winamp and compaact), so you could start another poll only using real and compaact 

Sorry, I won't use that poll in any fashion. People abused it, and I believe later I'll be criticized for using it neverthless.

I could create another poll, but I doubt there will be enough time until the weekend (the point where I will start doing the bitrate deviation tests helped by Spoon) to do two polls. I don't want to run two polls at the same time, to avoid confusion.

Regards;

Roberto.

[JohnV]

But, still, people aren't discussing other very important issues: What samples should be replaced, what codec (Winamp, Real or Compaact) will be left out...

I repeat what I said earlier. Imo at least Real and Compaact should be included, because their developers attend to the community discussions (ok, Karl is not audio developer, but anyway), and this kind of activity should be rewarded somehow.
WinAmp would be interesting also though. That's why my vote still is no anchor and include also WinAmp.

[ff123]

But, still, people aren't discussing other very important issues: What samples should be replaced, what codec (Winamp, Real or Compaact) will be left out...

Looking at the results of the mp3 test, I think EnolaGay is a candidate to be replaced, since it didn't seem to separate the codecs apart from each other all that well.

People have complained about riteofspring, but it does seem to do the job of separating codecs. That has to be traded off against the possibility that nobody will download the file to listen to it

Same story with Polonaise -- supposedly it's an "easy" file, but the codecs strengths and weaknesses were highlighted by this sample. Again, not too many listeners. Possibly more sensitive listeners are choosing Polonaise and riteofspring, causing this effect. But that's just a guess.

I know I've recommended keeping waiting before, and it does play a number on the codecs (it's generally rated low), but it doesn't separate them very well.

ff123

[bond]

imho winamp should be used in any way cause it surely has the potential to get one of the most widely used aac encoders

hm i think the last poll clearly showed that people want winamp (as most voted for winamp and real or winamp and compaact), so you could start another poll only using real and compaact 

Sorry, I won't use that poll in any fashion. People abused it, and I believe later I'll be criticized for using it neverthless.

hm one guy abused it to cheat for compaact, that doesnt mean that the whole poll cant be used as a guideline

well whatever, i will simply let surprise myself which codecs will be used

[guest0101]

Definitely keep Real AND COmpaact please. Since their developers are active here we should give them a shot. WinAmp would be nice, but not at the expense of Real and Compaact!

[rjamorim]

Since their developers are active here we should give them a shot.

I agree with that mentality.

(Karl, please get Ken to post here some time )

OK, third and (hopefully) last poll created.
http://www.hydrogenaudio.org/forums/index....=ST&f=3&t=18610

Winamp vs. l3enc vs. Compaact.

Please vote and continue discussion there.

Regards;

Me.