In the 48 kbps test I spent a lot more time with bibilolo than with any other sample. The bibilolo files were obviously very bad, so hearing various problems was not a problem. The problem was to rate the contenders fairly. On each listening time different problems seemed to be more annoying.

After spending an hour or so with bibilolo I ended up with this judgement:


(I added the encoder names.)

However, I am not sure if I could repeatably end up with the same results.


Aquatisme was a killer sample too. There was no doubt about the huge amount of problems. In addition it was even more difficult to rate. I really couldn't put the contenders in any order. After trying for a little while I decided to pass the sample and move on.

3 of 18 are too many IMO. Your last 128kbps test didn't have such a sample at all.

ff123 explains listener psychology.

I think this applies to overall samples. The preference of music highly depends on personal taste. But I don't think people feel like listening those samples time after time. I believe test itself has to motivate a listener since this is a collective test.

(spmg54_1 is a boring sample too. But spoken voice sample is interesting at this bitrate. 48 or 64kbps would be a good compromise between quality and bitrate for long speech encoding.)

So you are proposing to exclude Aquatisme and Bibilolo because they are:
A - too hard to encode
B - do not suit your musical taste

Regarding A, 64kbps has 30% more bits than 48kbps, which should improve results. But you got an interesting point regarding potential difficulty to rank contenders.

Regarding B, at least there won't be complains about bias toward a specific musical genre
;-)

My little musings on the sample-selection issue: I think the best ones to remove would be some of the ones that are of a similar style & therefore not adding anything major to the process e.g. if there are three (or maybe even two) samples of guitar-based pop, get rid of one of them. I'd tend to favour keeping a wide range of sample styles to reflect the wide range of music that the HA community (or even the entire listening public) doubtless listens to. I'd also tend to favour keeping 'difficult-to-encode' samples for research/intellectual purposes, but these samples are perhaps less important if very few people actually listen to that style of music recreationally - after all, that's the end purpose of music encoding, isn't it?
As regards the difficult-ranking issue raised by Alex B, I don't think one should get too hung up about it - if a given set of encodings sound about the same, then they should be ranked about the same. (Conversely, if a sample sounds better to you, rate it higher!) Each person is only one 'data point' as it were, and the end result will be a statistical one based on many people's responses - this is the point of having 'public' or 'community-based' listening tests i.e. to get multiple responses & thereby increase statistical validity, but also to see how things sound to the folks who are going to use these things. I'm not saying that we shouldn't spend a little time trying to rank contenders, just that we shouldn't stress out about it.
Of course, this is all just "my 2 cents".
Regards,
rc
Edit: I feel a bit hypocritical reading this now, 'cos I vividly remember spending inordinate amounts of time trying to rank contenders in the last listening test I participated in! Oh, well...

Besides being generally very different from the reference all aquatisme and bibilolo samples were also clearly different from each other, like each sample was encoded from a different version of the source file. It was difficult to put the various obvious artifacts in an annoyance order because these samples were not any kind of familiar music.

When is the test starting? Not so much discussion anymore, it seems.

I received the new encoder today, but I am currently busy searching for a new apartment to finally move out. If everything works out well, I have time next week.

I think I will remove Locomotive Breath since it's way too long and also not very special. At 48 kbps, the modern contenders were tied and received a pretty good rating.

What else to exclude?

OK, no feedback - how lovely. Locomotive Breath and Debusy are the samples that I am going to remove. Debusy also got a very good rating at 48 kbps, so I expect it to be (near)transparent at 64 kbps. Any new samples you guys want to propose? I would like to prepare the test this week.

So after all, the same samples as in previous 48 kbps test, will be used?
I think it would be good to cut a bit from beginning and end of Paganini_Allegro_spirituoso, because sample is long and there are lot of repetitions. Also WhiteAmerica can be shorter.

OK, I'll see what I can do. As for the overall samples selection - I have no problem including new ones, but nobody suggested anything yet.

Is an 8 second sample long enough?

/Kef

Sure.

Try this one: http://www.hydrogenaudio.org/forums/index....showtopic=56083

I just send in our final encoder for the test. Still fixed at one q value, because we still need to do some tuning on some bitrates. Release will be pretty soon though.

Maybe I can already tell some of the changes that have been made for this new release:
* Linux version
* New q value mappings (now better matches http://www.hydrogenaudio.org/forums/index....howtopic=44310)
* Retuning of almost all bitrates
* Minor bugfixes in the psychoacoustic model
* Fix in PS encoder/decoder delay
* Fixed incorrect streamlength written in MP4 files
* Speed ups

I received the new encoder menno, thanks.

Also, thanks for the sample suggestion. Would you like to propose what sample to replace with the one you posted?

Do we have reggae samples yet?
I propose this one:
http://www.hydrogenaudio.org/forums/index....showtopic=56091

Last chance to post samples folks - I want to have the discussion over by tomorrow evening!

How is the preparation for the test going?

The test will most likely start this weekend. Had several appointments this week regarding my new apartment that's why it took a bit longer.

Crap, spent the whole Saturday at Mann Mobilia looking for furniture - thought it would take only a few hours.

I also haven't seen Roberto online for several days and he's the person who is usually hosting the samples and taking care of the torrents.

Anyways, what I wanted to ask - is Java 1.5 now finally available for all major operating systems like Windows, Linux and Mac OS?

Edit: BTW, I removed aquatisme but kept bibilolo. Hope nobody is mad at me because of this...

OK, regarding the Java thingy, it seems that all users can download and run the latest ABC/HR. If you're on a system which does not support JRE 1.5, please use a Linux live CD like Ubuntu, SuSE, Fedora, Knoppix, whatever.

Roberto, can you host my samples again, pleeeeeze? *makes pretty eyes*

I have to get the test up and running since I sold my old WD 250 GB on eBay and the 500 GB Seagate that I ordered is no longer in stock. Will have to use Ubuntu live CD myself the next few days... Yeah, I was dumb, I know...

OK, here's the bitrate table



Anything critical?

Edit: Added average bitrate at the bottom of the table. High anchor is within 10% tolerance (105.6 kbps), so is Vorbis (70.4 kbps). Not worth mentioning Nero which hits exactly 64 kbps as well as low anchor and WMA which both use CBR coding.

Edit 2: Encoder settings:

Vorbis: -q 0
WMA Pro: Codec: WMA9PRO (it's actually 10, but the VBS file I use is outdated) / Mode: 0 (1-pass CBR) / Setting: 64_44_2_16
Nero: -q 0.24 but hardcoded anyways

High Anchor: iTunes LC-AAC 96 kbps VBR
Low Anchor: iTunes LC-AAC 48 kbps CBR

OK, everything is set. Roberto is hosting the samples (thanks mate!) and I've got everything ready to go. I am waiting for Menno to send me the whole Nero Digital Audio package with tagger, decoder and encoder tomorrow morning. This package will be hosted on the listening test site or on some Nero server, I don't know yet.

So, the test will start tomorrow. Thanks to everyone who participated in the pre-test discussion.

Done Sebastian will host it somewhere for now. Sometime soon it should also be available on the Nero website.

From the listening-test page:



They took WMA 9. The test ("link below") is from 2005. Shouldn't you write: "It is also interesting to see if Microsoft's claims regarding the superiority of WMA Professional 9 over HE-AAC (see link below) are still valid for the most recent codecs."

Or did I miss something?

Ups, wrong thread

Have any proof?

Edit: BTW, maybe an admin / mod can move elmar3rd's posts to the other thread and close this one since the pre-test discussion is over.

On page 3 of the NSTL-Report you can download here they wrote " To encode the original clips into WMA Pro format, Microsoft Windows Media Encoder 9 Series Version 9.00.00.2980 was used".

Yeah, because there is no such thing as Windows Media Encoder 10. The Windows Media Encoder 9 makes use of whatever Windows Media codec is installed and in NSTL's case, 10 Professional. The VBS file I use for encoding also required me to use the "WMA9PRO" codec, but in fact, it is WMA Professional 10.

Thank you for clarification!

I'm interested in the bibilolo sample (the lossless original).
Searched for it but couldn't find it.

Can somebody point me to a link please?

It should be in the sample3.zip package on Gabriel's 48 kbps AAC test samples page:
http://www.mp3-tech.org/tests/aac_48/samples/

In general it would be good to keep all test material on-line some time after a test is finished (Like Gabriel has done). This would be useful for anyone who would like to try the test personally, even though missed the public test. Also, the codec developers may need the samples for reproducing the discovered problems.

Thanks a lot.