Public Multiformat Listening Test @ ~64 kbps [March/April 2011], Nero & Apple HE-AAC, aoTuV Vorbis, CELT Options

[Alex B]

The test is extended until the 10th of April.
We still need more results for some particular samples.

Thanks, I was just about to request for an extension.

I noticed this thread only 5 days ago (didn't expect a new test) and did 5 samples on last Sunday. This week has been busy for me and I may not have any spare time and energy left for testing more samples before the next weekend.

I will post the numbers of those samples in a few minutes.

I was also going ask for a list of the samples that do not have enough results yet.


Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

For some reason no admin or mod has directly announced the news, even though we requested that earlier. The "news submission" route is quite unnecessary for a HA listening test. Who ever organized the hosting of the samples (was it Garf?) could have posted the news in the very beginning.

EDIT

If (hopefully when) the news post is validated and published on the front page this thread could be linked as the comments thread and moved to the Validated News forum as I requested earlier: http://www.hydrogenaudio.org/forums/index....st&p=749581

This post has been edited by Alex B: Mar 30 2011, 20:29

[IgorC]

Thanks, I was just about to request for an extension.

Don't be angry, but from Day 0 it was clear that test will be extended. Previously such test had to be extended twice http://listening-tests.hydrogenaudio.org/sebastian/mf-64-1/
There are a lot of people who still have not finished to submit their results and they are important.

I was also going ask for a list of the samples that do not have enough results yet.

We need more results for samples 11 and 14-30 (14,15....29,30) and results for other samples are welcome as well.

Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

Done

This post has been edited by IgorC: Mar 30 2011, 20:44

[Garf]

Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

Done

This was completely useless for the news page so I just cannibalized it.

For some reason no admin or mod has directly announced the news, even though we requested that earlier. The "news submission" route is quite unnecessary for a HA listening test.

No admin or mod approved anything because nothing suitable was posted to the news submissions forum.

[Alex B]

Thanks.

I think IgorC did a small mistake when he did not post the first post of this thread to the "News Submissions" like Sebastian used to do. (For example: http://www.hydrogenaudio.org/forums/index....howtopic=56397).

This post has been edited by Alex B: Mar 30 2011, 21:02

[Garf]

Thanks.

I think IgorC did a small mistake when he did not post the first post of this thread to the "News Submissions" like Sebastian used to do. (For example:
http://www.hydrogenaudio.org/forums/index....howtopic=56397).

I would have been rejected anyway (or just sat there) because the format of the post was totally unsuitable.

Edit: I've edited it into submission, but I had to move part of IgorC's comment into his next post. Merged both threads. Should be fine now.

[IgorC]

Thank you, Garf.
Sorry for the mess.

[Alex B]

Of course you could have followed the rules at http://www.hydrogenaudio.org/forums/index....showtopic=20857.

I.e. posted only the bare minimum, essential "news" info in the first post and...

... All other additional information like screenshots, full changelogs, personal opinions/comments may be added in a 2nd post, provided that they are really important.

However, this a collaborative effort. IgorC is not alone. Others who are more experienced can and should help (as Garf did now).

I just did a quick calculation of the work effort that is requested from the testers. Assuming that we would get 20 results for each sample and each codec would need 2-3 minutes of testing time, making 10-15 minutes for each sample, 30 samples would need 30 x 20 x 12.5 minutes = 125 hours of intensive testing work. On average each tester would spend 6 hours and 15 minutes for testing all 30 samples. Actually, I don't know if 2-3 minutes per codec is even a realistic estimate. In my experience at least some samples/codecs that are difficult to test need more time.

This post has been edited by Alex B: Mar 30 2011, 22:55

[Dakeryas]

I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?

[Garf]

I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?

Check the settings to see which output audio device is selected.

[Canar]

The test is extended until the 10th of April.
We still need more results for some particular samples.
I will post the numbers of those samples in a few minutes.

Awesome. This should give me a window of a day or two to tear through some samples. Thanks for the extension.

[mixminus1]

I'm in the process of plowing through all 30, and should be done by tomorrow afternoon.

HTH!

[Dakeryas]

I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?

Check the settings to see which output audio device is selected.

Solved because the microphone was indeed the default output, many thanks !

I'll test a little as from now on!

[IgorC]

Thank you to everyone for their results.
All participants have received the answer.
If someone haven't receive it then post here and also can send me PM or email.

Some participants initially had problems with ABC-HR Java because previously their ABX tool was other (like foobar ABX, ff123's)
I will ask if someone already has a few results send them as soon as possible to avoid the loss of their time. This way there is time to analyse them and report to listener if everything is good.

More results are needed for samples 9,11,13,14 and 16-30 (16,17,...,29,30). Especially for 9,11,27,29.
Though results for other samples are welcome as well if the listener has enough time.

[IgorC]

The test will end on 10th of April (at 11:00 of GMT -3:00). Definitely.
Finally there will be the results.

[IgorC]

Reminder.
The test will end in less than 12 hours.

[enry2k]

One of the samples is the Susan Vega's Tom's diner song, the track originally used by the mpeg researchers to tune up the MP3 format:-)

[jmvalin]

One of the samples is the Susan Vega's Tom's diner song, the track originally used by the mpeg researchers to tune up the MP3 format:-)

I think that applies to *all* audio codecs, really.

[IgorC]

The test is finished.

[sauvage78]

IgorC:
Whatever the result, I am grateful that you helped jmvalin with CELT when so far I miserably failed to find the motivation to help him.

This post has been edited by sauvage78: Apr 10 2011, 17:59

[IgorC]

Thank you
Sure, no problem

The main purpose of this test was directed not only for us (users) who are interested to know what codec to use in some particular scenarios but also to help to developers of AAC, CELT and Vorbis to improve their encoders.


All results have been already processed.
Though the approval of other organizator is still required.

This post has been edited by IgorC: Apr 10 2011, 18:33

[mixminus1]

IgorC:
Whatever the result, I am grateful that you helped jmvalin with CELT

So am I!

All I'll say about CELT right now is: "Impressive. Most impressive."

[Garf]

I just confirmed IgorC the results look OK and sent some additional analysis.

Thanks to everyone that submitted results. There were over 500 valid ones! Thanks to this, the results of the test are highly significant, and will allow for some clear conclusions. Listening tests like this are not possible without volunteers that put in some of their free time, so really, this is your accomplishment. Also thanks to IgorC for conducting the test and setting up everything.

Now that I've seen the results I must say, they aren't even remotely close to what I expected! A lot has changed in 4 years...

So IgorC, hurry up on those webpages to satisfy everybodies curiosity

[IgorC]

thank you, Garf

and thank to all participants
http://www.hydrogenaudio.org/forums/index....showtopic=88023

[IgorC]

I would ask to Admins to close this topic.
The test is end.

Thank you.