I forgot about needing to manually select sound source; that was the first workaround I instituted. I haven't experienced clicks, but the audio doesn't always start/stop in exactly the same place. The latter is known; I saw it mentioned somewhere.

Speaking of bugs, your handle breaks quoting.

Well, speaking of bugs, creating the ECF files was a pain since I had to close and restart ABC/HR every time because otherwise, if I opened the Create New Test window again, it would have only half of the normal size.

I can confirm this (JRE Version 1.6.0 (Build 1.6.0_02-b05 / Win XP Pro SP2). But I can't confirm any other problems with ABC-HR neither in this test nor in the previous three or four tests.

Guru plays hard too. I can't imagine ABXing all files. In fact, I didn't ABX any of them. If I think I might not be hearing a true difference I just give it the benefit of the doubt and don't pull the slider down.

ff123

I have noticed this too. In my case it is enough to close the cropped "New ABC/HR Test..." window and reopen it. On the second time I get the complete window. I don't need to restart the program.

(XP SP2, jre1.6.0_02)

Maybe a mod can split this and create a new thread called "ABC/HR Bitching" or something.

Really?! I will have to try your solution. I was up to now always restarting application like Mares.

BTW, you can take your time on Wednesday because the test will end around 20:00 since I am in the new apartment fixing things and talking to various mechanics (or what is the correct translation of the German "Handwerker"?).

Handyman?

I'm looking forward to seeing the results of this test

craftsman / craftswoman

Samples 1 to 9 are OK. I did 50% of the job and I have 44 hours to finish the 9 others.


At this bitrate I've no problem to ABX these encodings. I can ABX most of them in less than a minute (some of them were probably done in ~20 seconds - I will see it on log files in two days). It's not very stressing and I can easily afford this for all samples. It would be very different at 128 kbps.
I only failed for one sample (bibilolo, the 7th IIRC).

__
I noticed that for most samples I already tested two are sharing the same kind of artefact: a grainy texture I only heard in the past with SBR (HE-AAC & MP3Pro). So I guess that the second one is WMAPro and that it uses a technology which seems very close to Coding Technologies' one.

Sample7 was the easiest one for me. I could even ABX (8/8) high anchor, which on most other samples I could not (at least I think it was the high anchor that I can not ABX).

Guess this means even you wouldn't easily hear problems in normal listening situations outside of abxing. Not talking about most of us with our common ears.
Sounds very good for these codecs at such a low bitrate.

I was maybe not very clear. I can ABX (8/8...12/12) most samples in a few seconds. It's only possible if obvious artefacts are audible (i.e. when quality sounds usually poor compared to the reference). For me 64 kbps are still unpleasant even on "normal" listening conditions ("normal" has of course to be defined... as well as "unpleasant": the ears adapt themselves very fast to poor listening conditions such as AM radio, etc...).
I would say (from my own experience) that a huge quality gap exists between this quality and the one obtained by the same competitors at 128 kbps. But without a solid experience in artefacts hunting I'm pretty sure that this quality would be amazing to my ears.

It is great that you are helping us with this test Big thanks from my side!
Also have in mind that results from this test will also help in improving Nero AAC at high bitrates.

Sebastian has now one new result for each sample. All must be valid (successful ABX or 5.0). I think wasn't particularly soft with the competitors, especially on the second half of the test. Now I'm waiting to see the final results and to discover mine.

I concur. It's not that the other planned tests would be invaluable (because they certainly will be), but it seems that many are very interested in a multi-format test at 96 kbps, myself included. I also think an aoTuV B5 Vorbis-only test at various bit rates, between Q1 and Q4, with non-music samples (perhaps both effects and voice) would be interesting, seeing as how many game developers are using Vorbis as a staple for audio assets, and the results of such a test may be particularly valuable for them.

With all due respect, these tests are made for users who want to listen to music either at home (>= 128 kbps) or outside the house (< 128 kbps). If game developers want to encode their car engine sounds or whatever to Vorbis, they should conduct their own tests - I doubt EA or any other company is going to come and pay me for conducting tests or pay you folks for investing your time in testing.

Sorry to ask something which may be highly obvious, but where are the sources of the ABC/HR program in Java?

I see that there's a directory with licenses and it contains some mix of BSD-like (both 3 and 4 clause) and LGPL 2.1. I would like to see the sources at least of the LGPL program and, quite possibly, package it for Debian (and hence, Ubuntu), so that people can use things like:

aptitude install wavpack abchr vorbis-tools faad

and use the corresponding programs to decode the files (with all the dependencies already pulled in).

This would simplify the process of distributing the binaries for at least one platform. I can also create a meta-package that pulls other pieces (or I can include scripts as examples) that helps in listening tests conducted by Hydrogen audio, with all the packing and unpacking going on under a Free platform.

BTW, such script in that package would help with users on MacOS X and even those on *BSD, as I always try to write programs in a a POSIX compliant shell like ash.

In summary, I think that we could indeed make this easier for at least a portion of the potential test listeners and I volunteer to make and maintain such a package in Debian proper so that it would be there even if I get hit by a bus.


Regards, Rogério.

Sources are hosted on rarewares: http://www.rarewares.org/others.php

I've done 17 samples of 18. Only White America sample sounds strange and confuses me. I hear difference at the first try but then can't abxed with 5 tries. Original already sound artificially however it's lossless.

Thank you very much. I see that the other .jar files are already in Debian (which probably mean that they are already in Ubuntu Linux) and now, I only have to modify the build process (with ant) to get a built package.

Nice.


Thanks, Rogério.

YES. Me too!!!

+1

It was the only sample where I could distinguish only low anchor.

Are you people inpatient to see the results?

Test is not closed, yet! But in case anyone is still testing, please hurry up - only a few hours left (can't say for sure - having some private work to do here before I can start evaluating the results).

Very much so, yes.

Closing the test in 25 minutes.

my friends do the test! please 30 minutes!!!

OK Sebastian! He does the test and send to you to mail listening-tests dot info.

We wait the results!!!

Woah, somebody's excited!

Though I don't blame you; I am too!

Huh? So, your friends are still doing the test or what?

OK, the test is over! No results will be accepted any longer. A total of 43 people submitted results.

OK, no problem decrypting the results. I know that one user who also received a mail from me a few days ago had a problem with three results actually pointing to the same test session (= set of samples). This is something I encountered in all test so far (OK, I only conducted two others ) and have no idea why it happens. The encrypted results file differ, but once decrypted, you find out that several files correspond to the test for sample 15 for example.

Anyways, was wondering what to do when people submitted results in two or more "waves", but only the first wave of results contains a nickname. Should I add the nick to the other anonymous results, or mark them as anonymous? Two people are affected by this problem. Also sending out a mail...

Bleh, it's almost midnight and I've only finished sorting the results. Since I am very tired, I hope you don't mind if I just hunt the results through chunky and friedman and assemble a nice overall graph only. The graphs for each sample along with a full results webpage will be up tomorrow.

Edit: What was wrong with Sample 7 folks? So far, it has the most invalid results (ranked reference) - something like 7 IIRC!

Second Edit: Wow, a total of 40 results were invalid. However, 470 results were valid!

Third Edit: In case anyone is interested, here's the number of valid results per sample:

Sample 01: 33
Sample 02: 33
Sample 03: 29
Sample 04: 29
Sample 05: 28
Sample 06: 27
Sample 07: 20
Sample 08: 21
Sample 09: 25
Sample 10: 21
Sample 11: 25
Sample 12: 27
Sample 13: 27
Sample 14: 24
Sample 15: 25
Sample 16: 26
Sample 17: 29
Sample 18: 21
Total: 470

Fourth Edit: Ah, missed an invalid result!

We want re-sults! We want results!

I'll start the count-down!

10
.
.
.
.
.
.
.
.
9
.
.
.
.
.
.
.
8
.
.
.

These results can be compared with the previous ones (from August, 9th):



People were very active during the "bonus" period. It's a good point!

IT'S ALIVE!

http://www.listening-tests.info/mf-64-1-results

Contains only the overall ratings and the zoomed plot. Details will be available tomorrow. You can also download the encryption key.

All post-results posts moved to the thread in Validated News.