Greetings!

I finally managed to extract my password database from a backup set and am happily using an Ubuntu live CD until my new HDD ships. Anyways, great news: the multiformat listening test at 64 kbps is finally open!

The featured encoders, as most of you know already, are:

Nero Digital HE-AAC (VBR)
WMA Professional 10 (CBR)
Ogg Vorbis AoTuV 5 Beta (VBR)

The anchors used in this test are:

iTunes LC-AAC at 96 kbps VBR as high anchor
iTunes LC-AAC at 48 kbps CBR as low anchor

The test is scheduled to end on August 5th, 2007 August 12th, 2007 August 15th, 2007. Its address is: http://www.listening-tests.info/mf-64-1/

Happy testing!

Is the WMA Encoder the "plain" one or the one from WMP11 that uses some kind of SBR?

It's 10 Professional that comes with Vista and WMP 11.

I downloaded the Torrents and I'm seeding them now. The samples 1-17 have been downloaded a few times, but the sample 18 may have a problem.

Azureus reports this when I try to update tracker for the sample 18:


All other samples show "OK" tracker status.

I'll try redoing that torrent file. If it still won't work, you guys can always download through http...

Redone the torrent file, seems to be working now! (please delete the old torrent file and download the new one:
http://www.rarewares.org/sebastian/Sample18.zip.torrent )

Yay! I might actually get time to contribute results this time.

Thanks. It works now.

Great job Sebastian, I hope we will get a lot of results.



That last part can be left out I guess. The uploaded package has the complete encoder, including the linux binaries!!

I removed the info about the hardcoded value.

God damn it, it's becoming increasingly difficult to detect even 64 kbps encodings. Either codecs are now really good or I'm going deaf. On the tests I've performed so far, the only sample I can easily and 100% reliably detect seems to be the low anchor. And I'm using $300 Alessandro MS-2 headphones with headphone amp and M-Audio sound card.

These tests are an excellent way to lower one's self-esteem

Hi all,

just to mention that the more accurate results on this test are, better quality of future Nero AAC Encoder you can expect. This is because results from listening tests are used to tune the encoder settings. So, please concentrate very hard when you perform the test. I don't have special hearing, but I can hear differences in these encodings and thus I believe that any of you with normal hearing can hear the differences. Thus please concentrate. At least you must be able to discover low anchor, but please try to grade all samples and all encoders.
And it is not important that Nero wins (though I would like it ), only that there are as many as possible results and that you tried really hard to hear differences.

Cheers, Goran.

This is my first time participating in listening tests (though I have done rather a lot of ABXing with foobar2000).

To be sure:

I have to send the SampleXX.erf files generated through menu File>Save Test Results ?
(so that would a RAR file containing 18 .erf files)

Second question: Why isn't included iTunes AAC 7.3.1.3 64 VBR? Seems to me unfair not to include this speed.

Third question: Will I be able to see my own results?

Thx

Edit: Added second and third question

No, you're not going deaf - codecs are becoming better and better

Correct - at least if you want to test all samples. You can also send me an archive containing only five results if you only tested five samples.



Because that is LC-AAC which is not really optimized for low bitrates like HE-AAC. LC-AAC and HE-AAC will be featured in an upcoming 80 kbps listening test.



Not until after the test when I publish the encryption / decryption key.

Ok, then what sense has to use VBR iTunes AAC 7.3.1.3 at 96 kbps and not higher speed? Why are high and low anchor included?

Just learning

Typical. I was hoping this one would be a walk-over. The last one I participated in, the huge ~128 kbps test around new year 05/06(?), was a nightmare.

I think that I may participate this time. This will be the first listening test in which I will be involved, but, as the old saying goes, practice leads to perfection.



Where can I find the linux binaries, please? I would love to encode my collection with your encoder, as it seems to be the "best of breed". And speaking of that, I can even provide Debian (also usable by Ubuntu Linux users) packages for that and I would be glad to provide packages for amd64 and powerpc too.

Regards, Rogério Brito.

http://www.listening-tests.info/mf-64-1/NeroDigitalAudio.zip

PowerPC?

The Nero encoder is distributed binary-only. Unless you get Nero's sources through some sort of NDA, you won't be able to create binaries for architectures other than i386.

PS: still not interested in creating a PowerPC repository at RareWares?

Wow, this really was much harder than I thought. I never thought I'd have to listen so hard to 64 kbps music to hear the artifacts.

One thing is certain: It's definitely usable!

You can read about use of anchors in ITU-R BS.1116. Also try searching on this forum, there were already discussions about anchors.

Fortunately transparency at 64 kbps is highly dependent on the sample. I had huge problems identifying the encodes with classical (Paganini_Allegro_spirituoso) and "normal" music (Senor), but it was much easier with a kind of music that is most familiar to me (symphnoy_metal [sic]).

Anyway I had to do several testing rounds and multiple listening passes to learn what to listen to; concentrate on cymbals, sibilants and on other high-frequency content and the encodes should be easier to spot.

Anyway at least two of the three codecs are really good.

Yes, I basically narrow in on a second or two, with lots of high pitch detail, and listen over and over and over again.

Same here =)
This is awesome, first time I participate in such test <3

The Deutsch man was an interesting soundclip, first I thought this is gonna be impossible to distinguish but turned out it was one of the easiest so far

Interesting. I thought he was pretty tricky. At least as difficult as the rest.

(5/18 samples done, almost 6.)

I'm a first time listening test participant too, using an old TB Santa Cruz with Sennheiser HD465. Currently 12/18 samples done. I'm completely surprised at how good 64KBps sounds! And the high anchor is transparent most of the time for me.

Last time I tried 64kbps sounded less than impressive, but if the improvements keep up, I'm sure it'll sound transparent for me in a not too distant future.

edit: all samples tested. I'm shocked that I for the most part only hear the difference in distortions that exist in the original but somehow get mangled in the lossy versions. The music itself doesn't sound much different in my ears (which admittedly aren't that great)

BTW, thanks to everyone who submitted results so far. I will check them out as soon as my HDD arrives.

Working on a few now. It's definitely still do-able at this bitrate, even though the codecs have significantly improved since the last time I tried one of these. I don't like the signature sound of one of the competitors (and I have my suspicions on which one it is).

I showed my 12 year old boy how the testing works. His young ears can hear more acutely than mine, but I still have the better ability to pick out differences in codecs. A lot of it is knowing where to listen.

Three real competitors seems like a good number to compare.

The high anchor's got me, can't abx ~96 kbps confirmed.

The test is not finished... Why is there no link to the samples?

It's in the readme: http://www.listening-tests.info/mf-64-1/

Thanks!
But I think the Unix scripts were not included, don't know why. They seem to still work perfectly.

How to for MacOS X:
* Get the Ogg Vorbis, FAAD and WavPack binaries from rarewares.org
* Rename faad_OSX to faad and make it executable:
chmod +x faad
* Make sure that the vorbis, faad and wavpack decoders are in your path. If you copied them in the 'bin' folder:
export PATH=${PATH}:bin
* Download the abc-hr_bin.zip from Roberto's listening test
* Copy decodeall.sh to Sebastian's listening test folder
* Edit clicodecs.cfg:

* Run the script:
./decodeall.sh

You will need to have all the samples unzipped before running the script, as described in the doc.

Edit: wavpack was missing from the config file

Thank you for the tutorial.
I would suggest using Nero AAC Decoder instead of faad, just like in windows setup.

There does not seem to be any MacOS X version on the Nero website or rarewares.org. Do you know a place to get it?

Oh, yes you are right.
Sorry.

Can I be one of the first to request an extension to the test as I've just got home from an overseas holiday (Fiji - very nice ) & won't have time to listen properly until this weekend, pretty please?

Man!

Is it just me or is anyone else having a tough time differentiating between Ogg Vorbis AoTuV 5 Beta and Nero HE-AAC Jul 20 2007? This is a nice test. I'll have to sit down and really go through all the samples but man... I'm impressed with how well these two codec's have been developed.

Well I atleast "think" vorbis and he-aac goes kinda on the same level
I also have my suspicion that its the wma that sounds crap (next to low anchor ofc) ^^
Time will tell when the test is finished, I still got 7 tests to go through...gaah!

I would like to see as many listeners as possible participating in this test and thus it would be great if Mares would postpone closing for a week or more.

Thinking of it since I am using FAAD my Nero HE-AAC results will be useless... I will be testing only the AAC part in it. Will be like a 48kbps contender!
Sebastian should I continue testing?

Can you decode files on some other system and then only copy wav files?
Can somebody put decoded files online?

FAAD2 also decodes the SBR part, no problem. Only problem is that the delay is not compensated and therefore it might be easier for you to pick out the AAC files.
I can check what the delay is

Hmm actually it seems the MacOS version on rarwares is pretty old. Maybe it doesn't have SBR?

The delay you should cut from the beginning of your wav files is 2624 samples.

Isn't FAAD capabile of decoding HE-AAC?

Bleh, too slow. That's what happens when you open sites in new tabs and view them only after minutes...

As for postponing, I will think about that. There are some days left, so no hurry.

Regarding the HE-AAC problem - bear in mind that the WAV files you produce have to be identical to the ones produced by the Nero decoder since the encrypted configuration files also contain hashes of the WAV files. Maybe someone who has some webspace can put the WAV files online.

The ABC-HR program did not complain when I used Quicktime-decoded HE-AAC files (which do sound different from FAAC-decoded files)

Yes please! WavPack format would be top.

Now that is strange. Then what are the verification hashes good for?

I uploaded the decoded Nero files somewhere. If there are more people who need this (so users that don't use x86 linux or windows), please send me a PM and I will give you the link (don't want too much traffic).

Regards,
Menno

I for one could use a few extra days. Each sample takes quite a while, and you only have so much free time after work.

Are we pressed for time? If so, why?

I woild like that too, im very short on hours this week