Public Listening Test [2010], Discussion Options

[IgorC]

Many people will be agree that it’s time for public listening test.

Short Agenda:
1. Organization
2. Samples
3. Codecs
4. Bitrates

Detailed Agenda:
1. Organization.
People with experience of personal or public test are welcome here.
Personally I can afford help for conduction of test.
Unfortunately if I understand correctly that Sebastian informed that he don’t want to conduct public tests.

2. Samples.
Different styles of music, different levels of difficulty, pointing issues etc....? To be discussed here or in separate topic.

3. Codecs
3.a) Multiformat test.
3.b) AAC test.
I think that it’s more appropriate to conduct AAC test because there are at least 3 AAC encoders to test. Nero, Apple and Coding Technologies/Dolby. All these codecs were updated during this year. More information in List of AAC encoders

4.Bitrates.
96-100 kbits/s?
It's possible to perform test at 128 kbits/s with very hard samples. Aemese-like samples will be very easy to spot from original.


Sorry for my English.

All kind of thoughts and suggestions are welcomed here.

[Sebastian Mares]

I could imagine an AAC test at 80 or 96 kbps. At 80 kbps you could feature both LC and HE encoders and see which one performs best. The winner could be used in a multiformat listening test at the same bitrate.

Regarding the samples, muaddib once had an idea to create a samples DB that should be divided into problem samples and regular samples. When preparing a test, one could pick X samples from that DB based on lottery numbers so that people don't complain that sample Y was selected with the purpose of letting encoder A appear better than encoder B. Additionally, samples collected especially for the respective test could be used like it was done in the past.

Good luck with the test Igor. Are you planing to use existing software (ABC/HR for Java) for the test or something new? ABC/HR's development is dead unfortunately and there were some problems in my last test requiring the installation of JRE 1.5 (which some people with JRE 1.6 found annoying).

[IgorC]

Good luck with the test Igor.

That sounded sarcastic.

Are you planing to use existing software (ABC/HR for Java) for the test or something new? ABC/HR's development is dead unfortunately and there were some problems in my last test requiring the installation of JRE 1.5 (which some people with JRE 1.6 found annoying).

Yes, I remember of this issue and speaking of me I could get rid of this without problem however with some workaround. Must see what happens with other people.

Definately AAC. Nero v new Nero alone will bring in a lot of interest. Add in Apple's latest, with it's true vbr options. And it'd be interesting to see CT/Dobly in the mix since it's been left out of so many AAC discussions. And I agree with you, it should be in the ~96k range.

True VBR mode isn't available in Windows version. Many people couldn't use it.

An AAC test would be interesting indeed. But at 96 kbps or so, you would have to test both AAC LC and HE-AAC, since they offer about the same average quality. With about 4 encoders under test (Apple, 2x nero, Dolby, ...), this would give 8 codecs under test. That's too many in my opinion (risk of overload and fatigue).

Itunes has HE-AAC only up to 80 kbps. Nero's defaults option goes to HE-AAC up to ~85 kbps.
From my personal test Apple LC-AAC is better than HE-AAC already at 80 kbps.
http://www.hydrogenaudio.org/forums/index....showtopic=74781
But yes it's personal test, not public.
96 kbps will be enough to test only LC-AAC codecs.
Anyways all bitrates should be shifted to 100 kbps as Itunes constrained VBR mode at 96 kbps produces real bitrates ~100 kbps. There is mediacoder which can shift bitrates for CT encoder.

How about 112 kbps or so? There you can be relatively sure that LC is better then HE on average, i.e. you would need to test only LC.
Chris

iTunes hasn't 112 kbps. Only 96 and 128.

As I can see people interesting in AAC test and next codecs:
1. Nero 1.3.3
2. Nero 1.5.1
3. CT
4. Apple
5. Divx (???) or other?

I think 4-5 codecs are already enough. Not more. What do you think?

I found that Divx AAC enocoder has all potential to get into test.
http://www.hydrogenaudio.org/forums/index....st&p=675456

This post has been edited by IgorC: Dec 26 2009, 20:41

[C.R.Helmrich]

I see. Didn't know that most encoders don't offer HE-AAC for bitrates higher than ~ 85 kpps.

As I can see people interesting in AAC test and next codecs:

...

I think 4-5 codecs are already enough. Not more. What do you think?

Agreed. But for completeness sake: Fraunhofer is currently finalizing quality tunings on their encoder which have been going on for about two years. Release is scheduled for end of January. If there is any interest, I can ask if it's possible to provide an evaluation encoder for this test.

Chris