64 kbps listening test 2005, Pre-test thread Options

[JohnV]

So Quicktime 7 will be widely available in a little over two weeks. Is that enough time to narrow down the contenders? And can anyone verify that Quicktime 7 includes an HE-AAC encoder? AAC isn't even mentioned on the preview pages for Quicktime 7.

I was just visiting Apple booth at NAB Las Vegas and saw what they have and asked this. Answer is there's no HE-AAC in Quicktime 7, maybe in the future.

This post has been edited by JohnV: Apr 20 2005, 22:29

[rjamorim]

Too bad. I suppose this test will have to wait then, as QT HE AAC would be the most interesting novelty.

[aspifox]

Too bad. I suppose this test will have to wait then, as QT HE AAC would be the most interesting novelty.

I'm still interested in the results without QT HE AAC. It'd be nice to re-compare where the various codecs are after a year of development.

[rjamorim]

I believe Sebastian will leave the decision to the forum members. He left me some offline messages on ICQ talking about not being online for a few days, but I couldn't understand why or when he will return.

[Sebastian Mares]

I believe Sebastian will leave the decision to the forum members. He left me some offline messages on ICQ talking about not being online for a few days, but I couldn't understand why or when he will return.

Well, IPS is updating the forum software to 2.1 Alpha and during this process, the blogs are offline, too. The problem is that I had a surgery near the Coccyx on Friday and am not allowed to sit or lay on my back for two weeks. I also have to go to the doctor and let him clean the open wound (which is a painful process, unfortunately) each morning.

[Halcyon]

ATRAC VS MP3PRO
=============
If it's not settled yet, I think the arguments for Atrac3+ and against Mp3Pro are pretty solid. I vote for Atrac3+ as well.

ATRAC encoder
------------------
However, I think somebody should really test whether the best (Sony) hardware Atrac encoders are _different_ from the latest SonicStage software release (assuming same version of Atrac now).

It's all very fine to assume there are no differences, but that's not a proof in scientific terms and it leaves way too much room for useless idle speculation, which is going to be rampant anyhow.

Maybe somebody with connections to the Atrac3 forum people could encourage a user there to encode a few tracks both with a hardware encoder and Sonicstage and see if they are bit-accurate?

Then again, even if we'd find out that hardware encoders are different in their output, it still doesn't solve the question of which encoder to use.

As such, for the sake of implementation easiness I recommend going with the encoder that the person doing the encoding is most comfortable using.

If somebody complains, we can ask him/her to conduct her own tests.

ANCHORS
=======
As for anchors, I think they should be of clearly higher/lower quality in many respects.

Remember, this is a subjective analysis and some people find some artifacts annoying, while other people are almost ignorant of them.

That is, making the anchors too difficult to spot will only muddy the results.

Bitrate bloat (esp. WMA Std)
=====================
I know this issue is not a favourite amongst many of us, but how will the bit-rate averaging issue be handled?

While a 5-25% difference in avg bitrate may not always be critical at 128-160kbps, it can have serious skewing at 64 kbps testing, no?

I for one have noticed in my own testing that getting WMA 9.1 Standard 2-pass VBR (ABR) to achieve anywhere near the advertised bitrate is really hard, considering there isn't much flexibility in choosing the target bitrates.

For example, I'm now encoding to 128 ABR for a test of mine and WMA9 constantly gives 140-160 ABR on most tracks, even though the target is set to 128 kbps (2-pass vbr, 9.1 WMA std, encoded from dbPowerAMP rel.11).

Can this issue be handled in any meaningful manner? Is it a problem with the chosen sample set?

BTW, the problem of average bitrate fluctuation with OGG (aotuv b3) and MP3 (lame 3.96.1) is of much smaller magnitude - at least on my encodings.

SOFTWARE
=======

What software will be (can be used) to conduct the test when the testers download the sample pack?

I'm not very fond of ABCHR Java version myself.

Also, this is probably a FAQ, but I couldn't find an answer for this by searching, how will the samples be decoded into the final test form (in regards to clipping, gain/limiting, dither)?


CHECKING
=======
Would it be possible to check the test data submissions for accidental clicks?

For example, if for a certain sample set both samples are rated below 5.0, this is more than likely a mistake. It wouldn't be too difficult to check for this, if it's not already checked for.

I have had this happen to me on various occasions during the previous test (when clicking play/stop buttons and moving sliders): i'd rate a sample that I had ABXed properly at say 3.5. However, I had accidentally also moved the paired sample rating to 4.8 without noticing it.

Other than that, my hat goes to you on staring to pull this test together. It's not easy work, but somebody's gotta do it

regards,
Halcyon

PS I hope you recover soon from you operation. Take it easy though. Health is more important than testing

[Latexxx]

Hardware and software versions of atrac3plus won't be bit-identical because all hardware encoders (Hi-MD players) always resample the audio at 44,1 kHz even if the original is at that sample rate to compensate jitter.

[Halcyon]

Ah well, another good reason to forget about the comparison or use HW encoders, imho. BTW, do you have a source for this? Do all Sony Atrac HW gear use asynchronous sample rate conversion by default?

[Defsac]

I for one have noticed in my own testing that getting WMA 9.1 Standard 2-pass VBR (ABR) to achieve anywhere near the advertised bitrate is really hard, considering there isn't much flexibility in choosing the target bitrates.

For example, I'm now encoding to 128 ABR for a test of mine and WMA9 constantly gives 140-160 ABR on most tracks, even though the target is set to 128 kbps (2-pass vbr, 9.1 WMA std, encoded from dbPowerAMP rel.11).

Windows Media Encoder seems to handle bloat reasonably.
Bit rate (expected): 64.02 Kbps
Bit rate (average): 64.19 Kbps

This post has been edited by Defsac: May 25 2005, 11:15

[Sebastian Mares]

I for one have noticed in my own testing that getting WMA 9.1 Standard 2-pass VBR (ABR) to achieve anywhere near the advertised bitrate is really hard, considering there isn't much flexibility in choosing the target bitrates.

For example, I'm now encoding to 128 ABR for a test of mine and WMA9 constantly gives 140-160 ABR on most tracks, even though the target is set to 128 kbps (2-pass vbr, 9.1 WMA std, encoded from dbPowerAMP rel.11).

Windows Media Encoder seems to handle bloat reasonably.
Bit rate (expected): 64.02 Kbps
Bit rate (average): 64.19 Kbps

According to my tests, I managed to get bitrates around 64 kbps for most of the test samples using WME, too.

This post has been edited by Sebastian Mares: May 25 2005, 11:35

[Shade[ST]]

Any news on what the final low anchor will be? Is it still to be Adobe Audition 1.5 FhG Encoder? and if so, in what mode? VBR (ABR)? CBR?

[Sebastian Mares]

Most probably same as in Roberto's last 64kbps test: 64kbps CBR, allow M/S, no I/S, allow narrowing, no CRC.

[ckjnigel]

I'm way late looking in on this, but I sure would have liked to see the bsiegel AAC+ encoder for dbPoweramp tested. The max rate supported is 64kbps.
I declare it the shiznit for Pocket PC users.

This post has been edited by ckjnigel: May 31 2005, 09:46

[guest0101]

How about using the Coding Technolgies AACPlus encoder as released in the new Magix MP3 Maker 10 Deluxe package for the listening test? See thread about it at:
http://www.hydrogenaudio.org/forums/index....showtopic=34361

The CT encoder is sure to be included in many new products shortly. This is the first consumer implementation of a file based encoder using the CT AACPlus encoder I know of. It seems to sound pretty good to me compared to Nero and Quicktime, and it really should be put to the test I believe. Thanks!

[Sebastian Mares]

As already mentioned, I don't mind people doing some private listening tests to decide which HE-AAC encoder to be used.

[ckjnigel]

I've already submitted my review of Magix MP3 Maker 19 Deluxe in the thread that guest101 initiated. I really do not think this codec can be excluded from testing because XM Satellite is so much in the news and their claims for high fidelity so loudly shouted (see: http://tinyurl.com/bk4c9 ).
Certainly, inclusion of this codec in the test will result in more widespread attention of the test results than would otherwise be the case.

[Woodinville]

As for WMA I would recommend using WMA std. It is more widespread than PRO and is what most people refer to as WMA

Why would you use an older version of WMA, when you are using the state of the art with an AAC codec? I think that isn't very equitable.

[Sebastian Mares]

As for WMA I would recommend using WMA std. It is more widespread than PRO and is what most people refer to as WMA

Why would you use an older version of WMA, when you are using the state of the art with an AAC codec? I think that isn't very equitable.

1. I don't know any hardware players supporting WMA Pro.
2. Most people rip to WMA Standard because that's the default setting in WMP AFAIK.
3. Music stores offer music in the WMA Standard format.

[Woodinville]

1. I don't know any hardware players supporting WMA Pro.
2. Most people rip to WMA Standard because that's the default setting in WMP AFAIK.
3. Music stores offer music in the WMA Standard format.

So it's just like AAC-HE, give or take a very little. Don't you think it's a bit improper to be comparing the state of the art in one technology to 3 year old technology elsewhere?

I would suggest that you use at least WMA-Pro, in the interest of simple equity.

[Sebastian Mares]

Well, I am not sure, but did WMA Pro change since the last test?

[rjamorim]

Can WMA-Pro be even encoded properly at 64kbps?

AFAIK, even Microsoft themselves don't advertize WMA Pro for such bitrates. The CBR settings only go down to 128kbps, and forcing 64kbps with VBR 10 doesn't work very well.

[Defsac]

Can WMA-Pro be even encoded properly at 64kbps?

AFAIK, even Microsoft themselves don't advertize WMA Pro for such bitrates. The CBR settings only go down to 128kbps, and forcing 64kbps with VBR 10 doesn't work very well.

As far as I can tell even at VBR Q10 you have to have either 5.1ch 16 bit or 2ch 24 bit, you can't have 2ch 16 bit with WMA Pro.

This post has been edited by Defsac: Jun 7 2005, 10:43

[moi]

I'd like to see both WMA Standard & Pro included.

I wouldn't include wma pro because no internet radio uses it and it would only confuse people.

I really doubt the people here would be confused. Most people who read these forums know the difference between WMA Standard and Pro.

I really think both should be included. Standard, because that is the WMA that by far most people use, and most mp3 players don't support Pro.

Pro, because I (probably many others as well) would like to know how it compares with the others. It can be played back on a Pocket PC (or Windows Smartphone), as well as from a desktop PC, and hopefully there will be more mp3 player support for it in the future.

[Sebastian Mares]

For your information, the test will start once Apple's HE-AAC is available.

[Busemann]

For your information, the test will start once Apple's HE-AAC is available.

hmm.. For all we know it could be years away.