Multiformat Listening Test @ 128 kbps - OPEN Options

[ep0ch]

Why are there 18 torrents and not just one big one?

[Sebastian Mares]

Hm ... Why nobody care about final size? It´s already normal.
¨It´s VBR. It´s normal that  real bitrate will be 140 kbit/s while nominal bitrate 128¨
I think it´s normal that audiocodec can produce some deviation from nominal bitrate . But there is difference between final size of files encoded by different  audiocodecs.  Ok , maybe itunes better than nero, but itunes almost always has a bigger file size. Maybe at the same file size the result will be different.

Yes, I´m caring about final size. My mp3player has 256 mb. And If my mp3s will have 264 mb I doubt I can explain point about VBR  to my mp3player.

The same discussion was about  testing of videocodecs. Some videocodecs always produced slightly  bigger size than others. But it´s  not reason to left it as is.  It´s impossible to feat 4600 mb gb to 1 standard DVD 4486 Mb.

We are talking about a special case here. The settings used were chosen because they average to 128 kbps on a large set of audio tracks. What we test here are short and usually difficult to encode samples and that's why the bitrate goes up. And if you want predictable file size, you can use ABR for your MP3 player.

[Sebastian Mares]

Why are there 18 torrents and not just one big one?

Because some people might not want to download the whole sample package. I will ask Roberto to additionally create a torrent containing all samples and maybe also ZIP all samples to one file for HTTP DLs.

[IgorC]

And if you want predictable file size, you can use ABR for your MP3 player.

Thank you for advice but it wasn´t my point.

[jetpower]

Why are there 18 torrents and not just one big one?

Because some people might not want to download the whole sample package. I will ask Roberto to additionally create a torrent containing all samples and maybe also ZIP all samples to one file for HTTP DLs.

I think it will be more convenient to have one torrent with all files included seperately. AFAIK BitComet and Azureus allow users to select individual files from a torrent.

[Sebastian Mares]

OK, as I said, I am going to forward this to Roberto.

[sTisTi]

Oh damn, these codecs are really too good for me
I've just gone through some samples, and while I can easily detect the low anchor, everything else sounds the same to me. All my ABX trials keep failing, this is really frustrating. I expected it to be pretty hard, but not this hard
How are your experiences? can anyone suggest a good starter sample which is easier than the rest? I spent 20 minutes concentrating on LesJoursHeureux because I thought it might be the easiest, but no luck - it all sounds the same to me

[Shade[ST]]

Sample 2 has one of them with quite noticeable lowpass, and the low anchor which is an obvious tincan;
Elizabeth has a part where one of the noise backgrounds sounds half a step down, (eg lower frequencies), plus the tincan
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html

Otherwise, I agree -- I'm quite surprised by the quality of modern codecs and the amount of "5s" that are in them. -- I tried a "private" 64 and 96 kbps test, too, and failed (to a certain extent) -- It seems I don't have good artifact recognition after all.

[naylor83]

Oh my... This is not going to be easy...

Think I'll just put them all at 5.0, send it all in and have it done with...

No, but seriously, this is going to take ages... heh.

[sTisTi]

Sample 2 has one of them with quite noticeable lowpass, and the low anchor which is an obvious tincan;
Elizabeth has a part where one of the noise backgrounds sounds half a step down, (eg lower frequencies), plus the tincan
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html

Otherwise, I agree -- I'm quite surprised by the quality of modern codecs and the amount of "5s" that are in them. -- I tried a "private" 64 and 96 kbps test, too, and failed (to a certain extent) -- It seems I don't have good artifact recognition after all.

Thanks for the tip with Carbonelli, I managed successful ABX results with that one finally It's a good sample because you can concentrate more easily one one thing at a time. Most other samples are pretty chaotic sounding, which makes it hard to focus one's attention.

[naylor83]

Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure...

[Sebastian Mares]

Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure... 

It should start playing at 2 seconds. That is not a bug, it's a feature.

[...]let encoders adapt themselves to the content by not testing the first few seconds of the sample.
If you encode a few seconds at the beginning that are not tested by the listener, you solve the problems of psymodel adaptation and bitrate management adaptation.

This post has been edited by Sebastian Mares: Dec 5 2005, 23:11

[naylor83]

Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure... 

It should start playing at 2 seconds. That is not a bug, it's a feature.

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

[Sebastian Mares]

Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure... 

It should start playing at 2 seconds. That is not a bug, it's a feature.

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

No idea, maybe schnofler can say something about this. I told ABC/HR to calculate offsets, so that shouldn't be a problem.

This post has been edited by Sebastian Mares: Dec 5 2005, 23:30

[schnofler]

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

Do you use the Java Sound Engine or a native output device?

[rjamorim]

Here's the torrent with all samples:
http://www.rarewares.org/torrents/Samples.torrent

[naylor83]

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

Do you use the Java Sound Engine or a native output device?

Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.

[Sebastian Mares]

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

Do you use the Java Sound Engine or a native output device?

Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.

Shouldn't it be the other way round? I mean, better result when using a native device.

This post has been edited by Sebastian Mares: Dec 5 2005, 23:34

[naylor83]

Shouldn't it be the other way round? I mean, better result when using a native device.

My thought exactly

[schnofler]

What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.

Do you use the Java Sound Engine or a native output device?

Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.

That wasn't a recommendation. Actually I'd recommend against using the Java Sound Engine, since, well, it sucks. Anyway, the problems you describe shouldn't occur either when using native output. Maybe twiddling with the buffer size will help. Otherwise, could you please describe in more detail how to reproduce these problems? Thanks.

[naylor83]

That wasn't a recommendation. Actually I'd recommend against using the Java Sound Engine, since, well, it sucks. Anyway, the problems you describe shouldn't occur either when using native output. Maybe twiddling with the buffer size will help. Otherwise, could you please describe in more detail how to reproduce these problems? Thanks.

Ok.

I tried changing the buffer size, and it seems to directly affect how far in playback will start when switching between samples. More details in a PM.

[skelly831]

For a 128kbps test, this if f'ing hard!
The results should be quite impressive

[rjamorim]

For a 128kbps test, this if f'ing hard!
The results should be quite impressive

Those sentences are almost contradictory :B

[skelly831]

For a 128kbps test, this if f'ing hard!
The results should be quite impressive

Those sentences are almost contradictory :B

LOL!, you're right, maybe you can tell English isn't my native language, I meant to say the results may not be that impressive, but we'll all be a little wiser in the matter.

[rjamorim]

LOL!, you're right, maybe you can tell English isn't my native language, I meant to say the results may not be that impressive, but we'll all be a little wiser in the matter.

No, I mean, they are contradictory in a practical point of view, not a semantical one.

If this test really is "f'ing hard", people will be scared by it and result submittal rates will be low. Or they will manage to test the codecs anyway but rank several encoders near the top, tying everything together. In either case, results will be less than impressive :/


I have already placed my bets on who will win, but such comments are unwelcome until the test ends, so I'll keep them to myself