Roberto, I know you are in listening test management retirement but I wanted to ask you your thoughts on a test at 192 kbps. Now I know many have stated before that it is almost impossible to ABX at 192, therefore the test has no value. Below are my list of reasons for holding the test and under what circumstances. I would like you to consider coming out of retirement!!!

1) Strictly limit the codecs tested to limit listener fatigue. LAME MP3 192, iTunes AAC 192, WMA9 192, and maybe OGG. No reason to test MPC, its transparent at this rate. No reason to test ATRAC3. I would say no reason to test OGG at this rate either but many will not participate in a test without Vorbis.

2) A chief complaint of many with pay-per music download services is that the quality of the audio stinks. What if they upped it to 192 like Real has? Would this provide quality that is very close to transparency? This test will prove on way or another

3) For those that trade MP3s or engage in download activities using P2P (not advocating it!!!!), MP3 192 seems to be the 'new' 128. While not all use LAME, it would be interesting to see what the quality of those 192 kbps files are. While we few at HA use LAME APS when we use MP3, we are the overhwelming minority

4) Use CBR, not VBR. Why, VBR is better? Yes, it is. But music services and those outside HA do not use it. For whatever reason. This test is to see what the quality of files is that are not necessarily created by us. It also provides a nice benchmark. If 192 CBR scores well, you can assured your VBR files with higher bit rates are even better.

5) A comparison to see quality improvements as bit rates scale up. We have the 128 test (wonderful job!). It will be interesting to see how much improvement, by codec, exists between 128 and 192.

6) WMA 9 standard stinks and I would never use it. The argument heard over and over here. Yes it does at 128 (proven on Roberto's test). But who at HA really uses 128 anyway? Most people here use 192 and up unless using MPC. If WMA sounds as good at 192 as LAME or AAC then let's cease the comments. If it does not, then this test will prove it and the argument will become fact rather than WMA bashing.

It seems that you didn't understand the previous discussions about this subject. When transparency is reached it's not possible to rate codecs anymore. Discerning between compressed and uncompressed might be possible for well trained ears on some samples, but reliably rating different codecs is almost impossible at this bitrate.


The MP3 'scene' switched to using --alt-preset standard in late 2003, other sharing communities have been using --aps and/or MPC for a long time.


If you really want such a test, conduce it yourself (feel free to use HA.org as a platform for discussion), but please don't ask someone else to invest his time in a pointless/uninteresting test.

I like that idea.
killersamples could be used and, perhaps more people are capable of distinguishing different codecs @192 then the majority thinks....

I cannot believe you would regard a test of 192kbps to be pointless and uninteresting. I believe all of us here, primarily, are after the highest quality imaginable... And it has been years since Musepac was given the crown of "most transparent lossy codec" - argue with the wording all you want, you know what I mean

Yes, as a community, we have an active interest in low bitrate acceptability, we own a lot of portables and have a lot of call for it, but I truly believe that we are here because we really CARE about the music quality, about the faithfullness of the reproduction.

I bet I am not the only person to encode MP3's with --APS, Ogg's at ~5, or MPC with -Standard. I am absolutely interested in this level of bitrate (yes, of course it depends on the type of music, but with the settings above, you'll get more like ~192, ~160, and ~180, respictively, roughly). I AM interested in transparency. WE are, otherwise we wouldn't have read more than the front page of this forum.

I completely agree, it would be difficult to orchestrate, and on the vast vast majority of everyday music, they are ALL Transparent. "Almost impossible" is a useful phrase, but with enough people taking the test, I am confident that in perhaps one or more sample(s), Ogg would falter, and on a couple of different samples, Ogg would beat MP3 and MPC hands down, and that this would be discernable in the results, and be both useful and interesting.

Maybe not, maybe everyone would rate everything 5. That would be interesting, and certainly not pointless; perhaps even stop a myriad of zealotry (Most overused word ever here ).


Bottom line is: We are here because we CARE about quality, and this type of test that is being talked about IS the quality setting.

Why a listening test at high bitrates (192kbps) wouldn't work

1) Most samples would have already reached transparency at that bitrate. And choosing only problem samples would make the test less significant since you wouldn't be testing a broad range of musical styles.

2) Only a handful of golden ears would be able to reliably ABX the samples, and even after ABXing they would hardly give scores lower than 4.5

3) Since all scores would be around 4.5, the error margins would be big enough to make all codecs tied. To avoid that, you would need to have hundreds of participants, in an attempt to bring the error margins down.

4) You would have a hard time finding hundreds of golden ears, and even more, hundreds of golden ears willing to participate, because the test would be very fatiguing and frustrating, due to the dificulty of ABXing.

5) At the end, even if you managed to bring the error margins down, the codecs would be ranked so close that you wouldn't be able to produce a decent conclusion. All codecs would seem (or be) tied to each other.

I can already imagine who would produce results: Guruboolez, harashin, [proxima], and a handful of other golden ears. Everyone else will take the test and fail miserably.



I'm not retired yet. But I wouldn't even dream of conducing a high bitrate listening test.

Really, look at the 128kbps test results. I got a load of empty result files and ranked references! That means people found 128kbps to be transparent to them on some samples. Now, think about 192kbps...

Regards;

Roberto.

@dev0: calm down, dude. You seem to be in a constant bad mood lately

A while back I considered administering a test at 160kbps generating this discussion. I never followed through, for a number of reasons, only one of which was that it would have been very difficult, perhaps impossible. I may yet pick it up if I get the motivation, but I think I would have to dredge up a huge amount of participation to make it work.

I agree with rjamorim - no matter how interesting a 192kbps test result would be, it simply isn't realistic to perform a test at that bitrate. Even a 160kbps test would be very difficult.

Looks like there is still some hope left.

Hey Dev0 thanks for your support! I feel so welcome here.

Roberto, your points are understood and I appreciate them. I will cease my request for this test.



Are you kidding? Check any P2P site, 5% at best of files are MP3 APS. I wish your statement was true. Its about 40% 128, 40% 192, and 20% other (APS, APE, 256, 160, 64, 320)

I guess everyone is saying that 192 is transparent, or at least to 90% of listeners. It is to me so I will not argue, but I have some questions then.

#1 Why does everyone say WMA 9 STD is so bad if its 'transparent' at 192? OK, so it does not cut the mustard at 128, but 192 is not a big file size either. If it sounds great at 192, why is it shunned here and ridiculed as an acceptable format to purchase music in (assuming it was encoded at 192 and not 128).

#2 If Apple, Napster, Real, and others used 192 CBR as a compression rate, regardless of codec choice, does this mean we would truly be receiving the transparent CD quality we so desire from a download music service? That was my main reason for requesting the test. I know that 128 does not quite get it done, but if iTunes, for example, could be convinced to up the bit rate to 192 - would we then have what we seek?

People unable to hear any difference will probably don't send results at all, so maybe the amount of useless answers won't be much higher than in the 128 kbps test.
Anyway, if such a test was setup, I'd take part.

I expect the test conducer to receive several ranked results, even if he makes ABX mandatory.


BTW, I would like to clarify that, in listening tests, it's important to consider the empty results, sent by people that couldn't hear any difference. Else, we would be creating artificial results that only reflect the portion of mankind with very good hearing.

So, these results aren't "useless" as Pio puts them. They are of no use to compare how codecs sound against each other, but they are useful to see how good codecs sound overall (since they always raise the ranking scales).

Right. But the real purpose of a high-bitrate listening test is not to proove that 95% of the population can't hear even the smallest difference, but to check the real difference existing between encoder. This difference is small, and that's why it's better to magnify it. And in order to magnify it, a contrasted notation is absolutely needed.
I think that 3.1 - 3.4 and 3.9 are more significative that 4.78 - 4.80 and 4.83.

Offtopic posts moved here.
Have a nice and productive day.