In the last few months I have read many concerns about using MPC due its lack of support by the author and the recent improvements of the newer encoding formats (AAC above all).
As a member of this community and lover of MPC quality I must admit that this concerns are true. Unsupported software lead to useless data in a couple of years.
Just for curiosity and to have to force to abandon THE KING (MPC) I request to the gurus of this board to setup and run a listening test between MPC and the relevant others.
I think the result of this test would help many of us in our choice for the codec to use in our next encoding.

Thanks in advance. The discussion is open !

Hecatombles

I see no need for such a test. The only thing you have to ask yourself is:what portable player do I want and what formats does it support?

If all you want are quality files,then just use the recommended quality setting for the format needed for the player-for lame mp3 aps, for wma-some higher bit rate 2 pass vbr, for aac nero-transparent profile, for quicktime/itunes aac-192 kbps, for vorbis q5 or q6. That basically ends the need for any test.

The discussion is not on the convenience of the format for using it on the portables.
Many of us are using MPC for arkiving. I need only to find the answer to the question: is the MPC quality still the best for lossy arkiving or have the recent innovations obfuscated it in some way ? If yes: how much ? Only a listening test could answer this question.

Hecatombles

Ask Garf or search around. He tried running such a test but found it was futile due to the lack of usable results. Some people here struggle enough with the 128k tests.

But most other formats don't focus on higher bitrates at all. Currently, their improvements seem to be almost exclusively for low bitrates. I don't think MPC has been surpassed in regards of being the "most often transparent" lossy codec for archival needs.

Dream on.

MPC Q4 won Roberto's 128kbit/s listening test http://www.rjamorim.com/test/128extension/results.html from august last year, so MPC Q7 would proberly win a high bitrate listening test today.

AAAAAAAAAARRRRRRRRRRGGGGGGGGGGGGGGGHHHHHHHHHHHHHHH

You seem to have missed this part:

One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment.

Yeah, it would be a tough test. Many untrained listeners already struggle at ~128 kbps. If the test doesn't focus on some notorious killer samples, only people with good ears and training will hear the differences. But, on the other hand, some codecs have specific problems that can't be solved simply by throwing more bits at a sample, for instance, default Ogg Vorbis doesn't improve on some problem samples anymore above a certain bitrate. It could also be the case with AAC; to my knowledge, high bitrate encoding efficiency was never seriously optimized there as well. Only a listening test would show.

I feel your pain

I don't see the need for such a test. MPC would win hands down, that is, if anyone could actually tell a difference at those bitrates. There is no mystery here.

My own personal opinion is that the next best option, is AAC. It may even be a better option, depending on what you are looking for. Vorbis lovers, hold your flames.

Now that it seems that Nero is gapless, I'm looking at making the switch. The lack of good seeking, and the lack of any hope of hardware support with MPC is enough to make me look real hard at AAC.

But a listening test at Q7? That's really pointless. We already know the winner.

I propose that if anyone "requests to the gurus of this board to setup and run a listening test", he first has to prove that it is humanly possible to actually participate in such a test. So as soon as you provide, say, ABX results proving you (or anyone else, let's not be nitpicking) can actually hear the difference between a reference, an mpc q7, lame ape and aac at around 200kbps or whatever else you're requesting, then discussion can begin.


I feel for you, Roberto.

Even for MPC this has been done before, see for instance this thread. There are also existing problems for LAME alt-presets, various AAC implementations and for Ogg Vorbis even at bitrates higher than 200 kbps.

Careful with that, buddy. Everybody knew Xing would lose on the MP3 test (me included, I admit) :B

You should cut and paste your usual response, every time someone says there was a winner in that test.

On the same sample? The only one I know of that can be done with is the castanets sample and maybe some other extreme impule clips.

I personally feel that Xing would have done much worse if the 41_30, Layla, Castanets and Fatboy samples had been used - Xing's lack of short blocks kills certain samples, while on other samples Xing’s sounds decent to OK. Personally, I don't think Xing is as horrible at encoding most clips as people make it out to be. Even so, the fact that it murders high frequencies and lacks short blocks would lead me to use Gogo or FhG over Xing any day. Just my 2 cents.....

What I conclude from the test is that apparently that's a much more reasonable tradeoff than previously thought. Problem samples are only just problem samples - they're not representative, and on a representative sample apparently Xing is simply good compared to Gogo and FhG.

In my test I found that even people with 'good ears and training' had very serious problems, to the extent that one started arguing that it had to be allowed to exchange information about the problem parts in the clips - which I certainly wasn't going to allow.

Every codec fell through for someone, and I didn't see any particular trend like MPC being 'more transparent' ( ) than the others.

I fear the only way to get a significant result is to pick known very hard problem samples. But what would be the meaning of the result?

I have carefully read your posts and one conclusion seems to rise over you opinions: if even the most trained people in ABX testing would find HARD to discover the differences in the overall quality of the modern codecs versus MPC Q7 MPC is not worth for arkiving anymore.
If no one could easy find a difference.. there is no difference, isn't it !? The answer is here ! so we can abandon MPC and we can say goodbye to Frank Klemm and his unsupported MPC.

This seems too easy as answer and the answer was not ABXed.

Citay Wrote:

> If the test doesn't focus on some notorious killer samples, only people with good ears and training will hear the differences

I think the results coming from this test should be:

1) Is there any audible differences between so called "general samples" encoded with different codecs at this high bitrate ?

2) Regarding to Problematic samples, transients, echoes etc, how is the different codecs behaviour ? In this way we will find specific defects of the different codecs and the particular attitude of one codec toward a kind of music.

With this set of answers I am sure anyone will be able to chose to abandon the idea of MPC as the only codec for lossy arkival purpose or not.

Your opionions are again wellcome.

Hecatombles

In my opinion (based on my experience), such test will only lead to statistically valid results if it's performed by hundreds of people.

The reason is that, no matter if they do spot the difference or not, it will probably be something so insignificant that the score will be 4.8 or 4.9 in that sample. In the end, all codecs will be ranked "up there" in the plots and there will be no conclusion to be made. My bet is that the confidence margins would all overlap, unless you use countless listeners (the higher the number of listeners, the smaller the margins, or so it seems for me).

IMHO, when using codecs/settings aimed at transparency, the right question to ask is not "which codec sounds more transparent at the given samples", but "which codec has less problem samples (i.e. sounds transparent more often than others)". This just means "which codec is more safe to use".

The sad thing is that this question is not any easier to answer

-Eugene

Using Q7 is really to high a bitrate to be of any use, and as we all know, you can't have something be more transparent. It is either transparent, or it isn't.

So, the question is, which codec produces transparency at the lowest bitrate (on average), and has the fewest and least annoying problem samples at that setting.

My understanding is that MPC at q5 is the leader in this category. I'm not very good at picking out artifacts, but I seem to be more sensitive to pre-echo. For this reason I choose MPC and/or AAC over Vorbis and MP3 whenever possible.

It may be near-impossible to arrange a useful listening test at this high bitrate to answer your question.

But it certainly is possible for certain individuals to hear a difference between various lossy codecs on certain samples at high bitrates.

So mpc is certainly not about to be abandoned by people who hear artifacts in other codecs at high bitrates, but not mpc.


As for the question of "which codec fails least often?", there are threads which begin to answer this - just count the number of known problem samples for each codec and you'll get some idea. Unfortunately this count is biased by the number of people who use each format, and how critical they are.

It seems to me that no codec is perfect, and that people with very exceptional ears/brains can learn to recognise the faults in all codecs, even at "relatively" high bitrates, on at least some samples.

This isn't relevant to most people in everyday use.

Cheers,
David.

Well said

I'm tempted to perform something like a ~200 kbps listening test. Not with killer-samples (as Garf said, there are not representative at all), with just with musical/common one. Recently, I've tried to oppose mpc --standard, vorbis GT3b1 Q6 and iTunes AAC 192 kbps (I haven't the latest Nero builds anymore) on a nice part of a Vivaldi concerto. I was able to ABX two of the three files: vorbis (added noise) and iTunes AAC (distorted sound). I've tried many times with mpc standard, without succes... It was only one sample, but it was the first I've selected. With some chances, a whole test is possible.

Lossy encoding for archiving purpose isn't critical anymore for my own purpose: I've HDD space enough for lossless, for ~1000 CD. I'm just curious. There are now promising encoders:
· wmaPRO
· AAC (iTunes & Nero)
· Vorbis
· mpc
and hybrid for very high bitrate. It's exciting

Yes, I know there are problem samples for any codec, but being able to ABX one specific encoder against the reference on some specific sample is far from a multi-format comparison.

But seriously, I think there are some misconceptions which lead to the pretty frequent requests of high-bitrate tests.

Most importantly I believe, modern audio encoders are much better than most people think. As Roberto's previous multiformat listening test showed, and most people who ever tried an ABX can attest, modern formats are pretty astonishing at bitrates as "low" as 128 kbit/s. The general doubt towards the capabilities of codecs at that bitrate is probably due to a kind of "boomerang" reaction from the discovery that 128 kbit/s mp3s are in fact not as good as it gets in terms of transparency (which seemed to be a general consensus among the masses for some time). Nowadays the trend definitely goes towards 192kbit/s as the new "128" but we shouldn't forget that modern formats today are a hell of a lot better than early mp3s.

What I tried to suggest with my previous post is that most of the people who root for high bitrate listening tests probably haven't tried an ABX at such bitrates for some time. Because, if they did, they'd probably realize that it's asking much more from the participants than could be hoped for in a public listening test (and I don't mean to discredit the more experienced users on this board, who have tried seriously to make plans about such tests, of course; but just proposing such a test and hoping everything would work out from there is a bit naive, to say the least).

Another thing I'd find difficult in a "golden ear" test is the doubtful validity of extrapolating the results. Interpreting the results of a listening test will always involve a fair bit of extrapolation (for example the results of such a test are usually taken as a statement about general quality, not as a statement about this particular set of 12 samples). At low bitrates this is not as problematic. There are lots of artifacts virtually everyone is able to hear, so if a majority of participants say something sounds "metallic", there's a good chance you'll be able to hear it, too. This is not the case in a high bitrate test. If you tested two or three samples of a particular genre with some codec at a low bitrate, and you can hear specific artifacts, chances are those artifacts will also be present at most other, similar samples. At high bitrates, on the other hand, you have to be much more careful about such statements.
Thus, you'd either need to have lots and lots of participants (as Roberto said), or you'd have to be really careful in concluding anything from the results.

In the end I suppose the current method of doing public listening tests at reasonable bitrates combined with the conlusions we can draw from individual problem samples and individual listening tests by some of the restless "golden ears" around here (you know who I'm looking at ), is probably more useful than anything a public high bitrate test would produce.

http://www.rjamorim.com/test/128extension/results.html

I had to hold real hard to avoid being blunt there :B

Schnofler wrote:

> Nowadays the trend definitely goes towards 192kbit/s as the new "128" but we shouldn't forget that modern formats today are a hell of a lot better than early mp3s.

This is an interesting opinion (please read the original message for the whole statement).
I think that a 192Kbps test would have a follow. As pointed in the 128Kbps extension test (http://www.rjamorim.com/test/128extension/results.html) the competition at 128Kbps is non existant because modern codecs perform the same way. However, the differences and the artifacts perceived by the listeners in that test could be the ground for this other listening test.

Guruboolez wrote:

> Recently, I've tried to oppose mpc --standard, vorbis GT3b1 Q6 and iTunes AAC 192 kbps (I haven't the latest Nero builds anymore) on a nice part of a Vivaldi concerto. I was able to ABX two of the three files: vorbis (added noise) and iTunes AAC (distorted sound). I've tried many times with mpc standard, without succes... It was only one sample, but it was the first I've selected. With some chances, a whole test is possible.

As I have written, someone has already successfully ABXed at that bitrate and found some answers.

I change the target of my proposal to from MPC Q7 vs. the rest of the world to a "simple" 192Kbps listening test (MPC Q6).

Hecatombles

quoted for truth