Observing the loss., How good a criterion for quality measure Options

[atici]

Ok after some other discussion that prodded me into this I decided to give it a try again.

That is, I calculated the pure loss with mp3 and mpc encoders in CoolEdit (Mix Paste, both channels inverted & Overlap) and listened the pure loss in each case with different quality settings. I observed that with mp3 standard preset, I can still figure out the melody because I can still hear some instruments (probably because of low pass filter). With MPC I hear the swoosh sound intensifying in some parts of the sample esp. when the original sample's volume is high. The average volume of the loss decreases as I increase the quality setting.

Could you tell me your points why this is not a good way of objectively evaluating how successful a lossy codec is? I think it's nice because the difference is not masked by the rest of the sample (which is usually higher in volume and dominates). But I can also imagine that using this way one cannot figure out stereo separation artifacts and even though when the pure loss is listened as a sample and sounds tolerable, the actual encoding result might have noticeable differences and is non-transparent. But isn't this also a reasonable method to supplement the results about which encoder is more successful? Can't we conclude anything objectively or subjectively by observing the pure loss? It sounds to me the discarded information is more tolerable a loss in q4 MPC than lame standard mp3 3.93.1.

This post has been edited by atici: Jul 12 2003, 20:51

[Xerophase]

Ok after some other discussion that prodded me into this I decided to give it a try again.

That is, I calculated the pure loss with mp3 and mpc encoders in CoolEdit (Mix Paste, both channels inverted & Overlap) and listened the pure loss in each case with different quality settings.
...
Could you tell me your points why this is not a good way of objectively evaluating how successful a lossy codec is?

I don't have a lot to immediately add to this (although I plan to research it, as it really interests me) but I wanted to applaud your thinking, atici. Hopefully some of the members will come in with some constructive brainstorming. It would be helpful if even some part of this process could eventually be used to analyze a codec because theoretically, the loss is something you can't obfuscate through hardware, human perception, etc.

My only concern is, based on the way lossy codecs work, we couldn't always rely on this data because we wouldn't be taking into account the "ear tricking" factor and how well the codec is doing that despite of what it's discarding, along with the other acoustical properties you mentioned.