Hi folks,
This might be a large and stupid question

I find that there are a lot of quality testing among a bunch of audio encoders. Usually some outperfom in one track but get beaten in the other. So how can I evaluate an audio encoder in terms of the peceived sound quality? And I am very interested to know how quality benchmark sound tracks are picked in those listening test. Are they categorized in some way?

To be specific for Nero AAC encoder, are there any sound tracks which can be used as quality benchmark for this outstanding audio encoder?

BTW, I have noticed there is a pinned sound track pool thread on the top of this forum. There are so many sound tracks that I simply get lost and can not figure out whether every single piece of them stands for something...

Any suggestion?

The best thing that anyone can do for themselves is to rip a few tracks that are representative of their entire library. Rip the tracks to a lossless format. Then encode those lossless formats to a lossy encoder and setting of your liking. Conduct a blind ABX test amongst all those tracks at that setting. If you can hear a difference, re-encode the lossless files to a higher bitrate setting and repeat the ABX tests. Do this until you fail the ABX test. That is normally the way that one would determine if an encoder and bitrate setting meets their needs.

As for encoder testing, I have no idea. I do know that every encoder and format have their own killer samples in which the encoder/format struggles with. The iTunes AAC encoder has trouble with thunder and the Lame mp3 encoder has trouble with a couple of Fear Factory songs. I don't know if killer samples are used to determine the quality of an encoder though.

Maybe a dev or someone keen on the process of developing an audio encoder can explain that a little bit more. Just so we can clear things up: are you do this out of your own curiosity or do you want to determine which encoder/setting is the best for you to use?

Well, maybe we have to rely somewhat on regression searching... The fact is that oftentimes we would state that some encoder does not have a good performance on some track but seldomly we would converge their advantages on similar tracks to make a conclusion or recommendation that they are good at those ones.

I am asking this purly out of my interest. But since every encoder has its strength in some type of music, is it reasonable to state that a recommended setting is for certain type of music? Or maybe trying to categorizing the input vectors is not a good idea