A friend of mine claims that he can hear artifacts in mp3, at almost any bit rate. He has very expensive equipment so the test will be done on his ordinary cd-player and not a computer. He would pick some tunes he’s familiar with and then I encode/decode them and put them on a cd-r together with the original samples.
So my question is what kind of problem could face? Like problems with clipping, different loudness etc. I haven’t conducted any of this tests before, so things that would look very obvious for you maybe aren’t that obvious for me.
So a detailed description would be highly appreciated.

You can use --alt-preset standard for a test of how real-world mp3 can sound, and --alt-preset insane to test maximum possible quality mp3.

Using this presets I think you won't face any of the problems you mention.

Some advices, try to make the tests as blind as possible, or in other words, try to not give your friend any unadverted clue of what he is listening to. Better if you go outside of the room or stand behind him when he's doing the listening.

You will also need at least 5 correct identifications of 5 trials (p=3.1%), in order to discount success by chance, but you can also give it more trials an calculate results at http://www.ff123.net/abx/abx.html

To avoid loudness differences, apply WaveGain and Mp3gain to the test samples.

Hanky,

I'm actually donig the exact same thing (and MP versus WAV test using a sony NS755V) and want to make sure to make the test as accurate as possible. Although I understand that MP3Gain and WaveGain are lossless, I'd prefer not change the MP3 or WAV at all if possible.

Since I will be ripping directly from the CD, then encoding from that rip, won't the volume be the same per track (which will be what is compared). I've tried it on a couple tracks and the volume appears to be the same. So I'm just wondering if this normalizing is necessary.

Zin

This normalizing is not necessary. Don't normalize if you want to compare.