I have a Diamond MX300 sound card, running Windows XP..

So I crank up RMAA to do some testing, and using this NIFTY AC97 mixer(http://www.upsystems.com.ua/support/alexmina), I figured out any PCM output level above -13.8dB makes the soundcard go NUTS. It starts throwing off tons of harmonics and noise. It doesn't gradually raise either, at -13.8 it's fine, one click up and it's horrible. Anyone know what exactly would cause this? Amp chip on the card being overloaded?

I tested using the Stereo Mixer input, no loopback cable or anything.