QUOTE
I really have no idea what a class E amp is
Class E is Pulse Width Modulation amplification. It's normally used in RF transmitters and power electronics (motor controllers, etc). Class E has some serious potential shortcomings which make it difficult to implement one without extremely high (in audio terms) harmonic distortion. They also don't do very well on Signal to Noise Ratio.
Wikipedia - Electronic AmplifiersWikipedia - PWMYour class E box could make a competent PA amp but I would certainly not recommend it for headphone listening. However there are a lot of very affordable hadphone amps available and it is very simple to build your own, if you feel that your Audigy is not good enough.
Try
Headroom - they make some great amps. A friend of mine built himself an Elliot DOZ headphone amp. It's very simple and cheap and sounds great - instructions (and PCB sales) are on
Rod's Site. I listen through an opamp based amp of my own design.