For great portability and great sound I love my Cowon D2 DAP. It can play MP3, OGG, FLAC, and APE formats. It has an SDHC slot that some people have loaded with a 16GB card.
It's internal amp is quite powerful and can drive many headphones without an external amp.

http://www.anythingbutipod.com/archives/20...o-d2-review.php

If you need to have a PC based solution, a sound card based on the Envy24 chip is a good bet.