I've had really good luck with Presonus. The unit I have is no longer in production but it does 24-bits at up to 96K, has six balanced outputs and a very good built-in headphone amplifier. It is made for recording and you say you don't do that but EVERY audiophile needs a why to MEASURE sound so yo need some place to plug in a measurement microphone. How else to measure room response and check out speakers? I'd look at this current model from them:
http://www.presonus.com/products/AudioBox-22VSL/techspecs
Parts Express sells a very nice Dayton brand measurement mic that come with a calibration file for only $50. The DACs inside this unit are very good, pretty much the same quality you'd find in a recording studio.