well, it's not from the sample rate, but a higher bit size (for example 24bit) allows for much larger dynamic contrast (difference in volume).
There are still a lot of CDs out there with HDCD encoding that allows a compliant processor to read extended bit information on the disc while still playing in "regular" CD players without issue. The resulting CDs are effectively 20 bit recordings instead of 16, and owning a couple of them, I can tell you that a good processor, a good amp, and some good speakers make it easy to hear the difference. I paid 2 dollars for Joni Mitchell's "Blue" album on HDCD at a garage sale, and it's much more open and dynamic than my other CD. My wife can hear a difference, which means there is a difference. I think what happens is that they compress what was once a dynamic master into more radio-friendly "loud" versions.
I got some Supertramp CDs from Japan that are head and shoulders better than what I've found in the states. Those albums were so well produced, it's the crime of the century that their current US releases are so flat and loud and dull. I would rather have a 128kbps mp3 with un-modified dynamics than a 256kbps AAC file with the now-standard levels of "loudness" compression. Yes, you have to turn up the stereo more to reach the same average volume, but when the drums kick, they REALLY kick, like they were supposed to.
I could say more, but I need to go listen to Supertramp, now.