Expensive glass in general has:
1) better quality glass itself (coatings, quality of the grind, higher tolerances, etc). This gives better sharpness, and less chromatic aberration, better contrast)
2) More light gathering ability, requiring generally larger elements which are more expensive to make. This allows shooting under lower light conditions with higher shutter speeds, as well as helping the camera to focus faster or more accurately in many modern bodies.
3) Shallower DOF - related to #2 - a wider aperture allows more blowing out of the background. What differentiates a snapshot from a portrait in many circumstances is choice of aperture.
4) Faster focus speeds / technology: this can be night and day difference in sports photography or low light photography
5) More durability, weather sealing. Self explanatory
6) Constant aperture (for zooms). It's extremely annoying to shoot in aperture priority mode (or manual) and having the minimum aperture shifting based on your focal length. Plus most zooms are used at the long end the majority of the time, so the cheaper glass ends up quite a bit slower than you think when you're shooting F5.6 instead of F2.8 on a higher quality zoom.
7) better bokeh. Related somewhat to #3, and not as consistently in place with higher lenses, but most of the best bokeh comes from expensive glass.
These are what comes to my mind. Personally when I bought my first L lens, I couldn't believe the difference in saturation and sharpness. It's been a downhill spiral since then
As noted above, if you stop down a lens to F8 or F11 you will minimize differences between lenses quite a bit - even craptastic lenses can look pretty decent at those apertures. The problem is that most pictures (barring studio work or landscapes) just don't have the same quality look at such apertures. I bet I shoot something other than wide open maybe 10% of the time, excluding studio shots where I'm usually at F8.
FWIW.