Having spent several months researching cameras, I've determined that full-frame cameras have three primarly adavntages:
1. The bigger full-frame sensor captures more detail. That's important if you plan to print really large (i.e., poster-size) prints, or if you do tight crops and blow them up to medium size. For web shots and typical print sizes (8" x 10" and even bigger), you won't be able to tell the difference.
2. There is less noise if you take low-light shots at very high ISO settings. For most people, that's not a big deal, but it is for some.
3. You can get a narrower depth of field, like in those arty oblique-angle shots of faces where one eye is in focus and the other isn't.
Aside from the higher cost of the camera, you also need to invest in lenses that are high-enough quality to achieve the above benefits. Such lenses can be quiite expensive. They're also bigger and heavier than lenses for smaller-format cameras.
If you're a professional photographer, you'll probably want a full-frame camera. If you're a very serious hobbyist, and money isn't an issue, and you don't might the extra size and weight of a full-frame system, then you might consider going full-frame. If so, check out the reviews of the about-to-be-released Sony A7, which is "only" $2,000 with kit lens not cheap, but less expensive than any other full-frame camera on the market. Being mirrorless, it's also smaller and lighter. I ended up buying a Sony NEX 6, which currently is $800 with kit lens. So far, I love it. If at some point I feel the need for a full-frame camera, I figure they'll have come down in price.