/System/Library/Fonts, do not touch.
/Library/Fonts, have at it. If, for instance, you don't care about reading chinese, Korean, Arabic, and so forth you can save hundreds of megs by removing the right fonts, and you won't break anything.
Of course, that isn't much disk space, so it really isn't worth it. (I futz with it more to trim the fonts that appear that appear in menus than for space.)
OSX will put back fonts if you delete something it really, really requires, so it is pretty safe. Have a bootable backup handy just in case.