Keep it around. While the system may be able to remember the image if you delete it, I've found that periodically and very rarely it will forget it, and if you don't have the image stored somewhere you'll have to track it down again to reset it as your background.
Yes, because even though the system will cache the wallpaper, you still should back it up in case you want to change it etc. Wallpaper shouldn't take up too much space, so IMO you shouldnt worry about it.