It's much better to restore, because there are tons of preferences & settings that you may have forgotten about that you will undoubtedly want on your new phone: all the individual settings for each one of your apps, all of your high scores within your games, all of your text messages, phone call history, the exact placement of all of your app icons, all of your system preferences, etc.
If something goes wrong, you can always start over with a fresh reinstall in the future. But for now, just restore everything and see how it goes.