I've backed up and restore onto the new iDevice every single time without ever seeing any performance issues.
Performance issues relative to what? How would we know that performance wouldn't have been slightly better with a fresh install? I did a restore both times I upgraded my iPad, but it wouldn't shock me to learn that a clean install would have yielded fewer crashes and speedier performance. There's really no way to tell other than some type of measurable regression testing with two new iPads with the same apps and content - one restore and one clean install.