I've upgraded my 2 macs + the MacPro 5,1 I use at work (they were running on 10.9.5 before) and I didn't didn't notice any performance slowdown. It's up to you I'd say you're good to go.
It shouldn't affect your peformance in any way, but there are reported cases where the update went astray. An update is always a very invasive procedure and can potentially mess your computer up no matter what OS or which version we are talking about. Make sure that you have up-to-date backups. You can always try clean install afterwards if something goes wrong.