I have both the Xbox One and the PS4 along with both last gen consoles (from Sony and Microsoft anyways).
For graphics quality, PS4 is way better hands down. However games seem more fun on the Xbox especially games that use Kinect and voice control. I also have a couple dance games and those are only good on the Xbox. I also like the Xbox controller a bit more. The PS4 does have Remote Play with the Vita which is neat, but I don't like the Vita controls; a bit small for me and terrible ergonomics.
I have a lot more games on my Xbox than my PS4 for the playability reasons. I actually don't have any of the PS4 exclusives, but I have all the Xbox ones.
For a few up coming games, especially racing, I'll probably get the PS4 version. Mainly for graphics quality reasons and quite a few more of my friends have an PS4 so I can do multiplayer with them.
The one big annoyance is you pretty much need Xbox Live Gold to get anything to work on the Xbox whereas you don't really PlayStation Plus. I have both subscriptions.
The other annoyance is fixed internal HDD and the lack of external storage support. I have a 1TB drive in the my PS4 and it's a third full, whereas on the Xbox One I switch games around quite a bit (especially when friends visit), that I'm always uninstalling and reinstalling games. Microsoft did say support is coming soon, so I look forward to that.