Things have changed somewhat in the last two years (when this thread started).
In order to do an Erase All Contents and Settings on your device now (later versions of iOS) you are asked for your Apple ID password. You can't continue unless you provide it.
Doing so means the device is automatically signed out from all Apple services and removed from Find my iPhone before it's wiped.
That means the new owner gets a stock device.
How can they access your stuff if on a stock device you've never signed in?
So, no, they can't.
Thank you for confirming