I use PGP tools in a daily basis in my mail app (Mac OS Mail). It's simple, secure and end-to-end.
You need both parties to have installed a PGP keychain management. So, it's useless if your receiver is not willing to do it.
I recommend you to store your private key in a secure Hardrive, even a printed copy can be useful. (If you loose your private key, you loose all access to the mails yo have encrypted).