You will have to install sendmail, exim or other mail server, configure PHP mail to use this mail server, and configure it to send mail. This can be confusing and messy. You may also have problems sending email due to issues with your internet service provider. This shouldn't be a big problem if you are using a VPS.
An easier way to send mail with PHP is through the PHP-Pear Mail extension. With this, you can actually send mail though any email account, provided you give the login info. No easy way to get this done with Wordpress. Have a look at the source code or see if there is a plugin for this.