They're kind of spread around so it's hard to completely remove them. The apachectl file is in /usr/sbin/. Some of the configuration files are in /private/etc/apache2/. The php.ini file is there in /private/etc/. PHP has pieces at /usr/bin/, /usr/include/, /usr/lib/.
I'd recommend just leaving it be though.