I would do just inside the folder. However, this is not a recommended thing to do. You can purge the browser cache by Resetting Safari(or defrag it on 10.5 instead, using a trick). You can purge the OS X caches by booting in safe-mode. (Hold Shift at Apple/chimes)
If you still really want to mess with the caches, perhaps download Onyx. It is an app that does those sort of things.