You can use CleanMyMac. But if you don't want to use any program, you will have to find files for yourself. Here is the list of files you must trash to delete libusb:
If you are in finder and hit " Shift + Command + G " then you can get there. I deleted my files that way but still I want to wipe all libusb components from my iMac. It should be map an app so you can just delete it.