If you use Spotlight to search for 'GarageBand', you'll get most of it. 'Loops' might find some remaining stuff. Just be sure to verify that something is really GB stuff before deleting it.
/Library/Application Support/GarageBand has a ton of stuff in it and should likely be at the top of your delete list.