Normally this should be fine. It can take several minutes until iTunes is responsive again.
You could select and delete smaller portions (a.e. 1000 albums)
If you don't need your settings, smart playlists etc. you can just delete your whole library (after closing iTunes delete the iTunes folder in <yourUserName>/Music/iTunes). When you open iTunes again, a fresh library will be created from scratch.