My Music folder is in my Home folder and looks like this:
View attachment 2493836
You copy it to an external. If you open my Music folder, if looks like this:
View attachment 2493837
If I look in Previous Libraries, there's nothing there. I could probably delete, but I don't.
Your main Library is in your Home folder. If it's not visible, do this:
The option to show the Library folder only appears if you are at the home folder.
So:
press Command-Shift-H to open your home folder; go to View->Show View Options (or press Command-J); check the box to show Library folder.
Find the Containers folder and open it. Then find:
~/Library/Containers/com.apple.APMArtworkAgent
Mine looks like this:
View attachment 2493838
Inside it has two items:
View attachment 2493839
Copy "com.apple.APMArtworkAgent" to the external drive.
After you rebuild your system and before you open the Music app, copy Music and com.apple.APMArtworkAgent from the external drive back to the locations on your internal. I'm not sure if there will already be folders there with the same names, but just overwrite them. Then open the Music app and everything should be just like it was.
I don't know why starting with Catalina, I think, Apple started moving the artwork to the AMPArtworkAgent folder, but they did and it's a pain.
I have nearly 40,000 songs for which I setup the metadata including the artwork. I also have several hundred playlists. So I want to make sure when I do a rebuild everything goes back like I had it. I've done it a bunch and so far everything has worked fine.