Did you previously have "Recent Music" turned on? If so, then whatever was downloaded to your watch at the time is probably still on the watch even though that setting is now turn off.
I had the same issue with my watch. After turning that setting off, my watch still indicated that it had 1 gigabyte of music on it. However, after restarting my iPhone, the Music storage went down to next to nothing.