I didn't notice this under WatchOS 4, but if you tap on "Now Playing" then tap on the song title being played, the screen flips to a new page. And even though the name page looks almost identical, you're now in "Music" and you're able to browse your iPhone or Apple Watch libraries, complete with album artwork. I'm pretty sure I tried this in OS 4 (after they re-introduced the ability to view the iPhone's music library) and this didn't happen. I'm very pleased.