There are two separate options when you select a song - "Add to My Music" and "Make Available Offline". If you select to make available offline, it downloads the song and adds it to My Music. The "Add to My Music" just adds it without downloading. So if you want everything offline, just choose that option when you select to add a song.
ETA
FWIW, I find it confusing with songs I own, songs I don't own but have downloaded, and songs I haven't downloaded all mixed together. So I've kept the "Music Available Offline" box checked and am just adding stuff to "My Music" by selecting "Make Available Offline." That way I don't have to wonder why some song doesn't play when I'm not streaming.