It seems that when playing music from iTunes, the songs aren't actually being downloaded, nor is there an option to tell it to. If you want local copies, you'll need to manually download them.
(It's easier if you view by album so you can download an entire album in one click.)