I try to keep 0 music on my iPad--I usually listen to music on my iPhone on the train and walking to and from my office. I do have my iPhone and iPad set to automatically download new music and books that I buy on the other device, so my newest stuff is always on the iPad until I delete it.
On days where I know I'll be listening to a lot of music at work, I usually load up my iPad from iTunes Match before leaving the house. That way I don't have to run down my iPhone battery and I don't have to kill my data plan with all the streaming music. I spent almost my entire 2 GB last month using iTunes Match on my phone. I have my entire library in iCloud, and I keep seeing stuff I haven't listened to in awhile--and when the moment hits, I just HAVE to grab it from the cloud and listen on the train.
PS--for the record, I freakin' LOVE LOVE LOVE iTunes Match.