Problem with music being deleted...

I had the option of show all music from icloud...and only have about 50 or so songs from iCloud in there and none on the device itself...

I shuffled the songs for a playlist from iCloud on the music app and once they had played the little iCloud icon which you press to download disappears... and check the usage and seems they are on my device where I haven't downloaded the songs at all... I turn the show my music feature off and they are all there, the songs I had played in the shuffle, like they are on my device but they really aren't!

When I turn show my music back on, the icloud icon reappears...like available to download to my device

Is this a bug? Why are they showing when the show my music setting is off?