It seems like sometimes, after a reboot, the iTunes Store is in the US store by default despite being logged on to an account registered to another country, which will make the Radio icon show up in the Music app. Although you won't be able to access it when hitting "Let's listen" or whatever it says.
Purchasing or updating an app (for instance) will transfer you to your local App Store, and the Radio icon will disappear...