Not only can you not turn it off, but as others have said it comes on when you close the app (its telling you there are new things in you feed.
You have to close the app, wait a few seconds, then double click and flip it up to avoid the badge telling you there are new tweets.
It does suck. I keep badges turned off for everything except texts because that's the only thing it matters if I miss (especially since I don't ever answer phone calls and don't have voicemail set up.)