It's been determined to be legal. Though like I said, while it fits the barebones legal definition of expectations of privacy, it's not exactly...for lack of a better word...kosher. Like if I'm having a conversation with you in a public park, sitting on a public bench, speaking in a normal tone of voice, someone sitting on the bench next to us could easily hear our conversation, take pictures of us, then post it and what we talked about up on the internet later. It'd be legal, because we don't have a reasonable expectation of privacy in that park.
But it's kind of a creepy thing to do. Yeah, the guy has the right, but...jesus.
The NSA is working off this same idea, that we don't have an expectation of privacy over our raw metadata while using an open network administered by a neutral service provider that's actively aware of everything we do on it. By a few good lawyer arguments, it's been found legal.
And the backdoor thing? Yeah, that's another hinky, but ultimately legal bit. Since they're only putting their foot in the door, but not actively invading your privacy until they have a good legal reason to do so, they can get away with it.
It's all 4th amendment proof, but the NSA is toeing the very edge of it with everything they do.