I only wear an occasional baseball cap (and that's just when it's raining.)
It's not so much that I think Amazon might be recording what I say (which they will for voice-recognition purposes etc. and I know that can be managed) but also the possibility of third parties finding loopholes etc.
There's a similar potential for any such tech in your house of course, it's just what you're comfortable having.
Now you're starting to make up stuff. How exactly is Apple different if "3rd parties" find a loophole?
Apple might still be innovating on the component level (touch ID, 3d touch etc), but they sure have run out of ideas on the device level.