I agree, but reality says define risk. When society feels threatened then the definition of acceptable risks changes. Not always for the good. History shows us that society at high risk will do many things we object to when at low risk. WWII the US moved all Japanese to camps away from the West Coast and so did Britain move Germans and Italians inland and to camps. Spying on their citizens was allowed during the WWII. I am not surprised that the government who tasks it is to protect us and our assets would ask for additional powers. Hopefully today we have learned from history and will not resort to as you say "some horrible surveillance state".
The problem is that a built-in security flaw (I refuse to call it a "backdoor" because it is nothing more than a security vulnerability) doesn't just open the device up to government spying. It makes your data vulnerable to everyone.
Since everyone is posting hypotheticals about bad guys being helped by encryption, here's the opposite scenario:
You lose your phone/it's stolen on the subway, etc. Thief/person who gets a hold of your phone downloads the free iPhone cracker app from the Internet and uses the security flaw in your decide to crack it.
Now, they have access to all your personal data on your phone. Your email, your banking apps, social media, personal photos, work data are now all compromised.
Thief then uses access to your email to change your passwords for various services. They get in to your bank account and drain all the money from it. Then they use that access to open new credit lines in your name. Now your identity has been stolen and you are flat broke.
But that's not it. Thief then decides he's going to have a little fun. He finds those special pictures you took of your wife and posts them online. Now your wife is naked all over the Internet. There's no undoing that.
Then they decide to mess with your social media. They do that and post things that are bad/controversial. They also find IP owned by your employer that you are bound by NDA not to share. They post that too. Your employer gets wind of that and the backlash against it and fires you. Then they sue you for compromising their intellectual property.
Now, you're broke, have a stolen identity, jobless, and your wife has her privacy and dignity stolen forever.
We are NEVER going to be able to prevent all terrorist attacks. Is it worth risking all of the above (and worse) to do something that is not going to have any impact in stopping crime or terrorism? Absolutely not.