You need to change the "Ringer and Alerts" setting that allows it to "Change with Buttons" in the Sounds setting panel. Then, when you turn the sound all the way off, your alerts and key clicks, etc., will be off, too.
Of course, we can now get into the debate about the fact that you might want key clicks off but keep alert sounds on, but that's another issue...