Because it's a lot easier to make a stand-alone keyboard than it is to make a keyboard that has to play nicely with other things around it. The Magic Keyboard is fully encased in aluminum. The MBP keyboard has to sit on top of a battery, electronics that are generating heat, and there may be design compromises that had to be factored in.
Also, I can't believe you're citing the Magic Keyboard as an example of perfection. People have complained about inconsistencies in that one too.
That's just an excuse. The Surface Pro type covers are way thinner than any other keyboard out there. Yet it doesn't suffer from wobbly keys like MacBooks do. You press on the outside keys and the key press feel the exact same no matter where on the key you press it. I would be willing to settle for a thicker MBP if it meant it would allow Apple to put in a much better keyboard.