Apple provides the Push Notification Server, but the app developer still has to send the notification to Apple. It's really not a feature that customers need to be made aware of - it will be a feature of individual applications.
As for reminder and calendar type apps, developers can keep your reminders on their server and send your reminders to Apple's server to then be sent to you. And Apple's push server isn't guaranteed. Therefore that's really not the best solution; although, it's currently the only solution. Apple needs to make an API for the iPhone's internal scheduler, the same one that handles your calendar alerts and alarms. The developers can set your reminders directly to the iPhone instead of handling them remotely. If someone currently owns an iPod touch, they won't get push reminders when they're not on WiFi - internal scheduling would allow this.
And as stated above, email is handled differently. You can turn off push notifications completely and still get yahoo and MobileMe push email. Push email is a specific email technology, unrelated to push notifications (they don't go through Apple's push notification servers). Now if Apple allowed it, any company could create their own email application (or even just a notification application) and service and use push notifications to let you know when you had one. But that application would be separate from the email application.
I checked Apple's weather app first thing because I figured that would be a cool thing for them to try it out with, but nope.
