Apologies, usually I avoid dragging up old threads, however this was at the top of my google results.
Under iOS 4.0.1 I've found that as mentioned, the delete button doesn't appear by default.
1. Rename the host eg smtp instead of smtp.live.com
2. Accept when it complains the server can't be contacted.
3. Choose Done.
4. Choose the connector now called smtp
5. Scroll down and click the delete button
I don't know why this is and I find it annoying, I've been looking for this solution for about 6 months and had many copies of smtp.live.com from re-adding my account.