I think the downvote button should be linked to an input box where you have to put in a reason. This would help downvoted Posts immensely since the poster would know why he/she is downvoted - let's face it most of the times you have no idea why. It shouldn't be a problem if you're a a downvoter and have a valid reason to downvote anyone.