what I do is i fill a tupperare container with dish soap and water and let the bumper sit in it for an hour. Than I wiped it with paper towel. If it gets really cracked and stuff apple will take it back.
i bought a non apple bumper and its working great for me. The quality is excellent compared to a previous bumper I bought on ebay. The official bumpers are a bit pricey and I've heard terrible things about them scuffing the band. I wouldn't try anything corrosive, I use iklear to clean all my apple products. If you buy the set they even include a cloth that polishes due to a rough texture on the cloth.
EDIT: oh sorry you meant cleaning the bumper not the phone >.<