I bought a refurb(I'm typing on it right now.) since beginning of last year 2011.
Honestly, i'll probably buy refurb for the rest of my life unless i find a better deal on some other store or website. Their is absolutely NO difference between a new one or refurbished one other than a refurb may have used for some time.
Mine looked BRAND NEW. Of course they change the hard drives out and install new one/s. They also change the power supply. Those two parts are the parts most likely to go out since they are still moving parts, and they replace them with fresh new ones.
iMac comes in a kinda cheap looking white box, not the nice iMac boxes when you buy them new, but who really cares about the box huh?
Here's some more info.

Hope this helps.
The Apple Online Store is the exclusive provider of Apple Certified Refurbished products and Clearance Items.
Refurbished Products
Apple Certified Refurbished Products are pre-owned Apple products that undergo a rigorous refurbishment process prior to being offered for sale. We back these products with our standard one-year limited warranty. For extended coverage, you have the option of purchasing the AppleCare Protection Plan with any Apple Certified Refurbished Mac, iPod or iPad.
Refurbishment Process
Each Apple Certified Refurbished Product:
is fully tested (including full burn-in testing)
is refurbished with replacement parts for any defective modules identified in testing
is thoroughly cleaned and inspected
is repackaged (including appropriate manuals, cables, new boxes, etc.)
includes the operating software originally shipped with the unit and the custom software offered with that system
is given a new refurbished part number and serial number
is placed into a final QA inspection prior to being added to sellable refurbished stock
Refurbishment procedures follow the same basic technical guidelines as Apple's Finished Goods testing procedures.
Clearance Items
Clearance Items are new, unopened products from Apple and other manufacturers offered with special pricing.
Policies for Refurbished and Clearance Items:
Sales To End-Users Only
The Apple Online Store sells and ships refurbished products to end-user customers only.
Order Assistance
For products and pricing, Apple Store specialists are available 7 days a week at 1-800-MY-APPLE from 4:00 a.m. to 11:00 p.m. Pacific time. For order status or assistance with an order you have placed, contact Apple Store Customer Service at 1-800-676-2775, Monday - Friday from 5:00 a.m. to 8:00 p.m., or Saturday - Sunday from 7:00 a.m. to 4:00 p.m. Pacific time.
Corporate Gifting and Rewards
For orders that exceed nine units, please contact our Corporate Gifting and Rewards Team at 1-877-275-IPOD.
Return & Refunds Policy
Apples standard Sales and Refunds Policy applies to all refurbished and clearance items. If you are not satisfied with your purchase, please call Apple Store Customer Service at 1-800-676-2775 to request a return within 14 calendar days of your receipt of the item.
Defective Items
If you discover what you believe is a product defect for an Apple-branded product, please contact an Apple advisor at 1-800-APL-CARE (1-800-275-2273). If you discover what you believe is a product defect for any third-party product, please contact the manufacturer directly for information regarding the manufacturer's warranty.
No Engraving or Gift Wrapping
The Apple Online Store does not currently offer engraving or gift wrapping on refurbished products.