If it was a refurb from Apple directly, I'd totally get it.
From Verizon, AT&T, Best Buy or any other third party seller, I'd be more hesitant. We USED to think that the third-party refurbs are coming from the same source as Apple-direct refurbs, but people have been reporting that there's a lot more physical defects and problems with the non-Apple refurbs. Whereas aside from the serial number, you usually can't tell the difference between a new iPhone and an Apple-refurbed one.
The problem, according to Apple's chat support, is that Apple does not sell refurbished iPhones--or is this incorrect? According to the Apple rep with whom I chatted, the only available refurb units are from Verizon/AT&T/Best Buy/et. al.