I had this happen to my 27 iMac about a year ago. It randomly started making a rather noisy (not extremely loud, but fairly noticeable) sound while I was using it one night. The next day, I woke it from sleep and it continued making the noise, I called Apple Care and she walked me throughout the "standard" procedures. Honestly, nothing she said fixed it.
Here is what I think it was.. Possibly a spider or small bug crawled inside and got logged somewhere in the fan and thus, created the noise I heard.
While this may sound outrageous to some, this actually happens more often than one would think. Talking to Apple Care on a follow up call, they told me that this is possible and they have heard of it happening quite often.
Let me know if this is still happening after a week or so, thanks!