Can you tell us more about which iMac you have?
You can get some information from the serial number label, located on the bottom of the stand.
If it is really 2008, then that will be too old for Internet Recovery, as that is not available until 2010 Mac models.
So, if the hard drive has been erased, you will need a bootable OS X installer, either a DVD, which would be 10.6 or older --- or maybe a bootable USB installer, which could be on a flash drive.
Do you know what version you have on your USB installer?
What happens when you try to boot from that?
When does it fail, and do you see an error message of some kind?
Have you tried booting to a recovery partition (restart, holding Command + r). That should boot to a menu screen, allowing you to reinstall OS X. If you did have Yosemite, then it should offer Yosemite then...
If that does not work --- what DOES happen?