Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Also, there are now several other ways to accomplish this, including recboot and TinyUmbrella.
Thank You!!! Tiny Umbrella's "Kick Device Out of Recovery" button did the trick! This was on an iPhone 4 and Tiny Umbrella 4.1.6.

For the life of me I couldn't get iRecovery even run (I had libusb in the same folder). I just kept getting command not found. Anyway, Tiny Umbrella seems to be the easiest way to get your iDevice out of Recovery mode.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.