Let me tell you, I was just as curious to the sound level of having the dual drive just as you were and was also debating back and forth between getting an external drive or not. I came from a 2010 27" iMac with an HDD only configuration and the hard drive noise really drove me nuts. I'm very OCD about noise.
Now I own a 2011 iMac 21.5", and its worlds better in terms of the HDD noise. Keeping the OS situated on the SSD helps greatly. I use my HDD for iTunes and sample libraries for Logic Pro only. Yes, there is a slight second delay when I try to load up a song or movie as the drive spins up, but it's still much more quieter than my HDD only 27" iMac. When spun down, it's dead quiet. Of course, noise levels are subjective and we're all going to have differing opinions of what is quiet.
Before getting my iMac, I also shopped for 2.5" external drives. And I also asked in the forums. I ultimately got the dual drive configuration because I also appreciated the minimalism of having less external devices and less cable clutter. However, I'm sure you could find a really quiet 2.5" external drive, though having it out in the open instead of buried in the chassis of your computer might also result in more audible noise, unless of course you can tuck the drive away somewhere farther from the computer. Unfortunately, it would be difficult to compare without actually trying both configurations back to back.
Which model are you also considering? The innards between 21.5" and 27" models may also differ, which may also affect the sound levels as well. All in all, I am extremely satisfied with the sound levels on my iMac. It's incredibly quiet for me and I'm very happy about it.
