You obviously have something setup incorrectly. The first time setting it up can be a pain. For it to work smoothly you need the premium Yahoo e-mail package. Do you have it set up as a yahoo account in settings?
I use Yahoo e-mail on my ipod touch 4g running 4.3.1 (last pre-IOS 5) and it works fine, but it took some fiddling to get it to work and I already had the paid yahoo e-mail.