If its not on a blank screen while in recovery mode then thats the problem.
To get it into recovery mode with a blank screen so that itunes can downgrade it you need to have your ipod connected to itunes and make sure you turn on your speakers so you can hear when it connects and disconnects.
Hold down power and home button until ipod disconnects then let go of the power button while still holding down the home button. When the ipod re connects to itunes let go of the home button and the ipod should be blank while itunes recognizes the ipod in recovery mode. Then do the downgrade and you should have no problem. Hope this Helps!