I think that you can see the LEDs from the bottom RAM opening. Remove the little cover where you change the RAM.
Put the iMac down on its back.
Plug the power cord while looking up inside past the right end of the RAM slots.
Watch for an LED to light up when you plug in the power cord. If you don't see it light up, unplug, wait 10 seconds, then plug the power cord in again. Be sure to watch for the LED to light up. It will be about 5 inches (maybe 12 cm?) from the bottom.
If you can see that one LED, then press and release the power button. You should see #2 light up. And you would be waiting for a #3 LED, which should come on. If the #3 does not light up, then likely a problem with the graphics card.
I don't think that all the optional graphics cards are the same size, i'm not completely sure about that. But your "new" card doesn't do as much as the old card ?? If you can discover those troubleshooting LEDs, and the "new" card doesn't get to the #3 LED on boot, then put your old card back in place, and try that one for the LEDs. Does it go any further, to #3, then you get the #4?
I had a 2011 21-inch iMac a couple of weeks ago with a similar problem - reseating the graphics card fixed the boot issue on that one.
finally, if you can't see ANY leds come on from the bottom, you SHOULD see one or two at least, but if you see nothing, then the only recourse is to remove the front glass, and unscrew the display panel. you can pull the top edge of the screen forward a couple of inches (Be REALLY careful here. You can do this without disconnecting any cables, but it is really easy to jerk the display and snap a connector loose. Watch for the cables, you only want enough space to look down from the top. You should see the LEDs from above, this is what I had to do on the smaller iMac that I was working on.