It's certainly possible, but not feasible.
1. The cost: 6970M is not your typical video cards, it's a MXM laptop card. Even if it is equal to 6850 in terms of performance, doesn't mean they cost the same. Typically it would cost you $200 - $300.
2. You have to find a specific 6970M for iMac, not random cards from AlienWare or something.
3. You risk damaging your iMac, it's not just plug in and out like typical PCI-E on a regular tower. Too many cables and screws you gotta open to reach GPU.
So, just go with the highest end iMac possible, it's worth it. Better CPU is a bonus too. Especially the refurbed one since they come with alluring price cut.