The USB adapters should be their own GPU - the 3 limit would be displays directly connected to the onboard graphics. So 2+2 should be be fine.
You'll have slower graphics performance on the displays with the USB adapters, but you should already be used to that from your iMac setup. In fact, the adapters should run exactly the same speed as before.
Out of curiosity, do you lose some of the display effects with your adapters? I have one I've used with my mini, and when using the magnification effect on dock icons, it looks much worse than a monitor hooked to the onboard graphics. I'm guessing mine has no 3D support under OSX, and wondering if thats true of all of them.
Thanks for your clarification. I did assume the same, but was a bit concerned.
As to the performance of the Plugable adapters:
As I don't use them for video (no iTunes movies or Flash/HTML surfing), so I don't see any issues. I use them strictly for tables & graphs in my trading platform. They do have constantly changing numbers, but I don't see much delay or color performance issues.
To investigate a better answer for you, I've just opened a streaming news channel player on one of the adapter run monitors. It is a Java based internet plug-in. In the original player size (3"x4"), with motion, it seems a bit jumpy and the color is faded a tad. When expanded to full screen, the jumpiness is almost unbearable. I wouldn't recommend it for full screen video or gaming. But it seems to be fine for text or fairly static information.
