No, it's not. In fact, quite the opposite. Whenever the GPU is called and utilised properly, the system flies. GPU accelerated apps are fluid, those that rely solely on the CPU can stutter.
Yosemite did some damage to the general impression of the iMac 5K GPU. The UI animations made the GPU (especially the M290X) seem inadequate. However, running iMac 5K on El Capitan is a whole different story. When you see the OS fly with 20 open windows, images, Safari tabs and numerous apps - you'll see that the GPU is no bottleneck. Far from it.