What function does removing the card do? Surely if you have a supported card for the grey boot screen you don't need to remove the GTX980?
I'll explain it clear for MVC here.
In order to for the graphic card work under OS X, including boot screen and installation, EFI, or a "flashed" card is required. Sure you can keep the stock, unfleshed 980 in the system but if somehow the nVidia Web driver is NOT present and the system simply will not have any display what so ever.
With a card flashed with EFI, even when the nVidia driver was NOT present at any given environment such as installation or fresh OS X, the card will work but to a limited fashion. i.e. no acceleration, no native resolution support and such. It is upon installing the nVidia driver, a stock card will then have extended function such as CUDA, acceleration and such.
The bottom line is, without a flashed card to present or display the desktop environment for you, there is no way you can install nVidia driver, unless you have remote control setup, which I highly doubt is enabled with a fresh OS X install.
Hope that explains.