i'm about to buy an AMD Sapphire Nitro RX 580,
Are you sure of your choice of GPU? Sapphire Nitro GPUs are oversize, too wide, and block the usage of Mac Pro's slot 2.
Sapphire Pulse is the one that you should use with a Mac Pro if you want the second 16x PCIe slot to work.
I want to know if this GPU can show the bootscreen
Since this is a PC GPU, not a Mac Edition one, the answer is no.
Pre-boot configuration with a Mac Pro requires that the GPU supports the Mac Pro EFI 1.10, which only AppleOEM/Mac Edition GPUs have. You can also flash the PC GPU, if the there is a public firmware that supports MacEFI, or use EnableGop if the GPU is UEFI GOP compatible (most GPUs relased from 2015ish forward are UEFI GOP compatible). EnableGop is the prefered method today.
For EnableGop to work the GPU is required to have the
FACTORY INSTALLED GPU FIRMWARE completely unmodified, so, 2nd hand mining GPUs or any other GPU that had the GPU firmware modified won't work until you re-flash the factory firmware.
or there's some workaround to enable it
Yes, this is the main thread for EnableGop:
You wait ten years for a driver to enable pre-boot graphics on unsupported GPUs in EFI era Macs, and then two come along at once... This post is to announce a firmware driver to enable the native Apple boot picker and early macOS boot progress bar (plus other parts of the firmware UI: target...
forums.macrumors.com
If this process is too much complicated for you, my BootROM Reconstruction Service already provide a fully upgraded 144.0.0.0.0 BootROM image with EnableGop already injected.