Here is
a list of AMD GPUs that work with Macs. The list is written for OCLP, but will generally work with bare-metal use if your Mac boots a needed OS version. The exception is cards that require spoofing via OpenCore. I like this list because it often covers oddball card versions, along with solutions for cards with issues.
The RX 560D is listed as Unverified, meaning no reports if it works (or not). Looks like it doesn't work without spoofing to a supported card, or flashing, as you did.