Vega cards do work in macOS, though I don't have one personally.
First thought: you may have some extension that isn't expecting a Vega GPU. Got a spare drive, you could install macOS clean on, add OCLP for Monterey, and try booting with the Vega card again? If it runs OK on a clean install, in either Mojave or Monterey, then you should investigate your Library/extensions folders. Pull out a few 3rd party extensions at a time, narrow down which one(s) are causing the hang.
Second thought: when OCLP does its configuration and root patching, it only takes into account the GPUs that are present. So your Monterey install will have drivers for the 680, but not a Vega card. This would explain the crash late in boot, when the high-level graphics driver is supposed to load. But ... this doesn't explain why Mojave would lock up. Mojave doesn't need any patching on a 5,1. Try holding Option and direct-boot into Mojave, bypassing OCLP. See if that allows your Vega card to run in Mojave.
To redo OCLP drivers for Monterey, put the 680 back in (Vega removed) and boot into Monterey. I'd create a blank Admin account, and set Monterey to boot only to the login screen. If there is something in your regular account that is tripping things up, this will avoid it.
Load the OCLP app, go to Root Patching, and revert (remove) them. Shut down, swap video cards, and boot back into Monterey. It will boot and run slow, because you'll be running with software rendering. Assuming it doesn't crash at the Login screen, log into the alt Admin account (if you created it) or your regular account.
Load the OCLP app again. Rebuild OCLP to disk (which takes into account the Vega card). Don't reboot yet. Redo the Root Patching, which also takes into account the Vega card. Then finally, reboot.
