I wanted to downgrade too without a clean install, but found no way to do it. I didn't try using a the firmware file from another Mac yet...
I always thought you could just downgrade by installing an older version over the newer one, without getting a downgrade error message, at least the main version has to be the same and the Mac is compatible.
Either this has changed or I confused something. Maybe it only works between double-point releases, what are mostly gone since 10.15.x, but would be the same now.
Since the advent of split volume (SSV and Data) downgrading has to be done using a bootable installer of the macOS you want to downgrade to.
You have to erase the volume pair of the latest OS that you are downgrading from.
Then install the older macOS that you want, from the bootable installer.
This will give you a factory fresh install of the older macOS. If you want to migrate all your data and settings from a TM or CCC backup of the newer macOS, you can do this during Setup Assistant when it asks if you want to migrate from another Mac a backup etc.
Choose the backup you want to migrate from.
You will get a warning that the backup was made on a newer OS and recommending you update to the newer OS before migrating.
But there is an option to SKIP. This has always worked for me.
The most likely problem is that your Photos Library may have been converted by the newer OS and be incompatible with the older. If this happens you can either download a fresh copy from iCloud or repair it, which will restore compatibility. This does not happen between 15.0 and 15.1.