I had the exact same problem with new iPhone 11 Pro. Set it up from iCloud backup on Sept. 21. Photo library has 44,000 photos and the originals are stored on the phone. I have done this process on numerous prior iPhones and iPads and never experienced anything this bad. From past experience I expected process to take 2-3 days. The first two days proceeded normally and got down to about 22,000 left to download, when whole process stopped and got stuck. For 11 days I tried every trick in the books. Turned on and off, hard rebooted, paused/resumed, etc etc. Phone was plugged in on very fast wifI every day and night. The only thing that seemed to unstick the process was turning phone off and on and that only worked sometimes, mostly at night or early morning. When it worked I sometimes got a handful of photos, and sometimes over 1000 before it froze again. After first few days I opened a case with Apple Support, and sent diagnostic logs that were forwarded to engineering. They had no answer other than a "deduction" that it had to do with a low battery issue and make sure it was charged. Clearly a bad deduction since it was always plugged in and virtually always at 100% charge.
In any event after 11 days of this frustration ( the last 18 hour freeze coming 2 days ago at less than 200 photos!). I finally got a restart to kickstart it to finish yesterday at 5:30 am. Clearly there is a problem that Apple needs to fix.