You are basically skipping the download from the phone to the watch via Bluetooth.
Though even 2hrs is very long, which either means you have a slow internet connection or Apple servers are really busy, or, and I’ve seen this happening more than once before, it goes from 2hrs to 3min very fast 😉