Depends on the amount of data you have to retrieve and restore and also on your internet speed.
Its alot faster if you have a local backup to restore from in your itunes library.
Also remember that millions of others are hitting the iCloud servers to restore their backups as well. Probably luck iCloud hasn't crashed based on recent performance of apples live stream and midnight preorder.