Yes that is how I use it but you can also just turn off Cellular. I also noticed the favoring of Wi-Fi Calling right after I got AT&T Carrier Update 30.1 that enabled LTE Band 8 for Puerto Rico Disaster recovery using Google's Project Loon as of Oct. 20.
Note: How do I update iPhone carrier settings? Make sure your device is connected to a Wi-Fi or cellular network. Tap Settings > General > About. If an update is available, you'll see an option to update your carrier settings.
Dave