If i understand you correctly...
You can disable auto-launch from the iPhone or the Apple Watch.
From the iPhone, go to the Watch app, find the General section from the My Watch tab, look for Wake Screen, and toggle Auto-launch Audio Apps off..
From the Apple Watch directly, click the Digital Crown from the watch face to find the app launcher, then look for the Settings app, find General, then Wake Screen, and toggle Auto-Launch Audio Apps off.