There is no way to mute a specific app, or mute the device through a shortcut. This is because it's a physical switch on the iPhones, and shortcuts need to be compatible across all devices.
The best thing you can do is what matrix07 described. You would need one automation to set the volume to 0% when the app is opened, and another automation to set the volume back to 50% or whatever you like when the app is closed. Of course you wouldn't be able to listen to background music or anything like that though.
For something slightly more complex, you could use the Get Device Details action to get the current volume, save that to a note or file, and finally set volume to 0% when the app is opened. Then when the app is closed, read the value from the note/file and set the volume back to what it was before.