I dont pause when i tell siri to pause the movie. I say it in one sentence and it still takes ages.It purposely pauses in case you’re still talking. It’s much faster to not pause, but just to continue your query. eg. ”Hey Siri turn off the lights”.
It could definitely do things faster but the pause after Hey Siri is by design.