It’s a bug since 16.1. They fixed it for 14 in iOS 16.1.1. iPhones 11,12,13 still have the issue. Report it to Apple through feedback. Maybe they will fix it one day . If you listen to voice message you receive in iMessage or WhatsApp and you listen with your phone in ear volume buttons will also not work