The difference between a lossless signal and AAC 256kbps is virtually indistinguishable for 99% of the population. You are either exaggerating wildly by using the word "terrible" or there is some other problem going on but I can guarantee that the problem is not solved by using a lossless format.
I get so tired of hearing people go on about the topic of needing lossless when they have in all likelihood never even tried a proper double blind test to see if they can hear any difference at all. Certainly to call one terrible in comparison to the other would indicate a dramatic difference which simply is not the case in the real world.
I'm a user of Shield TV Pro 2019, Sony X950G, and Samsung Q90r. In the beginning, I have no idea that how much audio pass-through will enhance the audio.
But after I tried to play the same video in and not in pass-through, I knew that I definitely underestimate the experience of pass-through.
And the difference is more than the audio quality itself, there are also things like the feeling of more accurate spaciousness, wider sound field. For example, if without pass-through, a sound of thunder might just be above the TV; but with pass-through, the sound would be full of the room, more like a real thunder happening. These kinds of differences are really easy to be noticed. So I agree with the word "terrible".
The better audio quality for HD contents is not limited to whether it's 32bit 192khz or not, but also how it performs a better surrounding sound. The reasons I need audio pass-through, are not limited to it can play lossless audio, the bigger part is that the audio technology like "Dolby Atmos" will have some special metadata to provide audio's spatial information, and the metadata will be lost if it decoded by the set then send LPCM to my soundbar.
And for most people, as you saying that they might not be able to tell the difference between pass-through or not, there's a probable cause that they just totally don't understand what better audio could be, like me before.
And though it's not that many people need pass-through now, customers still have the right to explain what we truly want. There's no critical that customers want a better device. And it's a good thing if more people could experience better audio quality.