Ok you are right for average consumer. But a lot of people me included definitely see the difference. It's a real comfort while scrolling. Especially news/text based material for my case. It's not only for gaming. I even notice pretty easily the difference up to 180Hz. (My monitor is 240Hz). The is definitely diminishing return but 120hz is not luxury anymore. For gaming the difference is even more obvious.I really don't care. I mostly read static text at more or less 0hz or watch videos at the common 30hz. Games usually don't play at more than 60fps. More than 60hz does make scrolling and UI-Animations slightly more fluid, but that's about it. For me, those animations are fluid enough, I can't read fast scrolling text at 60hz just as I can't read it at double that rate. I very much doubt that 95% of consumers will see the difference, even if they compare it side by side.
My gf who is not a geek/gamer at all notice the difference instantly when she uses my phone instead of her iPhone 15 (60Hz) for instance. She did not know/care at all about refresh rate before realising the difference