Personally, I don't much care for sports either. I used to watch NFL games because my father did, and I fell into the habit. I got so wrapped up in it, until one day when I realized I had spent an entire afternoon, from 1 p.m. until 6 p.m. or so, watching football. Five hours of sitting and watching, and nothing to show for it except maybe a new 2 pounds of fat.
I haven't watched a game since.
At least with the Olympics, especially the individual competitions, it's supposedly the best "amateur" (at least used to be) athletes competing against athletes from other countries. And college sports, it's about real students from that school competing against students from another. With professional sports, it's about which team can afford the best players, many of whom have no real connection or allegiance to the town they play for. IMO, it's more an entertainment product than an athletic competition.
But, more people care about this (and there's subsequently more money involved) than some local tragedy which directly affects relatively few people.