Because how you fare against top-25 teams is much more meaningful than how you fare against teams that just happened to make it to the tournament.
Woohoo Indiana gets beat on the road against Minnesota. That means a lot less to me than Indiana beating Georgetown by 10
Woohoo Indiana gets beat on the road against Minnesota. That means a lot less to me than Indiana beating Georgetown by 10