Yeah, I think it's just decades of conditioning. There have been portrait photos since the late 19th century. Portrait video, meanwhile, is a phenomenon less than a decade old.
I'm not denying that portrait video looks… weird. It does. I'm just not a great fan to the snobbish reaction.
I must say I can understand the reaction somewhat since quite a few displays for computers and TVs are in landscape and it's difficult to turn them 90 degrees while a phone it is relatively simple to turn the phone around according to the video presently viewed.
Another thing that I think goes against the idea that we're simply not used to portrait video and that's why it's considered weird is that I don't remember anyone complaining that it looked weird when TVs went from 4:3 aspect ratio to "widescreen" (16:9). I think the vast majority agreed that it looked better. But then again maybe that's because we have seen the wider aspect ratio at the cinemas?
However, I still think a movie is one thing (more or less never OK as portrait) and a photo another (OK with whatever aspect ratio used as long is it "works" for the photo).