Yeah, gestures won't be any stranger than people "talking to themselves" by talking to a bluetooth earpiece. And I think the gestures required could be quite a bit simpler than what movies depict when characters interact with hologram interfaces.
With eye tracking, you could do most of the pointing with your eyes, and just hold your hand in your lap and tap your forefinger and thumb together to click something.