I can see how it would be difficult if used like a more traditional mouse, with the device mostly under your palm.
But the iMac mouse was shown by Apple as something to grab between finger and thumb (on the sides where the coloured, soft-touch plastic was), index finger resting on the button.
I found it very fast, precise and comfortable to use. The notch in the button helped orient it by touch in the correct direction, and the two-tone mouse ball was a fun detail to notice through the translucent top shell.