Just you. I've had mine since release day and haven't noticed any difference. If you're talking about literally clicking the trackpad, I have only used that a handful of times since purchase, so I can't comment.
I haven't noticed it getting louder, but this may help you fix it http://forums.macrumors.com/showthread.php?t=647323