Magic Trackpad goes nuts, thinks button is depressed all the time

So, I got the magic trackpad a few days ago and have really taken to it. It's now sitting right by my keyboard with my mouse further over to the right (getting used less and less).

However, twice now, the Trackpad has sort of 'lost its marbles'. It starts to act like the button is always being depressed and so any time I try to do anything it starts to drag stuff around the desktop, selecting everything in sight, it's really quite annoying.

Rebooting it will clear the problem for a while.

Is this driver flakiness, or is this thing defective? If no one else is having this problem I will exchange for another unit and hope to have better luck with that one.

I'm starting to wonder if it's using the trackpad settings 'Tap To Click' and 'Dragging' under the one finger section.

Could it be these or clicking on some outer edge of the trackpad doing this?

I will turn those settings off for a bit and test.

With Tap to Click disabled I cannot reproduce the problem. I expect that with this turned on and the click to drag enabled that occasionally the system gets confused and thinks you've done a click/drag selection and will not undo it with the escape key, clicking the track pad again, etc.

I will try turning on tap to click again but leave click to drag turned off and verify I still can't get the problem to happen.