Trackpads, especially the unibody glass trackpads, seem to be highly sensitive to invisible dirt .
Get some cleaner specifically made for a computer (e.g., iKlear), spray some on a microfiber cloth, and then rub the cloth on the trackpad glass and on the edges of the trackpad. Use some pressure (but don't break it). Next, use the opposite side of the microfiber to dry the trackpad.