In normal usage, OS X is pretty good about cleaning up the disk after itself where files are less than 20MB in size. So the average user doesn't have to defrag manually.
The only caveat is that if you do a lot of video work where files can be several GBs and being regularly written/deleted, then you may want to look into getting a disk warrior type app that will defrag for you.