Deleting an iApp might cause slowness
The funny thing about the iMovie, iDVD, etc, is that iTunes and iPhoto will run slower if you have never run iMovie and iDVD. It even says so in Apple's support pages, although I'm too lazy to look it up for you. Anyway, I'm guessing that since that's true, then if you DELETE iMovie and iDVD, you'll make iPhoto slower. Not sure about Garageband, but the way it used to be is that if you had iLife on your computer, it was new, and you never opened iPhoto before, then iMovie would run slower because of how integrated the software is. iMovie will always try to make a connection with the other iApps when you open it because you can add photos, music, and make DVDs from iMovie, and if you've never opened iPhoto, iTunes, or iDVD ever before, then it can't make that connection necessary for iMovie to say, "Oh, there they are!!" You need to open all 4 iApps at least once when you first get your computer.
Again, it even says so in Apple's support pages, or at least it used to. I'm not sure if it still works that way, but unless Apple rewrote all 4 programs from the ground up and changed that part of all the apps, things probably still work that way.
My guess is that if you delete something, you'll make the other parts of iLife a bit slower, but I can't say that for a fact because I'm not certain.