Performance Issues

I have imported my CD collection into iTunes in the aiff format (want the FULL CD quality, not AAC) Anyways iTunes plays the songs, but will sometimes skip (as if the song was paused for a moment). It doesn't do it in the exact spot on the songs, sometimes it plays through the song without a beat, other times it skips. During this time, (only about a second or so) the playhead and time freeze. This is very annoying, and has been going on for sometime.

Here is what I did.
Copied the entire library to another computer (no issues whatsoever)
Wiped my hard drive and reinstalled OS X Mavericks
Reimported the song.

Yet it still skips and stuff. This wasn't an issue with Snow Leopard, Lion, or Mountain Lion.

Anyone have this issue, or know of a solution? The issue also occurs with QuickTime, but not in VLC for example. The songs play fine on my iPhone and iPod, so the file is fine. Also anyone notice how terrible OS X Mavericks is with memory?

I have Mail, Safari, Messages, iTunes, and Activity Monitor open. Mail used about 160MB, iTunes 60MB, and Kernal Task 401MB!!!

Physical Memory: 4.00GB
Memory Used: 3.99GB
Virtual memory: 4.00GB
Swap Used: 0 bytes

Sometimes the entire thing runs slow. I ran extensive hard drive tests, and no bad sectors were found, passed the SMART test, no audible signs of failing. For those wondering about the specs,

2.4GHz Intel Core 2 Duo
4GB 1066MHz DDR3 RAM
750GB 5400rpm HDD

Perhaps upping the ram to 8GB would make a difference, then get an SSD.

Any help regarding these two situations? One might argue to not use iTunes and to use a real jukebox, and I would except that I would rather stick with iTunes as I'm used to it, and I sync my phone and iPod with it.

 

No audible clicking noises are coming from the HDD when this occurs.

 

The thing is, when this occurs and iTunes pauses momentarily, iTunes also freezes. If I hit the space bar to pause when it starts skipping, it can take 10-15 seconds for iTunes to pause the music. When iTunes does stutter, the play head and timers freeze, and clicking or hitting the space bar cause a beach ball to appear.

Again, these files play fine on my other Mac.

I also did a in-depth Hard Drive scan (took about 5 hours) no problems found.

I also completely formatted and reinstalled Mavericks (did not restore from Time Machine backup) and am having this issue.