It should be the other way around...but sometimes
other stuff causes this.
What OS version are you using?
I've seen this behavior occur with third party
plugs in being attached to tracks.
Do you have many or any for that matter?
How many tracks are you running; and how many plugs in, soft snyths etc...are you running?
How much ram do you have?