2000 messages? Try 10000 for me. I think the reason it is slow is that everyone is downloading thousands of headers for the 1st time.
Here's a question- can spotlight search the body of your IMAP messages? I have had spotty luck at best with message body searching through POP mail, so the loss of this isn't huge. The real killer feature of the gmail web interface for me is simply the ability to search these 10000 messages in seconds and be sure that I didn't miss anything.
damn, thats a lot. i just checked mine - its actually 4700, but 2000 are unread.
in any case, for pop, there is a setting that says 'only deliver messages from this day on' - i dont see that for IMAP. is it all or nothing?