iPhoto can get sluggish when you get into the 20k+ photos range. If you only have a few thousand pix then it may be memory or disk that is slowing you down. What are the specs on the mac you're using? Do you have a SSD? Have you run activity monitor while using iPhoto to determine where the bottleneck is?
Sorry for the delayed response- I actually try to avoid using my computer because it is so slow, I just can't be bothered. But I finally went and found out what I think the answers to your questions are:
I have 15k photos (57GB) at the moment. The mac is a macbook pro, the "about this mac" section says the following: OS X 10.9.4
Processor 2.5 GHz Intel Core i5
Memory: 4GB 1600 MHz DDR3
I'm not sure what an SSD is, and I've never tried activity monitor, but I am going to give it a try now and see what I can figure out.
EDIT: The activity monitor has iPhoto at between 24-39% CPU and CPU time is 39:30.76. I have no idea what any of that means but I know its at the top of the list, the next item is the activity monitor itself and its only 1.7% CPU. This is after iPhoto has started up and loaded completely (it is obviously much worse while starting).