Which slots have RAM installed now?
The RAM should be installed in identical pairs, from the center slots outward.
You said it won't boot up.
What does it do? (fans, blinking power LED, boot chime, no video, etc)
The SMU reset button is near the bottom of the logic board, underneath the RAM slots.
You will see a power supply connector to the right of the battery. Immediately to the right of that connector is the reset button.