Low RAM shouldn't cause a Kernel Panic, though if you can post a log of that (in a new thread would be better) we might be able to help you determine the cause.
If a kernel panic occurs, information is added to a log file in the folder /Library/Logs/DiagnosticReports (Mac OS X v10.6)
Low RAM will cause the OS to page data out to the disk, which is extremely slow compared to using RAM. As long as page outs and swap used are low and stay that way, you probably don't need more RAM, and there would be almost no benefit. For example, I have 6GB of RAM and I currently have 6GB page ins, 3MB page outs, and 3MB swap used, and I rebooted three days ago. I've seen those numbers occasionally go to 100-200MB without significant slowdown. Back when I had 4GB and I'd try to run VMWare Fusion and a bunch or other stuff at the same time I would get a GB or more of page outs and the system would stop responding for a minute or more. I'd say as long as page outs and swap used stay under a few hundred MB you don't need more RAM. You will never get them both to stay at 0, though, and that's OK.
Also, it's normal for a lot of RAM to be "Inactive RAM". According to Apple that means it is used for things like caches of recently closed programs that you might open again. They are kept in RAM that would otherwise go to waste, and the idea is that if something needs the space that RAM can be quickly freed up (It doesn't always get freed up like it's supposed to, but that's another issue). So Activity Monitor might show almost no "free" RAM when in reality there is a lot available.
That said, prices for 8GB of RAM are at an all time low right now, averaging $40 or less. At this price manufacturers are just about giving RAM away. My guess is that prices have just about hit rock bottom and won't go much (or any) lower. At some point prices might start to go back up, but it's hard to say if/when that would happen. With DDR2, IIRC prices never really increased (though I stopped paying attention) but they just replaced it with DDR3, marketed that as the next new thing, and phased DDR2 into obsolescence. For reference, 8GB (2x4gb) of laptop DDR2 costs $100-$150 right now.