If it was the RAM - wouldn't you expect Memtest to fail? That's always been the rule of thumb for PC's/Windows (which I'm coming from) so I can't imagine why OSX (FreeBSD/Unix) would be any different.
I always thought this was the case, yes, but I've seen Macs PASS all RAM tests, but be unstable in general usage. I replaced the stock RAM and ALL my issues disappeared.
RAM tests are a good rule of thumb, and like you, I always thought that a good Memtestx86 result was enough, but not on Macs it seems.
As someone suggested, replace the RAM with stock, rebuild the OS and see if you are stable.