Don't worry about it. The way OS X handles RAM is by leaving as much in memory as possible until it starts running out of free ram. Then it will start dumping inactive ram by oldest unused memory. Only when your page outs start to outnumber the page ins should you be even remotely concerned.
You have nothing to worry about. If the OS constantly dumped inactive RAM to give you more free memory, then if you were to use that memory again, it would have to go back to the hard drive again, which is MUCH slower.
thanks for the responses, at my office now, computer in question is at home. I thought wired memory was constant so I didn't list it and I forgot to check the RAM config.
The reason why I even noticed my RAM was that some videos I was watching and a game I loaded were excessively slow. They were running as if I were out of RAM.