Are you talking about the eye dropper type of tool? In what application? It should be dead on to the pixel you selected. If you have a background that's dithered, has a gradient, or otherwise is not 100% uniform in color, then of course, you'll get whatever that pixel is and it may not be the overall color impression you get from the background.
Otherwise, can you post an example?