What I read is that matte displays distort the image more because they have to put that anti-glare film on the screen. Glossy screens are brighter and more vibrant simply because theres not as many films over the original screen.
I wonder if the whole "matte is more color accurate" arguement is not based on mere word of mouth.