Originally posted by madamimadam
The problem with tests like this is that they are made to give an example of where the machines are at. The only way to make it accurate would be to factor in a few statistical equasions.
I do not study statistics to go into too much detail but to get a close to accurate study it is necessary to run your results through equasions to factor out the errors in life.
I don't trust these guys' data - some of it just seems contradictory, and while they know little about cpus I think though know even less about statistical errors. I love the idea of 'factoring out the errors in life.' The random errors I think you're talking about (while you can't discount) should in a test of like this be very minimal. Random errors occur from human inability to reproduce results among other things, the distribution will be strictly normal and an equally weighted least squares adjustment of the data will yield simple average, my point is though - there shouldn't be random errors. These are calculations performed based on code that doesn't change. The same machine should produce the same results in the same test under the same conditions. If it doesn't than there's a flaw in the design of the experiment (hard drive being accessed, other applications, or complications). There will always be outliers, I understand that, but this should be really freaking consistent.
You also mention user bias but if these guys' data is worth ANYTHING at all, than they should have checked that **** at the door.
As far as systematic errors, sure, the chip frequency could be buffed up a little or could be running a little hot. There are discrepancies in chip manufacturing, each chip is different, but under the same test it should result the same. These can be modeled though again by taking a larger sample of the same machines, the distribution of errors for these machines I have to believe is going to be minimal. That's an entire different test though, we wouldn't even be measuring the speed of the computers at that point - we'd only be looking at the distribution of errors in speed of the same machine. In effect, measuring Apple's ability to produce the same machine.
I don't like these guys, their experiment designs don't seem fair to all the machines, but that's just my opinion. Anyone else?
On a side note, I went out with this amazing girl last night and I'm just waiting on myself to screw it up... I really like her but jesus, I know I'm going to **** it up.