Speed tests show 1 Ghz DDR faster than 1 Ghz pre-DDR [Archive]

MacBandit

Sep 29, 2002, 12:40 PM

I own a Dual/Ghz/DDR and I do love the thing especially having a machine that is as fast as the previous top of the line for a lot less money. I do believe that it is faster then the previous duals in one situation that is multitasking several programs at once. The more programs you're running on this machine the more it shines because it just doesn't slow down.

That said the tests on the railhead website have a flaw. The tests a) are not multiprocessor aware b) are not altivec aware. This leads to a problem the old G4 has a larger L3 cache which doesn't really come in to play unless you are doing altivec aware tasks. So in my opinion these tests have basically illuminated the L3 cache from the test. That being the case you would expect the new machine to shine it has a faster bus then the previous machines. Which does make a big difference.

Something else that I haven't really been shown is the video performance comparisons between the two machines. This is where I expect the new DDR machines to really shine. The new machines with DMA (Direct Memory Access) directly to the DDR from the AGP and PCI bus should see a boost. If the video drivers were written to take advantage of it the cards should be able to cache directly to the system memory with out much speed loss over there own onboard memory. This is why I don't think there is any speed loss with the new G4MX with only 32MB of ram as compared to the previous model with 64MB of ram. It doesn't need it because it can use the system memory at the same speed as it's own onboard memory.

barkmonster

Sep 29, 2002, 01:44 PM

for a non altivec, non MP aware test this really seems flawed as any kind of measurement.

So a 933Mhz G4 with PC133, a 2Mb L3, 256K L2 & a 7 stage pipeline, without the aid of altivec has score that makes it 273% faster than a 300Mhz G3 ?

The 733Mhz G4 score is laughable to say the least, it's not even seperating the QS from the digital audio models and there's no way it's 63% faster than a 533Mhz G4 at anything. They both have a 133Mhz FSB and PC133 RAM, 1 has a 4 stage pipeline and 1Mb L2 at a 2:1 ratio, the other has a 7 stage pipeline, a 256K L2 at a 1:1 ratio and a 1Mb L3 at a 3:1 ratio and that cpu didn't use DDR on the L3 either because it was the PPC7450 not the PPC7455.

MacBandit

Sep 29, 2002, 03:36 PM

Originally posted by barkmonster
for a non altivec, non MP aware test this really seems flawed as any kind of measurement.

So a 933Mhz G4 with PC133, a 2Mb L3, 256K L2 & a 7 stage pipeline, without the aid of altivec has score that makes it 273% faster than a 300Mhz G3 ?

The 733Mhz G4 score is laughable to say the least, it's not even seperating the QS from the digital audio models and there's no way it's 63% faster than a 533Mhz G4 at anything. They both have a 133Mhz FSB and PC133 RAM, 1 has a 4 stage pipeline and 1Mb L2 at a 2:1 ratio, the other has a 7 stage pipeline, a 256K L2 at a 1:1 ratio and a 1Mb L3 at a 3:1 ratio and that cpu didn't use DDR on the L3 either because it was the PPC7450 not the PPC7455.

The speed difference between the 933Mhz G4 and 300Mhz G3 seems about right if you aren't using Altivec. Also on the speed difference between the 733Mhz G4 and 533Mhz G4 is also very possible correct. You have to take into account that there are several generations of the G4. So it is very possible that it is actually 63% faster. Baically they could be running the same Mhz but because on chip is a later generation some improvements it would still be faster. All in all I think these tests are fairly good at testing all kinds of chips though that's all it does. It really doesn't give you a view of overall system speed because of the missing tests for Altivec and such.

BongHits

Sep 29, 2002, 03:55 PM

Originally posted by barkmonster
So a 933Mhz G4 with PC133, a 2Mb L3, 256K L2 & a 7 stage pipeline, without the aid of altivec has score that makes it 273% faster than a 300Mhz G3 ?

i can put money on it that my 933 would smoke a 300 mhz G3 in anything, including non altivec tasks. One it is a later gen chip, meaning it has more technological advances...it has an L3 cache...and none of this even mentions the fact that the mhz rating is over 300% of the 300mhz G3 (100% denoting equivalency)...if my 933 is only 273% faster than a 300 mhz...im very disappointed

MacBandit

Sep 29, 2002, 04:03 PM

Originally posted by BongHits

i can put money on it that my 933 would smoke a 300 mhz G3 in anything, including non altivec tasks. One it is a later gen chip, meaning it has more technological advances...it has an L3 cache...and none of this even mentions the fact that the mhz rating is over 300% of the 300mhz G3 (100% denoting equivalency)...if my 933 is only 273% faster than a 300 mhz...im very disappointed

Sorry but a 3x faster chip doesn't mean 3x faster performance. There are other bottlenecks such as hard drive speed and ram speed.

P-Worm

Sep 29, 2002, 04:13 PM

Poor Quadra... :(

P-Worm

MacBandit

Sep 29, 2002, 04:16 PM

Originally posted by P-Worm
Poor Quadra... :(

P-Worm

Remember when you would have cut your arm off to have one. :D

SilvorX

Sep 29, 2002, 04:57 PM

Single 933MHz G4 PM
452.1

Single 867MHz G4 PM
420.1

Dual 867Mhz G4 PM
418.1

Dual 867Mhz DDR G4 PM
417.9

uh that doesnt sound too promising concidering the single 867 is faster than the dual ddr 867 :(

MacBandit

Sep 29, 2002, 05:08 PM

Originally posted by SilvorX

uh that doesnt sound too promising concidering the single 867 is faster than the dual ddr 867 :(

By 2.2 that could be the difference in the organization of the drive. Nothing to base a judgement on.

MacBandit

Sep 29, 2002, 05:12 PM

Single 933MHz G4 PM
452.1

Single 867MHz G4 PM
420.1

Dual 867Mhz G4 PM
418.1

Dual 867Mhz DDR G4 PM
417.9

I don't know why it shows a Dual 867 DDR and a Dual 867. Apple never made a Dual 867 before the DDR version that is out now. They made a single processor 867 and a Dual 800.

ddtlm

Sep 29, 2002, 05:49 PM

MacBandit:

That said the tests on the railhead website have a flaw. The tests a) are not multiprocessor aware b) are not altivec aware. This leads to a problem the old G4 has a larger L3 cache which doesn't really come in to play unless you are doing altivec aware tasks. So in my opinion these tests have basically illuminated the L3 cache from the test. That being the case you would expect the new machine to shine it has a faster bus then the previous machines. Which does make a big difference.
You are very much wrong. The L3 holds data and instructions and cares not what type of instructions those are. One does not need AltiVec to use L3. It is also not correct to say that AltiVec makes better use of the L3 than straight PPC code would, because AltiVec could very very easily end up working on data set much larger than the L3, at which point the L3 is rendered unimportant and the FSB speed becomes important.

barkmonster:

The 733Mhz G4 score is laughable to say the least, it's not even seperating the QS from the digital audio models and there's no way it's 63% faster than a 533Mhz G4 at anything. They both have a 133Mhz FSB and PC133 RAM, 1 has a 4 stage pipeline and 1Mb L2 at a 2:1 ratio, the other has a 7 stage pipeline, a 256K L2 at a 1:1 ratio and a 1Mb L3 at a 3:1 ratio and that cpu didn't use DDR on the L3 either because it was the PPC7450 not the PPC7455.
You are very wrong as well. While the longer pipeline CAN hurt performance, there are many many types of programs in which it does not (note how fast P4s can be in the "right" test). There are many things that I can think of computing that would show perfect scaling with clockspeed, and there are certainly things I can think of (as far as instruction mixes) that would show a 745x being far superior to the old G4, even clock-for-clock. The point is that I can pick a test to show the different versions of the G4 however I like, I just need to exploit the strengths and weaknesses of each. Discounting this recenct becnhmark as BS (or whatever) on grounds of it not agreeing with other benckmarks is stupid. This is a real test as much as all the Photoshop tests (or whatever you have in mind that shows the 533 in a more favorable light). The whole point in having a lot of benchmarks to look at is so you can get a better picture of actual performance. One test does not reveal all. Different chips are strong in different areas.

MacBandit:

Sorry but a 3x faster chip doesn't mean 3x faster performance. There are other bottlenecks such as hard drive speed and ram speed.
I assume that you realize that there are things with will scale perfectly with clock speed, and there are things that will scale perfectly with FSB speed, and there are things that will scale perfectly with hard disk speed. Most things are a combination of all three, but making a blanket statment as you have is plainly wrong.

SilvorX:

uh that doesnt sound too promising concidering the single 867 is faster than the dual ddr 867
Well the single 867 has 2mb L3 whereas the dual has only 1mb per chip, and the FSB is the same. Remember earlier in this post where I scolded MacBandit about the L3 being useful in non-AltiVec? Well this may be an example.

ddtlm

Sep 29, 2002, 06:12 PM

I was examing the results a bit more closely and I think that there are some "issues" that need to be addressed:

Just strange:
iMac G4-800 > eMac G4-800
iMac G4-700 < eMac G4-700
eMac G4-700 ~= eMac G4-800

733 vs 800:
9.1% higher clock => 18.6% higher performance
(L3 size issue?)

Scaling with 133FSB 2mb L3:
800 DP => 866 SP = 66mhz for 11.0 points
866 SP => 933 SP = 66mhz for 32.0 points
933 SP => 1000 DP = 66mhz for 15.8 points
(Note that going between SP/DP is neither good nor bad.)

Anyway, the new G4 core is clearly very good at this. Note the 550mhz PB compared to the 533mhz DP.

MacBandit

Sep 29, 2002, 06:22 PM

Originally posted by ddtlm

Thanks for pointing this stuff out. Though I did know this already. My point on Altivec is it seems to be a bandwith hog so it excels when it can use the L3 cache because of the very high bandwith there. I wasn't saying the L3 Cache was not being used at all it's just not being used to fullest.

I also realize that some tests will show that a chip with twice the Mhz is actually 10x as fast this is known. Overall though most of the time when you analyze a complete system the speed does not go up equally with the Mhz speed of the cpu.

I don't disregard these tests at all in that they give you a very good break down of how your system might perform doing the tasks it was tested in. That's all a benchmark tells you anyhow. It just tells you what your system can do with those specific tests it has very little to do with real life and real applications most of the time.

madamimadam

Sep 29, 2002, 07:03 PM

Originally posted by SilvorX

uh that doesnt sound too promising concidering the single 867 is faster than the dual ddr 867 :(

According to what people have been saying, though, this is a single processor test only so a single 867 with 2MB L3 is going to perform better than one processor in a DP system with 1MB L3 Cache.

madamimadam

Sep 29, 2002, 07:09 PM

Originally posted by ddtlm
I was examing the results a bit more closely and I think that there are some "issues" that need to be addressed:

Just strange:
iMac G4-800 > eMac G4-800
iMac G4-700 < eMac G4-700
eMac G4-700 ~= eMac G4-800

733 vs 800:
9.1% higher clock => 18.6% higher performance
(L3 size issue?)

Scaling with 133FSB 2mb L3:
800 DP => 866 SP = 66mhz for 11.0 points
866 SP => 933 SP = 66mhz for 32.0 points
933 SP => 1000 DP = 66mhz for 15.8 points
(Note that going between SP/DP is neither good nor bad.)

Anyway, the new G4 core is clearly very good at this. Note the 550mhz PB compared to the 533mhz DP.

The problem with tests like this is that they are made to give an example of where the machines are at. The only way to make it accurate would be to factor in a few statistical equasions.

I do not study statistics to go into too much detail but to get a close to accurate study it is necessary to run your results through equasions to factor out the errors in life.

For example, it is possible that only certain people will send in their scores. It is necessary to factor in that people with "faster" machines are MORE likely only to send in good results where as people with machines that are considered to be slower are MORE likely to send in any result.

You also have to factor in the fact that newer machines or more newly formatted are MORE likely to be more effecient where as machines that have not been formatted or partitioned are likely to perform less effeciently and show off a score less than maximum.

It is a VERY complex process and the results should only be used as a guide.

Catfish_Man

Sep 29, 2002, 08:01 PM

...*could* actually hurt performance under certain circumstances. If the system overhead is negligable and is the only thing being passed off to the second processor, then three things could hurt performance: the benchmark task swapping back and forth between processors (unlikely if it's high enough priority), extra bus bandwidth eaten up by second processor, extra time needed to snoop in the other processor's cache. You'd have to work pretty hard, but this would result in single processor machines beating/equaling duals. In real life (or even most benchmarks), of course, the duals would almost invariably win.

ddtlm

Sep 29, 2002, 11:21 PM

Catfish_Man:

I think the SMP overhead is actually high enough that single-CPU machines are better at single-CPU tasks, although it's not something I worry about much (small effect).

Bear

Sep 30, 2002, 08:43 AM

Having both an iMac G4 and a TiBook 800, I have seen noticable speed differences between the two.

Any disk intensive activity tends to be faster on the iMac which has a faster disk.

Any CPU bound processes tend to be faster on the TiBook which has the L3 cache.

There are any number of factors (some of which have been stated above) that can cause speed differences. And for a chart to be useful, these differences need to be listed as well.

For reference the TiBook is at 10.2 and the iMac at 10.1.5, I have a couple of pieces of software that broke on 10.2 and am awaiting updates for them before upgrading the iMac.

Falleron

Sep 30, 2002, 10:29 AM

Sorry, I dont agree at all! All of the articles I read say that the old 1Ghz DP powermac is quicker than the current dual 1ghz systems. The reason, the extra cache in the old systems. Sorry to all those people with the new DDRAM system! :-)

gopher

Sep 30, 2002, 10:45 AM

Do you believe just anything that's written? Unlike Barefeets, these tests were compiled on several Macs and averaged out. Barefeets relied on ONE third hand report. And then others quoted Barefeets like it was gospel. Law of averages and law of means. If you are going to rely on reports verify the source has multiple tests to confirm results for more believable statistics.

Sure you don't have an L3 cache, but as someone else observed there is a faster bus. Everything will tend to balance out in the end.

Regardless of the constraints on the bus, it is faster, and so is the RAM. Run your own tests and come out with your own results, and publish them.

Falleron

Sep 30, 2002, 11:15 AM

There is obviously not much in terms of speed differnce. However, in the benchmarks I have seen in various places around the web, the older model is quicker. One place I saw the benchmarks were for Macworld Magazine to name just one. I dont believe everything I read, just ones that are backed up over in many different independant sources.

Jimong5

Sep 30, 2002, 12:04 PM

cr2sh

Sep 30, 2002, 12:55 PM

Originally posted by madamimadam
The problem with tests like this is that they are made to give an example of where the machines are at. The only way to make it accurate would be to factor in a few statistical equasions.
I do not study statistics to go into too much detail but to get a close to accurate study it is necessary to run your results through equasions to factor out the errors in life.

I don't trust these guys' data - some of it just seems contradictory, and while they know little about cpus I think though know even less about statistical errors. I love the idea of 'factoring out the errors in life.' The random errors I think you're talking about (while you can't discount) should in a test of like this be very minimal. Random errors occur from human inability to reproduce results among other things, the distribution will be strictly normal and an equally weighted least squares adjustment of the data will yield simple average, my point is though - there shouldn't be random errors. These are calculations performed based on code that doesn't change. The same machine should produce the same results in the same test under the same conditions. If it doesn't than there's a flaw in the design of the experiment (hard drive being accessed, other applications, or complications). There will always be outliers, I understand that, but this should be really freaking consistent.
You also mention user bias but if these guys' data is worth ANYTHING at all, than they should have checked that **** at the door.
As far as systematic errors, sure, the chip frequency could be buffed up a little or could be running a little hot. There are discrepancies in chip manufacturing, each chip is different, but under the same test it should result the same. These can be modeled though again by taking a larger sample of the same machines, the distribution of errors for these machines I have to believe is going to be minimal. That's an entire different test though, we wouldn't even be measuring the speed of the computers at that point - we'd only be looking at the distribution of errors in speed of the same machine. In effect, measuring Apple's ability to produce the same machine.
I don't like these guys, their experiment designs don't seem fair to all the machines, but that's just my opinion. Anyone else?

On a side note, I went out with this amazing girl last night and I'm just waiting on myself to screw it up... I really like her but jesus, I know I'm going to **** it up. :)

Falleron

Sep 30, 2002, 01:49 PM

Originally posted by Jimong5

you know, who cares if it's a bit slower? the benefits outweigh the losses. you gain More HD bays, another optical bays, and another on board HD Bus. I own a dual 867 and obviously chose it over the old dual gig for these reasons.
Good point. The only reason I was defending the old dual ghz was because I have one! What does it matter, its not like anyone will notice the speed difference.

MacBandit

Sep 30, 2002, 02:01 PM

Originally posted by Falleron
Sorry, I dont agree at all! All of the articles I read say that the old 1Ghz DP powermac is quicker than the current dual 1ghz systems. The reason, the extra cache in the old systems. Sorry to all those people with the new DDRAM system! :-)

Sorry but there are 8 or 9 sites posting benchmarks of the new Dual/Ghz/DDR and they all show the new DDR machine faster at about %70 of the tasks given it. All this for $1,000 less then the old dual new.

This ArsTechnica (http://arstechnica.infopop.net/OpenTopic/page?q=Y&a=tpc&s=50009562&f=8300945231&m=4140912135&p=1) thread is one place where you can see that the new machine is actually quicker. Also from my personal experience when you start mulititasking several high load apps at once the new Dual blows the old one in the dust.

Falleron

Sep 30, 2002, 02:39 PM

Here is an extract from that page.

(Dual 1Ghz - DDRAM)

SMP Results:
# Threads Time Charged Time Score
2 56.9s 82.9s 163.5%

(Dual 1Ghz - NON - DDRAM)

SMP Results:
# Threads Time Charged Time Score
2 50.6s 97.7s 184.0%

Ok, I could well be interpretting the results wrong, but, the final score is higher for the Non-DDRAM (184% to 163%).

Falleron

Sep 30, 2002, 02:58 PM

Here is my output, so, if someone with a DDRAM system wants to post theirs :

Time on my system = 40.4s (quicker than previously stated system 56.9s.

I know the systems are differently configured, + so its not a fair test really.

barkmonster

Sep 30, 2002, 04:07 PM

i can put money on it that my 933 would smoke a 300 mhz G3 in anything, including non altivec tasks. One it is a later gen chip, meaning it has more technological advances...it has an L3 cache...and none of this even mentions the fact that the mhz rating is over 300% of the 300mhz G3 (100% denoting equivalency)...if my 933 is only 273% faster than a 300 mhz...im very disappointed.

I was meaning it terms of faster. By pure clockspeed a 900Mhz CPU is 200% faster than a 300Mhz cpu assuming all factors are equal.

I'm just say they're kind of meaningless because of the fact that there's a lot of benchmarks for the audio app I use and all of them put the 933Mhz G4 and similar models at over 200% faster than my 300Mhz G3. This goes without saying, the hard drive controller is faster, the bus speed is twice as fast, the cpu has a far higher clockspeed and the PPC7455 used in the G4 has a faster, larger cache than the one in mine. Not to mention the PPC74xx has significantly faster floating point performance than the PPC750 aswell. I'm just saying that in realworld terms if a 300Mhz G3 had a score of 100%, a 933Mhz G4 would be more like 310%

The same goes for the PPC7410 vs the PPC7450, on the same motherboard a lot of tests on barefeats came out with the 733Mhz G4 barely scraping past or even being beaten by the 533MHz G4.

Here's a link to a table of results for the audio benchmark I'm refering to.

Protools LE 'Dave C Test' Results (http://www.calculatedrisc.pwp.blueyonder.co.uk/ptbench/index.htm)

This stresses the CPU mainly but a SCSI card that handles the throughput of the data with it's own controller chip combined with a SCSI drive adds a lot to the performance. So does lot's of RAM and extension tweaks. Seeing as protools runs under OS 9 right now the OS doesn't have much overhead on the performance but people with dual CPUs don't get any extra performance out of the second one and nothing in protools is altivec enhanced either.

The sad thing is, we've got all the cool plug-ins and nice easy midi and patchname manangement on the mac, not to mention a great OS (9) and a downright fantastic OS (OS X) but an Athlon XP 1700+ with only ATA drives and 256Mb of RAM is about 50% faster than the fastest tested mac. In this test at least, the 2Ghz+ Northwood Pentium 4s and the more recent Athlon XP 2200+ based PCs are so sickeningly fast compared with the mac it's not even funny.

MacBandit

Sep 30, 2002, 05:30 PM

Originally posted by Falleron
Here is my output, so, if someone with a DDRAM system wants to post theirs :

Time on my system = 40.4s (quicker than previously stated system 56.9s.

I know the systems are differently configured, + so its not a fair test really.

Here are my results.

Time on my system = 40.3s

madamimadam

Sep 30, 2002, 06:58 PM

Originally posted by cr2sh

I don't trust these guys' data - some of it just seems contradictory, and while they know little about cpus I think though know even less about statistical errors. I love the idea of 'factoring out the errors in life.' The random errors I think you're talking about (while you can't discount) should in a test of like this be very minimal. Random errors occur from human inability to reproduce results among other things, the distribution will be strictly normal and an equally weighted least squares adjustment of the data will yield simple average, my point is though - there shouldn't be random errors. These are calculations performed based on code that doesn't change. The same machine should produce the same results in the same test under the same conditions. If it doesn't than there's a flaw in the design of the experiment (hard drive being accessed, other applications, or complications). There will always be outliers, I understand that, but this should be really freaking consistent.
You also mention user bias but if these guys' data is worth ANYTHING at all, than they should have checked that **** at the door.
As far as systematic errors, sure, the chip frequency could be buffed up a little or could be running a little hot. There are discrepancies in chip manufacturing, each chip is different, but under the same test it should result the same. These can be modeled though again by taking a larger sample of the same machines, the distribution of errors for these machines I have to believe is going to be minimal. That's an entire different test though, we wouldn't even be measuring the speed of the computers at that point - we'd only be looking at the distribution of errors in speed of the same machine. In effect, measuring Apple's ability to produce the same machine.
I don't like these guys, their experiment designs don't seem fair to all the machines, but that's just my opinion. Anyone else?

On a side note, I went out with this amazing girl last night and I'm just waiting on myself to screw it up... I really like her but jesus, I know I'm going to **** it up. :)

You are VERY incorrect.... you have to remember that the machines could have been configured differently and, as I pointed out, over time machines slow down if they are not maintained.

It is therefore HIGHLY possible that the DP 1GHZ DDRs in this test performed better than tha DP 1GHZ NO-DDRs because the DDRs are newer and still not heavily influenced by the factors that slow machines down over time.

Also, as I also pointed out, you have to think about the types of people that would be submitting results. Only a certain type of user post results which gives a biased view but then you have to remember that the type of DP 1GHZ user could be VERY difference from the iMac user and they could, therefore, have their machines configured and optimised VERY differently changing the distance between the machines.

And, so, as I mentioned in my earlier post, these results are not in stone, they are a guide and should be used in conjection with tests from other sources.

cr2sh

Sep 30, 2002, 07:20 PM

You are correct that differently configured systems will perform differently and a fragmented harddrive will operate more slowly and effect overall performance of the system (when was that in question?), but what is the point of benchmarking systems then? I have no interest in knowing what Joe Shmow managed to tweak out of his DP1gig (without at least knowing what exactly he did!) If you're going to run serious benchmarking and try to make ANY sense of the results and have it actually mean something - do stock configurations or do exact configurations with the only difference being integral system hardware (cpu type, fsb, mobo specs).

I have NO doubt that if you tweak a single 933 to the gills and freshly defragment the harddrive that you could smoke a dual cpu 1.25 with DDR that the harddrive has been written to a gross amount with no defragmentation and has had some stupid resource configurations. But what is the point!?? That in no way accurately shows the ability of a system, its just smoke and mirrors and complete nonsense.

If my idea of tweaking a dual 1.25ddr system was dropping it down the stairs, my 9600/300 could beat it hands down everytime.. but whats the point? ;)

But I agree with you, these guys have flaws in their results. There is I'm sure a bias in reults and I'm sure even a few submissions were doctored, and the results should be used for only a guide - but I'm not even sure that theyre worth that much guidance all together - I think its a lot of bunk.

madamimadam

Sep 30, 2002, 07:38 PM

Originally posted by cr2sh
You are correct that differently configured systems will perform differently and a fragmented harddrive will operate more slowly and effect overall performance of the system (when was that in question?), but what is the point of benchmarking systems then? I have no interest in knowing what Joe Shmow managed to tweak out of his DP1gig (without at least knowing what exactly he did!) If you're going to run serious benchmarking and try to make ANY sense of the results and have it actually mean something - do stock configurations or do exact configurations with the only difference being integral system hardware (cpu type, fsb, mobo specs).

I have NO doubt that if you tweak a single 933 to the gills and freshly defragment the harddrive that you could smoke a dual cpu 1.25 with DDR that the harddrive has been written to a gross amount with no defragmentation and has had some stupid resource configurations. But what is the point!?? That in no way accurately shows the ability of a system, its just smoke and mirrors and complete nonsense.

If my idea of tweaking a dual 1.25ddr system was dropping it down the stairs, my 9600/300 could beat it hands down everytime.. but whats the point? ;)

But I agree with you, these guys have flaws in their results. There is I'm sure a bias in reults and I'm sure even a few submissions were doctored, and the results should be used for only a guide - but I'm not even sure that theyre worth that much guidance all together - I think its a lot of bunk.

I guess we all just want a rough guide from somewhere since we have no hope in hell of Apple every producing true results.

MacCoaster

Sep 30, 2002, 07:41 PM

Originally posted by gopher
snip... Law of averages and law of means. If you are going to rely on reports verify the source has multiple tests to confirm results for more believable statistics.snip...
Uh. Law of averages and law of means are the same. :p

Averages or more scientifically proper--means--by themselves are worth almost nothing in real life statistics except as a "point" in here and there, but IMHO the median is more significant than mean. I have yet to see a normalized distribution, a linear regression, a standardized distribution, or anything other than just the mean. If everyone used SPEC [http://www.specbench.org/] to benchmark computers, it would make life much simpler in benchmark statistics. Now to find out in a pretty much standard benchmarking suite [SPEC] what σ [standard deviation, in this case] are for all computers measured to figure out what area the Power Macs are in--in percentiles and compare to the rest of the computers tested. Now that'd be a lot more significant.

gopher

Sep 30, 2002, 08:44 PM

Originally posted by MacCoaster

Uh. Law of averages and law of means are the same. :p

Averages or more scientifically proper--means--by themselves are worth almost nothing in real life statistics except as a "point" in here and there, but IMHO the median is more significant than mean. I have yet to see a normalized distribution, a linear regression, a standardized distribution, or anything other than just the mean. If everyone used SPEC [http://www.specbench.org/] to benchmark computers, it would make life much simpler in benchmark statistics. Now to find out in a pretty much standard benchmarking suite [SPEC] what σ [standard deviation, in this case] are for all computers measured to figure out what area the Power Macs are in--in percentiles and compare to the rest of the computers tested. Now that'd be a lot more significant.

Yes I know about that redundancy of means and averages, but I've also heard them spoken in the same sentence before as well in that manner. Though Spec benchmarks are based on tests that assume processes that have zero errors, a non-realistic assumption, and thus don't show what really can happen to code on a Pentium vs. a G4. They unfairly bias the Pentium because its very weakness is its error checking which gets stuck in its more numerous stages. Most all code you are going to run across has some minor errors, and it is up to the processor to decide whether or not they are significant enough to terminate the program. I wouldn't trust a benchmark unless it used realistic code.