Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

'Digital Video Editing' Magazine Slams Apple and G5

X

Xnet

macrumors member
Original poster
Jun 26, 2003
40
0
I am very disappointed in the fact that Video Editing Magazine picked up and is running with the idea that Apples SPEC scores are false by comparing them to the ones Intel or AMD have posted on the SPEC web site.

http://www.digitalvideoediting.com/...com/2003/06_jun/editorials/cw_editorial79.htm

The author of the article clearly does not understand how SPEC scores are achieved nor the effect that compliers have on the scores.

I dropped him the following note I think that other Mac users who understand SPEC scores should do the same.

Remember if you do, please keep it polite as insults will only make him more anti-Apple.

Mr. Charlie White,
I feel that I should point out an error in the table calculations at the end of your article.

B/c SPEC scores are 100% dependant on the compiler used the data from Apple and AMD cannot be compared.

AMD and Intel use the ICC compiler whereas Apple used GCC for its test as it is the only cross platform compiler. Its is a UNIX compiler that is not native to the PPC.

As you might recall, Apple ran the test on Linux on the Xeon, so GCC was the best compiler to use and disabling Hyperthreading actually increased the Xeon performance per DELL
http://www.dell.com/us/en/esg/topics/power_ps3q02-khalid.htm

IBM has run some SPEC on its processors using another complier not mentioned (but I suspect it was Visual Age) and it scores the following at 1.8 GHz

Submitter: IBM
CPU: PPC 970 1.8 GHz
Compiler: Visual Age?
SPECint Base: 937
SPECfp Base: 1051

But remember we are still not using the same compiler here either, note how the compiler affects the score.

I feel that a correction should be made as the data analysis is misleading and incorrect.

---
what does everyone think?
 
i dont know much about this kind of stuff, but i give your props for sending that note.
i cant stand when people write articles for mass publication and do not even bother to get their facts correct.
 
whoops really sorry about not posting the link, here it is...

http://www.digitalvideoediting.com/...com/2003/06_jun/editorials/cw_editorial79.htm

also sent in this note to him from info gather from another user.

Mr. Charlie White

Just to clarify my previous e-mail to you, this is the information from Jeffrey Reilly at SPEC who assured me that the only thing that can add optimizations for processors from the SPEC CPU 2000 is the compiler.

"The SPEC CPU benchmarks were designed to provide a means of comparing
different computer architectures on a fixed and known workload. As such,
SPEC CPU 2000 is provided as source code (since the same executable
will not work on all possible computer systems i.e. - Sun Workstations
can not run the same executables as an IBM workstation) and requires the person
benchmarking to compile the code. Since it is source code, SPEC does not
include any processor specific instructions in the CPU2000.

This means that for any platform specific instructions to be generated it
needs to be done by the compiler (and is not part of what SPEC provides).
This means the answer to your question is: No, SPEC has not done anything
to optimize the code for Alivec (and it is equally valid to say that SPEC
has not done anything to optimize the code for things like SSE or SSE2
instructions).

This means that SPEC CPU2000 is a measure of the processor, memory
hierarchy and compiler used in the test. (Something that will be true for
just about any comparison that is attempted across architectures).

Jeff
Associate Editor, http://www.spec.org
Chair, SPEC CPU Subcommittee"
 
Comparing the G5 to the Boxx does not make sense to me. The G5 is NOT marketed as a workstation-class computer, whereas the box is. The Boxx is very obviously in a different league of computing than the G5, as evidenced by the fact that getting a price on one requires a custom quote. I assume they cost quite a bit more than the G5 as well.
 
I noticed that people who slammed G5 in recent days after WWDC are either trying to troll, trying to push an agenda or simply hate macs.

The question is, Do most of us give a flying F**k whether Operton is faster than G5? The answer is an emphatic "NO". Operton hasn't even been tested in WinXP with 64-bit applications, where Adobe apps are fine tuned and the rest of the stuff is backwards compatible being able to run natively. I'd like to see the poor guys who decide to put together an Operton Dual chip workstation. Just the price of all that labor, drivers, AMD's <sarcasm>unsurpassed stability and silent cooling</sarcasm>, plus WindowsXP are just not worth the hassle when there is a ready solution which has been thourougly tested and retested. GHZ is irrelevant just as it was a month ago. The IBM and Apple partnership and cutting new technology is what I care about, knowing Macs aren't stuck in the old ages with shi*ty MOTO chips. When WindowsXP beats OSX in terms of increased user productivity, then maybe I'll have a look at Opertons or Xeons.

The whole comparison with Xeons during the WWDC was done for advertising purposes mostly. I don't care whether there is some chip that beats the G5 by 10% or 20%. They don't run OSX, and they don't have solutions like FCP4 and Shake3 at dirt cheap prices when compared to their counterparts on Windows workstations.

In regards to the article, screw that bald idiot. When he decides to hire someone older than 12y/o to design his site, I might actually want to listen to his incessant ramblings.

In other news: Some guy on the internet said Apple Sucks. Lets go debate it! :rolleyes:

I say sour grapes.
 
Charles White and this site are known Mac Bashers. He's the one that posted the Dell 3Ghz crushing a Dual 1.25Ghz Powermac(which was shortly replaced by the dual 1.42Ghz)

I'm tired of reading his crap. EVERY marketing team fudges the number. Personally these feckin' geeks need to shutup and wait for the hardware to ship. Trying to beat on your chest because of numbers on a paper is simply stupid.

What you'll find is that the G5 Dual is going to beat PCs on lots of tasks and lose on lots of tasks. That's computing. No one has an absolute advantage.

The poster above is correct...only the Haters are really making a big deal out of this.
 
I don't care what this guy has to say about the SPEC tests, I am buying a G5 becuase I use the best editing software out there FCP and it doesn't run on anything but a Mac. All the editing programs that run on PCs suck and are not very intuitive, and as far as an avid machine goes, it might be better than a G5 in performance, but it costs 10 times more.

This guy needs suck it up, so what if the SPEC scores aren't 100% accurate or twisted or whatever else, they are benchmarks and are always going to be different in any circumstance.

The bottom line, the new G5 sets the standard for Prosumer video editing machines, it is a great price for a great machine.
 
I'm tired of this BOXX machine

Why compare the G5 to the 3DBOXX. The high end model (who the editor in question is so proud to announce is 34 to 57 percent faster (on SPEC, not real world), comes in at 5,230 with specs close to the G5 (DVD-R and other things).

Clearly the BOXX is in quite another league (namly the workstation league). It's all a different duck...
 
Re: I'm tired of this BOXX machine

Originally posted by crassusad44
... (who the editor in question is so proud to announce is 34 to 57 percent faster (on SPEC, not real world), comes in at 5,230 with specs close to the G5 (DVD-R and other things).
Click to expand...

The problem is that you cant compare SPEC scores unless the same compiler is used.

This guy took two scores computed with different compliers and came to a flawed conclusion b/c he doesnt know what SPEC scores do.

He should correct his article...it is wrong and can be proven so with factual evidence not from Apple. (see first post)
 
Re: Re: I'm tired of this BOXX machine

Originally posted by Xnet
The problem is that you cant compare SPEC scores unless the same compiler is used.
Click to expand...

I agree with you, but the intention of benchmarking is also to see how good the cpu can be, so it's also important to use highly optimised, real world compilers that bring out the best in a system.

What's got up everyone's backside is that Apple is claiming to have the world's fastest personal computer, which isn't a new claim, but this time everyone is taking notice.

Now they are looking for anomilies in the documentation that will fit their theory that Apple must have cheated .

What people don't credit Apple for is that this time all their bakeoffs have been properly documented. This means firstly that anyone can make the same experiment to verify that the figures are true. Secondly it means that people can extent the benchmarking picture to include other configurations.

But this is all good for Apple. This controversy will help sell G5s.
 
You'd figure anything but a brain damaged grunt working at an AV Magazine would know...

It's not the SPEC score that matters, it's the speed of the I/O behind the CPU that matters for his readers.

And that it's not the MHz and/or GHz that matter when you talk about an AV Workstation -- it's the MB/s and GB/s that matter.
 
This BOXX thingy, does it run Windows XP? this is silly, no normal consumer cum pro is going to buy one anyway if its not even marketed as a PC. If this magazine are going to take a workstation processor and say its quicker, i might as well state that the IBM PPC POWER 4 is faaaaaaaar quicker stilL!! people should realise that there is a line between workstation, server and desktop PC.
Well in anycase, I know there are servers that can use more than 2 Xeon processors, will Apple have a clear server G5 strategy with a multiple (over 2) G5
based server? any ideas?
Anyway who gives a toss, I'd rather my 500mhz G3 iBook than any multiple P4, 5, 6 Xeon clawhammer prescott or whatever anytime!:D
 
whenever you are reading Charles White's articles, put these lines in your head:

#define Charles_White Bone_Head
#define fast slow
#define slow fast
#define good bad
#define bad good

then his article will seem about right.
 
Re: Re: I'm tired of this BOXX machine

Originally posted by Xnet
The problem is that you cant compare SPEC scores unless the same compiler is used.
Click to expand...

If you are after pure hardware capability, this is true.

However:

1) You will NEVER get the same compiler on two different platforms. It is just not possible. gcc/i86 and gcc/ppc, while sharing a lot of higher-level code, have radically different code generation patterns. There is no cross-platform equivalent compiler.

2) If 90% of OS X software is compiled using GCC and 90% of Windows software is compiled using MSVC (which itself performs worse than Intel's compiler but better than gcc by a fairly wide margin), then your REAL WORLD APPLICATION performance, to the extent that SPEC is measuring such, will best be affected running the SPEC tests compiled on those compilers. High-end software buyers tend to be willing to spend the bucks for the best platform compiler for their custom code, which s why SPEC scores are routinely submitted with the "best" compiler available.

3) There are some quite convincing arguments that Intel's compiler does some special-casing of the SPEC routines, meaning that its results are unduly inflated. This type of skulduggery is a bit more difficult (though not impossible) to orchestrate in an open-source compiler such as gcc.

4) Given the relative age of the gcc/ppc compiler, it is nowhere near as platform-optimized as ANY of the Intel (gcc, MSVC, Intel's own) compilers. At the very least, we can imagine that the gcc compiler released with Panther-final will perform better than the current gcc 3.3 compiler.

So, yes, there is a lot more to the issue than either side is talking about. But, at the end of the day, SPEC scores give you a very isolated and narrow view of machine performance. What matters is how it performs on software you use most, and of course how well the platform's software fits your work flow.
 
re: Charlie White

Ya know, I spent some time emailing this guy, and in his responses he sounded fairly non-rabid. I believed him when he told me that he doesn't hate Macs. In fact, he loves OS X. It's slow processors he said he hates. Fine.

But now that Apple has put out the best thing they have in years, you'd think this guy would have something positive to say. Whether or not these new Macs are faster than the Opteron or P4 Xeon, the point is that they've *finally* made a much-needed performance leap. They are at least closer to the x86 world, if not past it. You'd think the man could at least acknowledge the former point.

I will write him and then possibly unsubscribe to DMN's newsletters. That at least should get his attention.
 
BTW - compiler has a reasonable effect on scores. how do i know?
I programme myself and when i recompiled my app under GCC 3.3 i got a 10% speed boost in the app. pretty good for a minor compiler update. Also thid should allow them to run on panter even with me owning it. very important.
 
Well - I've just sent him this response...


Hi there, Mr. White!

I've just finished reading your very interesting article entitled "Apple's Prevaricating Pinocchios of PR ". And where I can understand your perspective on the argument (and can agree on many parts - i.e. the style and hype associated with the benchtesting involved, and the not-so credible use of the word "incredible"), there are certain things that should be addressed regarding the new G5.

Firstly, this is a PERSONAL computer - the start of a new breed of affordable supercomputers that can do tasks once limited only on high-end machines. It's 12 x FASTER than it's predecessor - an amazing jump in processor ability, with a 1GHz Bus - and it's only in it's first incarnation. Mr. Jobs happened to mention that Apple were expecting 3GHz Dual-processor-equipped G5's to be shipping by the end of the 4th Quarter of 2003 during his keynote speech at the WWDC, so it seems now that the Megahertz myth has truly been dispelled.

Also, considering that a computer is only as good as the software written for it, (performance wise) the new G5 would be a very strong contender for raw processing power. After all - is it not true that the PowerPC chip still utilises a RISC architecture over the Intel/AMD CISC style (therefore, fewer computations to get to the same answer - which is why the G4 was, in many cases, on a par with faster rated PCs in terms of chip speed...). And as you would know - software optimization is a major key in the performance of the CPU. As Adobe, for example, have coded their software to take full advantage of the Altivec technology embedded in the processor, you will see that there is a significant speed increase when using programs such as Photoshop, in comparison to PCs running the same version.

And Finally, bear in mind that OS X.3 (codenamed "Panther") has yet to be released, offering a hearty jump in performance levels (some say over 30%!).

I'm not completely computer-literate. However, I can see the advantages of buying a new Power Mac G5 for £2500 over a Boxx Technologies' system with the same configuration (albeit no software, less hard drive space, smaller dual-processors and a smaller graphics card) for over £1000 more - which does, ostensibly, do the same thing (and doesn't look quite as good - or run any of my programs!).

Still - I suppose it's all based on what system you're used to. But when you want "bang for your buck", it looks very good on paper. I've had Macs for nearly 20 years now (4 in all) - and I can say truthfully that Apple have the best build-quality I have ever seen. My first Mac - purchased in 1984 - STILL boots up without a hitch. Unfortunately, I cannot say the same for many of my friends and family that decided to buy PC-based systems - many of whom have to update every few years; not due to obsolescence, but to system failures.

Anyway, thanks for a thought-provoking article. It's generally nice to get criticism - it usually makes the recipient double their efforts. So, on the basis of this story, just imagine what Apple will churn out NEXT year... Perhaps a dual-processor G5 17" laptop?!?
Click to expand...

It'd be interesting to see if he replies...

:)
 
Originally posted by kiwi_the_iwik
Well - I've just sent him this response...



It'd be interesting to see if he replies...

:)
Click to expand...
Your statement about Jobs saying that there will be a dual 3GHz G5 Macs shipping by the end of 4th quarter Q3 is false and may damage the credibility of your message.
 
Re: Re: I'm tired of this BOXX machine

Originally posted by Xnet
The problem is that you cant compare SPEC scores unless the same compiler is used.
Click to expand...

I disagree! Why so? It is still very interesting for a lot of people.
I have written code (scientific/numerics, similar to that in the SPECfp bench) and I am interested in what platform/compiler might solve my problem best. The vendors' SPEC results give me a very good clue (at least they did in the past). In some cases, the SPEC benchs are themselves a real-world benchmark.

There is no "same compiler" for different processor architectures. gcc on Intel and PPC are different pieces of software, and it is hard to tell in how far they are "alike" in terms of code optimization. And to me, it's not even relevant. If Intel's compilers are superior, this is an advantage of the platform.

If I chose Intel/AMD, I'd look for the best compiler and try out some settings to get the best results (without touching the code!). If I chose Apple's G5, I'd do the same. If I ended up with CodeWarrior or some IBM-compiler, that's fine with me.

To sum it up, I do compare each vendors' own SPEC-results, as long as they are independently confirmed or reproducible.

And I will always prefer a Mac. :)
 
Your statement about Jobs saying that there will be a dual 3GHz G5 Macs shipping by the end of 4th quarter Q3 is false and may damage the credibility of your message
Click to expand...

If I remember correctly, Steve Jobs actually said during the keynote that Apple and IBM expect to be shipping a 3 ghz 970 within 12 months.
 
Re: Re: Re: I'm tired of this BOXX machine

Originally posted by leo
I disagree! Why so? It is still very interesting for a lot of people....
Click to expand...

leo,

The problem with Mr. White's article is that he extrapolates speed scores based on two incongruent sets of SPEC data.

The VeriTest SPEC scores are lower than normal b/c it was not about which complier was faster but which hardware was. Maybe the test is not 100% perfect (after all cross platform benchmarks never are) but its better than comparing GCC to ICC compliers. It is not possible to take compare SPEC data done with different compliers and expect to get a true speed comparison.

If Mr. White believes like you do that the most optimized SPEC scores should be used then the scores he shows are not the ones to use. Use the IBM SPEC scores for the PPC 970 which are a heck of a lot higher.
 
The sooner people learn that benchmark is a synonym for bullsh*t the better off we'll be. ;)
 
Steve and the 3Ghz G5

I'm not sure about the precise timeframe, but Mr Jobs did indeed say they'd have a 3Ghz G5 shortly. I came away from it with the impression it would be here within the year as well, but I suppose the devil is in the details. The truth would be easy enough to ascertain, as the feed is still online.

-Chilton
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back