Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Hilarious to see the VMware fanboys jumping up and down about how the methods and numbers must be wrong. Kinda like what you'd expect around here in general if an objective benchmark came out showing that Windows/Linux/non-Mac is better than OS X in some task. :rolleyes:

While it would have been nice to test the newest VMware, the authors seem to have a very good reason for not doing so: they took their time and methodically ran a huge set of tests over and over to get the most accurate results. So the latest versions of each were not out yet when they started. Or in other words, those were the latest versions when they started. If they'd switched to a new version every time it came out, it's quite possible that they'd never finish.

Hopefully they re-run their tests with the newer versions, now that the initial results are out.


On the question of how virtualization was able to beat out the real thing, I'm surprised that very few people seemed to understand that this is, indeed, possible. AidenShaw got it:

It looks like the tests that Parallels was better than native were graphics and display intensive (like scrolling text).

I'd suspect that the native drivers are poorer than the OSX drivers (does Apple supply the graphics drivers, or are they from the graphics manufacturer?), and that Parallels has a good implementation for their graphics.

In addition to that, there's also the possibility that either Parallels or OS X (or both) is able to speed up things like disk accesses by doing a better job of caching parts of the virtual hard disk file than XP did of caching the actual hard disk. If OS X preloaded a piece of data into memory and XP did not, the version running under Parallels would still benefit from the OS X caching while the Bootcamp version would not.

And we see that Vista, which reportedly caches much more aggressively, performs much better under Bootcamp than when virtualized. It's probably a combination of this and the 3-d graphics requirements that make Vista run best on bare metal.


Also keep in mind that as the authors point out, this is a limited set of tests designed to simulate certain tasks. Like all benchmark tests, they perform better under certain conditions and worse under others; and they may or may not represent real world usage.
 
In addition to that, there's also the possibility that either Parallels or OS X (or both) is able to speed up things like disk accesses by doing a better job of caching parts of the virtual hard disk file than XP did of caching the actual hard disk. If OS X preloaded a piece of data into memory and XP did not, the version running under Parallels would still benefit from the OS X caching while the Bootcamp version would not.

Anything can be done with the "right" benchmark :cool:
If you know of some specific tasks in which parallels does better, using caching for instance, than xp alone on the all hardware (bootcamp), then you can "build" a benchmark that voluntarily gives an unfair advantage to one of the challengers...

The question is: would you trust such a benchmark? :eek:

I propose one thing. Let's do our own benchmarks! Trial versions of both softwares are available. Many of us own one or both of them. Let us create a "true" benchmark.
Let's put inside something like zipping a huge file (an ubuntu iso) with 7-zip under windows, opening and scrolling to the end a large word file with ms word and a large pdf file with adobe reader, etc.

Or, even better, let's download one of the many windows trustable benchmarks available for free, let's run them on our parallels and vmwares and bootcamps and then let's use this thread to publish the results.

Let's show these guys how benchmarks ARE to be done! ;)
 
It looks like the tests that Parallels was better than native were graphics and display intensive (like scrolling text).
But the graphics card is simulated in Parallels, while in BC, they are just like the real thing (because they actually are).

I'd suspect that the native drivers are poorer than the OSX drivers (does Apple supply the graphics drivers, or are they from the graphics manufacturer?), and that Parallels has a good implementation for their graphics.
Except that the Parallels drivers made by Parallels. Which really aren't that great. The BC drivers are Apple's, which are actually pretty good (as per PC World's comments that they're the fastest PCs), but you can use the manufactures as well. ATI or Nvidia. They should be, and are, faster.

One of the many reasons those of us who actually use them know this can't be correct. ;)

Hilarious to see the VMware fanboys jumping up and down about how the methods and numbers must be wrong.
Yes, of course. We're all just VMW fanboys. Every single one of us. Except for that 1 poster who had a problem with it and now uses Parallels. :rolleyes: Or those of us who've used them are crying foul because in real world experience, they don't match reality. If they did, we wouldn't be complaining. ;)

While it would have been nice to test the newest VMware, the authors seem to have a very good reason for not doing so: they took their time and methodically ran a huge set of tests over and over to get the most accurate results.
But it still doesn't make sense. I've been using Fusion since before it came out, and besides a few minor issues, it's been better than Parallels, which I've also been using since early beta days (as I said, we use it at work, and only still do so because it came out of beta sooner and corp is slow to move to upgrade). Nether of them are better than BC though, even with the older drivers. Something must be wrong with these tests, and we're right to question them. Looking at the vast majority who do, I can't say I blame them since my experience matches theirs.

Hopefully they re-run their tests with the newer versions, now that the initial results are out.
Looking around at other sites that also question their credibility, I doubt it would help, but maybe they could at least do another preliminary one with the current software, at least before trying to defend this one.

On the question of how virtualization was able to beat out the real thing, I'm surprised that very few people seemed to understand that this is, indeed, possible. AidenShaw got it:
Except he was wrong. Parallels drivers and Apple drivers maybe, but not native ones. And emulation is still going to be slower than the real thing. Which it is. Which again, is one of the reasons why we don't trust the tests.

In addition to that, there's also the possibility that either Parallels or OS X (or both) is able to speed up things like disk accesses by doing a better job of caching parts of the virtual hard disk file than XP did of caching the actual hard disk. If OS X preloaded a piece of data into memory and XP did not, the version running under Parallels would still benefit from the OS X caching while the Bootcamp version would not.
I supposed it could, but it doesn't.

Also keep in mind that as the authors point out, this is a limited set of tests designed to simulate certain tasks. Like all benchmark tests, they perform better under certain conditions and worse under others; and they may or may not represent real world usage.
They don't, which is why we're questioning them. :)
 
I personally don't have a problem with the methodology. Arn said he was impressed with it, which is good enough for me.

My beef is with the poor selection of tests, dreadful writeup, and appalling presentation of the results. As I've said, the graphs are almost meaningless. Where is a table, or list of actual measured times?
 
Vt-x, VT-D. Your world will change in 2008.

Funny, So much said for either side. One camp gave up on Parallels back before 3.0 was released(Build 3218 ver 2.5, 4128 :3.0, 4560:3.0, 5160:3.0, 5582 3.0; 5 builds ago in one year) another sides with Fusion 1.x/1.1.

These two companies will be flip flopping you endlessly with development, options and better builds.

Having two companies pull for my money is better than either giving up.

The proof in the pudding will be with VT-D comes out from Apple. Windows machines already have it, called Vpro.

Parallels has it for 4.0 coming this year, Fusion, not sure.

You'll be able to assign a graphics card to Virtual machine and not deal with writing code hash to run your video with "experimental" shaders.

Being able to directly communicate with the hardware and not through the OS shell will greatly improve your experience with 3D and Games.

VT-x? Sorry that means new hardware for you.

Bootcamp? Try running 36 different Guest OS's. Or at least 5 or 6 at a time.

Try carrying Bootcamp on an external Drive, can't do that, Virtual Machines can. You can even run a virtual machine from a USB key (8gb or 16gb preferred). Virtual machines can be portable.

Apple only writes drivers for Vista and XP. No plans for all the 36+ other operating systems of the real world.

So bootcamp for games/3d software. Better yet, get Parallels or Fusion too! They both support Bootcamp and Virtual Machines at the same time.

Buy Both $160.00 because Parallels does run some VM's better than Fusion and vice versa. You paid as much for 2 of your Xbox 360/Wii games and even more for your iPod Touch.


Cheers,:)
 
freejack86 said:
The proof in the pudding will be with VT-D comes out from Apple. Windows machines already have it, called Vpro.
Cute, but VPro is a hardware management system. Virtualization is a requirement (I'm looking at you E4500, E6540) but it's not the cornerstone. Windows has nothing to do with this.

freejack86 said:
VT-x? Sorry that means new hardware for you.
Already implemented. ;)

freejack86 said:
Bootcamp? Try running 36 different Guest OS's. Or at least 5 or 6 at a time.

Try carrying Bootcamp on an external Drive, can't do that, Virtual Machines can. You can even run a virtual machine from a USB key (8gb or 16gb preferred). Virtual machines can be portable.
Not that we don't know this.

I could never get parallels or bootcamp to work.
That's very detailed.
 
bootcamp could never partition it hard drive and parallels always froze when I tried booting XP. VMware was the only thing that worked but I perferred bootcamp.
 
bootcamp could never partition it hard drive and parallels always froze when I tried booting XP. VMware was the only thing that worked but I perferred bootcamp.
Boot Camp does have issues partitioning when the file system lacks a large enough section of contiguous space to split off.

VMWare works so what's the problem?
 
Boot Camp does have issues partitioning when the file system lacks a large enough section of contiguous space to split off.

To expand on this. Boot Camp has to have X Gigabytes of "empty" hard drive space at the end of your disk. Where X is the size you specify for the Boot Camp partition.

However, data is stored somewhat randomly on the disk at a physical level. When the disk is less full it concentrates data in the beginning and middle of the disk, but over time some will get stored toward the end of the disk. Boot Camp will not move this data and thus fails when trying to create the partition because the space is not empty. (Note: The process is anything but "random" in reality, but that's a good way to think about it if you don't want to get into the technical details.)

Two solutions: Format your entire hard drive and then install Boot Camp asap before any data happens to get stored out at the end of your disk.

Alternatively, you can purchase a program like iDefrag to do an offline defrag of your hard drive in "compact" mode to move all your OSX data to the beginning of the disc and leaving the end free and allowing the Boot Camp setup to complete.

P.S. I recently setup Vista under the latest Parallel 3.0 build and used it for a few days. I'm currently installing Vista in the latest Fusion 1.1 build and I'll comment on the two once I've had some time to test Fusion.
 
I have both Bootcamp and Fusion running Windows XP. I am so surprised how fast Fusion does run. I am extremely impressed with the speed. That being said I have not done any benchmarks, nor do I have any concrete evidence that supports my claim. But I have not booted into bootcamp since I installed it. I personally could get away with just fusion. The only negative thing I have noticed with fusion, it takes about 15 seconds longer to load the windows desktop than bootcamp. But that is just the "Tools" loading.
 
Real World Experience with Vista on Parallels and Fusion

Here's my anecdotal evidence, take from it what you will.

First a bit of background to let you make your own assumptions on any biases I may have. I work in the Windows Universe and began counting down the days to carting around a Mac laptop as soon as the MBPs were announced. I started out using Parallels (Fusion wouldn't even be in beta for a while yet) and was amazed at the performance. My old laptop wasn't too shabby, but Parallels still out performed it by a notable margin. -- Largely due to the Mac hardware.

I tried the beta for Fusion, but didn't really get into it. It didn't have the performance Parallels did. So I stuck with Parallels for my production VM and left Fusion for the experimental ones. However, every new build with Fusion seemed to provide notable performance increases - which is to be expected in a new product.

Fusion was the first to support Boot Camp. As soon as that came out I went to that and have been using Fusion running the Boot Camp install of XP ever since. I have continued to notice performance increases as Fusion as evolved.

My point in mentioning performance thus far is to demonstrate that Fusion has made considerable progress since the beginning. While I used Parallels there were similar upgrades, and only assume that they continued once I retired my Parallels VM.

I'll not say much about the much debated benchmark in this thread because most of what's worth saying has already been said. One thing I didn't see explicitly mentioned was that I think it was good practice that they stuck with one build all the way through the test. But one must acknowledge that based on a typical software development time table version 1.0's often have performance issues. So one would expect greater improvements (in the general case) among the initial followup releases to a 1.0 package than a 3.0 package. The reason being that the latter has already had the time to release several versions, each improving on the first. One of advantages of being first to market. Meaning that one would expect "larger" performance increases and fusion as it went from 1.0 to 1.1.1, while one would expect "smaller" performance increases in the various builds of 3.0.

On to my non-scientific review of Vista SP1 Ultimate on the latest builds of Parallels 3.0 (5584) and Fusion 1.1.1 (72241). My MBP has 2.16MHz Core Duo processors, and 2GB of RAM. Both VMs are running off 1GB of RAM and 40GB expanding virtual hard drives.

I will preface this by saying this was NOT a benchmark, just my real world experience when using Vista on the two platforms. I was using OSX at the time for various tasks as well. Obviously conditions in OSX and the two copies of Vista were not identical, but they were comparable as both tests working under my "standard" operating load.

I used Vista Parallels for two days. I spent the first day loading Windows, Office and applying SP1, installing software, etc. I spent the second day doing my day's work on Vista. However, the performance so poor: delays when opening menus and documents, screen refresh rates, choppy scrolling, etc. I that spent some time turning off the majority of the eye candy Vista allows you to. -- I did stay with the Vista (non-Aero) theme though. I'm a fairly impatient person when it comes to general responsiveness (as mentioned above) and I deemed Vista on Parallels as a valuable experimental tool (I'm a sys admin,) but too slow for daily use.

So I setup Vista SP1 Ultimate in Fusion. The only notable configuration difference in hardware and software is that I was using 2 virtual processors in the Fusion setup. I'd like to reiterate this was not a benchmark, but a write up on what I experienced using this software in the "real world," so it may or may not be considered an apples to apples test. So far Fusion has performed much better under Vista than Parallels did. Install times of the various programs and updates, while not timed, seemed to go much faster. The responsiveness of Vista on Fusion is much better. Usable even. Audio is a bit choppy, but I assume that is a driver issue that can be corrected (haven't bothered to look into it as I don't do anything with audio in Windows.) I did notice that OSX did take a larger, but livable, performance hit when the VM was cranking away at full speed, like during boot up. - I attribute this to Fusion running in multi-threaded mode, where as before it would never use more than 50% of the Mac's cpu power. OSX had comparable performance during standard usage of both VMs though.

I have my Boot Camp XP install (that I run through Fusion) stripped down to just about bare bones, and the performance of XP vs Vista is what you would expect. My XP has better responsiveness in general than Vista. Interestingly Outlook 2007 was more responsive in Vista than XP. Vista does run fast enough for me to consider using it for my production VM, a few more days of testing will determine that.

I notice two problems with the Vista VMs. 1) The Netflix streaming movie player would not work on Parallels, the error message said the video card was not compatible. I only spent a few minutes troubleshooting and did not resolve. A problem just with the Netflix Windows Media Player plugin or possible DRM issues with Parallels and Vista? 2) The a fore mentioned sound problem in Fusion. -- I did not experience any stability issues with either, but also haven't tested them long enough to say anything conclusive about there stability.

Some of my non-performance related opinions on Parallels and Fusion:
-I use the Windows VM for work, and OSX for personal use. I like keeping them isolated or sandboxed from each other, I don't want anything more than shared network folders for the occasional file transfer. Fusion is built more towards that end than Parallels.
-Related to the previous item, I always work in full screen (putting Windows on my external display and keeping OSX on the laptop display) or windowed mode. However, I would use the Unity view on occasion if it supported spreading windows across multiple displays. Parallels' Coherence does just that.
-When running in Windowed mode, Parallels has a hardware icons that flash with activity. Fusion's icons are in one of two states, connected/disconnected. I like the activity monitors; one of those "little things."
-I run bridged network connections. When I connect my Mac to different networks Parallels would automatically disconnect/reconnect the VM's adapter, triggering a renewal of the IP address. Fusion does not do this and I have to renew my IP manually (via a simple script I wrote). Another one of those little things.

Conclusion: If I were going to be running Vista, I would only run it on Fusion using 2 virtual processors. I currently use Fusion for running my Boot Camp install of XP, and am quite happy with it. If I had a day to kill I would setup an XP install on Parallels to see if 3.0 holds up against Fusion (I've never ran XP in 3.0.) If Parallels performed well I would probably switch over and disable all of the integration features so I could use the dual screen coherence feature.
 
native rocks the socks out of any emulation

Sure it does. But if you don't spend much time in Windows, and need it only to do a few things, it's nice to have it within a minutes reach, do what you need, switch back over to the Mac side, then switch back. It's beats having to reboot multiple times just to do a few things.

If, however, you do a lot of processor intensive work under Windows, then Boot Camp is your choice.

Fusion gives you the best of both worlds, by allowing you to use your Boot Camp partition and the choice to open a Windows session along side your Mac session.
 
Fusion gives you the best of both worlds, by allowing you to use your Boot Camp partition and the choice to open a Windows session along side your Mac session.

Parallels advertises support for running the boot camp install now too. I've never used it though.
 
real world

I have been using Parallels for approx. 10 mos. In the medical world, EMRs (electronic medical records) are windows only. In addition, I use Quickbooks pro, and this too has to be in Windows. Finally, I have to use Explorer to run payroll for the practice, and this too I can only use in Windows. Parallels worked well enough that I gave away my PC last summer. I was very happy to see it go. When Leopard came out, things were really bad for a few weeks or months with multiple crashes. However, with subsequent Leopard upgrades and Parallels updates, things are now wonderful for me. I never use windows except for the above reasons, and do everything else on the Mac side. I have been more than happy with the experience over the last few mos. Note that I'm not a gamer. In addition, in my group of 12 docs, 4 have purchases macs over the past 6 mos. These are people who have never owned a mac in their lives. I haven't tried VM, but I plan to try it on the MBA my wife just got to see how it compares. If your needs are similar to mine, then I think that Parallels has been a great product, despite the negative vibe it has received on this thread.
 
These benchmarks are very wrong. As soon as I saw them ranking emulated windows over it running directly on the hardware, I discounted everything else they said.
 
These benchmarks are very wrong. As soon as I saw them ranking emulated windows over it running directly on the hardware, I discounted everything else they said.

You have to play close attention to the report, as it is written poorly, because it does not say that the virtualization solutions out performed the tests run in boot camp, or "bare metal."

What it did say is that they out performed their "base line PC," which they made some brief mention to being spec'd similarly. It is quite possible, likely actually, that different hardware with the same "specs" will perform differently.

So this doesn't point out an impossibility in the report (like others have mentioned in this thread,) but a bias/ignorance on how to conduct a sound benchmark.

The way I see it they could be out to intentionally misrepresent the results by introducing irrelevant and misleading stats like their "base line PC" and you should then assume the rest of the tests were designed to give the "right" answer. Or they could just be ignorant of the fundamentals of benchmarking and think the "base line PC" contributes to their findings. Even with pure intentions, you can't conduct an accurate test if you don't understand what you're testing.

In either case, the inclusion of the "base line PC" calls into question the validity of the entire test suite.
 
I tried vmware first since it's already familiar from running on linux - a version they have given away free for years. It worked well enough I never got around to trying out Parallels and just bought the trial version. It's not perfect, like how it gets confused when unplugging dual-screen monitors, or how unity-mode gets all kinds of confused sometimes and you have to kill it, for example, but good enough.

I use two setups, one for 'serious' work that uses both cores and 2GB ram that runs in its own 'space' in full screen mode with dual monitors. The other is mostly for browser testing with just one core and one GB ram that is kept suspended between uses. That setup works really well for me.
 
acray did a better test in my opinion. It is much more representative of real-world usage.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.