OS X Metal vs OpenGL benchmark

[jeanlain]

Now that GFXBench Metal is out on OS X, when can at last test Metal performance and compare it to OpenGL.

Here's my results (13" MacBook air with 1.5 GHz i5, intel HD 6000). I only use offscreen tests since V-Sync is clearly forced on the onscreen openGL tests. I indicate the Metal results first, units are fps except for texturing.

Manathan: 24.7 - 30.5
T-Rex: 73.7 - 85.7
ALU 2: 74.5 - 77.6
Driver overhead 2: 56.8 - 51.8
Texturing: 4992 - 5144

So a clear advantage for OpenGL across the board, except for the driver overhead test.
HOWEVER, it should be stressed that on this weak GPU, all other tests are GPU-bound and Metal is not predicted to bring ANY advantage here. So I think these results just show that the Metal drivers are not mature yet in respect to the GPU code they produce.

I'd be interested to see results from other GPUs.

 

[Janichsan]

Mid 2012 Retina MacBook Pro with Nvidia GT 650M with 1 GB (Metal/OpenGL – all onscreen):
Manhattan 3.1: 20.5* / –
Manhattan: 30.1 / 24.7
T-Rex: 68.1 / 43.7
ALU 2: 43.4 / 29.9
Driver Overhead: 119.7 / 27.2
Texturing: 14491 / 11863
(* It looks actually like a lot less, somewhere in the single digit range.)

So, there are in fact improvements, although to a varying degree: the OpenGL 3.0 feature set based T-Rex test runs significantly faster. The Manhattan test, which is based on an OpenGL 4.x feature set, on the other hand gains much less.

 

[jeanlain]

What about offscreen tests?
The onscreen tests were also in favor of Metal on my Mac, but results were highly suspicious in openGL as they were lower than the offscreen results (this was the contrary on Metal) despite onscreen tests running at lower screen resolution. And several results showed 59.9 or 29.9 fps. If it's not due to V-Sync, I don't know what it is...
That doesn't look to be the case for you though (except for ALU 2, which could be a coincidence).

Also, does it run a full retina resolution? I find strange than my Mac performs better than yours in some tests. The offscreen tests simulate a 1080p display. This is useful to compared between Macs.

If my Mac gets better results than yours, which has a faster CPU, it clearly means that your results are GPU-bound (due to higher screen resolution). In this case I wouldn't expect Metal to bring any improvement. Weird...

 

[imacken]

So, the results on my late 2015 iMac with M395X are:

Onscscreen
Manhattan 3.1: 32.2/-
Manhattan: 50.5/32.6
T Rex: 111.5/59.9
ALU2: 73.1/59.9
Driver Overhead 2: 119.5/59.9
Texturing: 25686/68253 (odd result!)

Offscreen
Manhattan 3.1: 235.7/-
Manhattan: 298.5/162.4
T Rex: 560.7/368.6
ALU2: 688.5/502.3
Driver Overhead 2: 380.7/90.9
Texturing: 67134/50858

Pretty astonishing differences between OpenGL and Metal!

A question though, what resolution do these test run in onscreen?

 

[Janichsan]

What about offscreen tests?

My offscreen results:

Manhattan 3.1: 51.0 / –
Manhattan: 66.8 / 57.8
T-Rex: 136.9 / 149.5
ALU 2: 114.7 / 124.5
Driver Overhead: 248.3 / 28.1
Texturing: 15303 / 15422

So, these are a bit more of a mixed bag, with OpenGL being faster in most cases.

A question though, what resolution do these test run in onscreen?

Also, does it run a full retina resolution?

The info text only says "full native resolution", but I can't say if it's really 2880x1800 or rather 1440x900 with lots of anti-aliasing.

 

[jeanlain]

Given the much lower performance of onscreen tests, I'd say they run at full resolution. It's strange that Metal has the advantage under these conditions. Vertical sync may be messing with the performance in the openGL tests, as it does on my Mac. See the results that imacken got: 3 tests at 59.9 fps. Clearly, onscreen tests cannot be trusted.
So based on offscreen tests, it appears that Metal has the advantage only on the AMD GPU.

 

[Janichsan]

Given the much lower performance of onscreen tests, I'd say they run at full resolution. It's strange that Metal has the advantage under these conditions. Vertical sync may be messing with the performance in the openGL tests, as it does on my Mac. See the results that imacken got: 3 tests at 59.9 fps. Clearly, onscreen tests cannot be trusted.
So based on offscreen tests, it appears that Metal has the advantage only on the AMD GPU.

It's a bit early to draw conclusions based on benchmarks of only three machines, of which one has a mediocre Intel GPU and the other has an outdated Nvidia chip that barely meets the requirements for Metal.

I agree that the apparent vsync limitation (at least on Intel and AMD drivers) makes the onscreen results not particularly reliable, but I wouldn't put too much faith in the offscreen results either: I don't really know what GFXbench does there, but the offscreen numbers do not translate into realistic framerates when rendering actually onscreen at 1080p. The offscreen numbers are almost always invariably higher, so there are other factors at play.

 

[Ferazel]

Cool, I'm glad a proper OS X metal benchmark came out. Here are my results for OFFSCREEN.

11.4 public beta
Mac Pro 4,1
GTX 670 using OS X drivers

Test Name (value): Metal / OpenGL
Manhattan 3.1 (fps): 188.6 / --
Manhattan (fps): 267.9 / 126.5
T-Rex (fps): 596.8 / 491.3
ALU2 (fps): 530.3 / 526.1
Driver Overhead2 (fps): 218.4 / 23.11
Texturing (MTexel/s): 68089 / 67053

In order to have a standard basis of comparison we should all be outputting it for OFFSCREEN so that all of the render target sizes are the same. We can then compare GPUs directly instead of relying on the machine's resolution which could vary widely.

Overall, the performance gains are pretty impressive for these benchmarks. Obviously, ALU and Texturing are GPU limited functions so their gains really wouldn't improve. However, a 20% bump with just the default OS X drivers are pretty good. Really wish we had more tests.

 

[koyoot]

It looks like Metal already almost ties performance between OS X and Windows.

M395X looks to have 600 FPS offscreen for T-Rex on Windows, 47 FPS onscreen for Manhattan benchmark.

My MBP Mid 2012 with GT650M(Non-Retina model, with lower core clocks than the one in Retina model) vs. GT 650M Windows/OpenGL:
T-Rex Offscreen: 107/121 FPS
Manhattan Offscreen: 60/68 FPS
Manhattan 3.1 Offscreen: 37/56 FPS
Offscreen is done in 1080p resolution. Regardless of platform. It looks like Metal pretty much ties with Windows platform currently in OpenGL. However the lack of some of features is becoming apparent in benchmarks that need them, vide the Manhattan 3.1 Benchmark.

The question now: how easy will be to port games from Windows to Metal?

 

[1madman1]

First OSX app I've encountered that's caused my Mac Pro (5,1) to shutdown from excessive power draw! Probably didn't help that I had half a dozen other things open at the same time.

Plenty of synthetic GPU benchmarks (though no actual games) shut it off in Windows. This is a consequence of using 6pin > 8pin PCI-E power adapters!

 

[hackintosh5]

Cool, I'm glad a proper OS X metal benchmark came out. Here are my results for OFFSCREEN.

11.4 public beta
Mac Pro 4,1
GTX 670 using OS X drivers

Test Name (value): Metal / OpenGL
Manhattan 3.1 (fps): 188.6 / --
Manhattan (fps): 267.9 / 126.5
T-Rex (fps): 596.8 / 491.3
ALU2 (fps): 530.3 / 526.1
Driver Overhead2 (fps): 218.4 / 23.11
Texturing (MTexel/s): 68089 / 67053

In order to have a standard basis of comparison we should all be outputting it for OFFSCREEN so that all of the render target sizes are the same. We can then compare GPUs directly instead of relying on the machine's resolution which could vary widely.

Overall, the performance gains are pretty impressive for these benchmarks. Obviously, ALU and Texturing are GPU limited functions so their gains really wouldn't improve. However, a 20% bump with just the default OS X drivers are pretty good. Really wish we had more tests.

Sorry to borrow but to keep standard results and I didn't have much time
Mac Pro 6,1 Quad 3.7 GHz
Mac OS 10.11.3
12 GB ram
D700
1080P

OFFSCREEN

Test Name (value): Metal / OpenGL
Manhattan 3.1 (fps): 173.366 / --
Manhattan (fps): 248.875 / 157.07
T-Rex (fps): 393.955/ 369
ALU2 (fps): 565.235 / 523.65
Driver Overhead2 (fps): 178.768/ 45.408
Texturing (MTexel/s): 68894 / 67652

 

[jeanlain]

Ouch, I didn't expect the Mac Pro to be that far behind the iMac for CPU-limited tests, especially the Driver Overhead one. The iMac is twice as fast.

 

[Adam552]

Are there any reasonably intensive metal games out yet that we can compare fps with?

 

[Irishman]

Are there any reasonably intensive metal games out yet that we can compare fps with?

Nope.

 

[Adam552]

Nope.

That's a shame... anyone know any games coming out on Metal soon? I heard Blizzard was getting ready to implement Metal into some of their games.

 

[imacken]

That's a shame... anyone know any games coming out on Metal soon? I heard Blizzard was getting ready to implement Metal into some of their games.

As ever, we hear of developers 'planning' to adopt Metal, but, what is it, 9 months on and nothing has been announced as far as I know.
Another dead duck? Hope not.

 

[Adam552]

As ever, we hear of developers 'planning' to adopt Metal, but, what is it, 9 months on and nothing has been announced as far as I know.
Another dead duck? Hope not.

Actually just looked it up, seems Blizzard is already testing Metal with WoW:Legion http://us.battle.net/wow/en/forum/topic/20043464897

 

[Janichsan]

Actually just looked it up, seems Blizzard is already testing Metal with WoW:Legion…

…which is not due before September 2016. So, it's still a looooooooong time before it is officially supported.

 

[Adam552]

…which is not due before September 2016. So, it's still a looooooooong time before it is officially supported.

At least, it'll be included in the pre-expansion patch too ^^ get to test it on the PTR at least

 

[Grade]

What about that game showned on the Keynote, where Metal was announced? Is it still in development?

 

[Ferazel]

Yeah WoW is the only "high-end" game to support OS X Metal that I know of as well. Early tests from people in the WoW forums say it is about as fast a the 2.1 OpenGL rendering path was (OS X's Core 4.1 profile is notoriously slower). Which still doesn't hold a candle to Windows performance on the same hardware.

What about that game showned on the Keynote, where Metal was announced? Is it still in development?

That's Fortnite, still being worked on by Epic. https://www.fortnite.com/

Both Unreal and Unity have versions of Metal API for OS X available for people who use those engine to export in. Seem that Unity's doesn't support parallel render command encoders yet, so it's not fully optimized and shouldn't be used for benchmarks.

 

[marksatt]

What about that game showned on the Keynote, where Metal was announced? Is it still in development?

That's Fortnite, still being worked on by Epic. https://www.fortnite.com/

https://www.fortnite.com/

Yep - Fortnite shipped an update that added Metal as an option in January. Its still in closed alpha so I've no public details to share about Metal - that will need to wait for the final public release of UE4 version 4.11...

Both Unreal and Unity have versions of Metal API for OS X available for people who use those engine to export in. Seem that Unity's doesn't support parallel render command encoders yet, so it's not fully optimized and shouldn't be used for benchmarks.

Does the Unity D3D11 renderer use deferred contexts? If not then comparing to Metal without parallel command encoders is quite justifiable. Not every game/engine will benefit from parallel encoding since you really need to be render-thread/CPU limited for it to be faster.

 

[Ferazel]

Does the Unity D3D11 renderer use deferred contexts? If not then comparing to Metal without parallel command encoders is quite justifiable. Not every game/engine will benefit from parallel encoding since you really need to be render-thread/CPU limited for it to be faster.

My guess would be no since Unity is often targeting the lowest-common-denominator. While not every engine benefits from splitting their rendering logic into separate threads, I would be surprised if Unity didn't realize that this is the way the wind is blowing and re-engineering so they can take advantage of it. They've been "jobifying" their rendering so that they can get the rendering off a single thread and into multi-threaded jobs they can execute in parallel while keeping a thread for submitting GPU commands. So it's like a fake command buffer, but still synchronous. In the current beta release, one of the graphics programmers has stated that only DX12 is using native parallel command buffers with PS4 and Metal soon. Source

 

[JordanNZ]

Yeah WoW is the only "high-end" game to support OS X Metal that I know of as well. Early tests from people in the WoW forums say it is about as fast a the 2.1 OpenGL rendering path was (OS X's Core 4.1 profile is notoriously slower). Which still doesn't hold a candle to Windows performance.

They're comparing the same presets in live vs alpha without realising that the graphics have been bumped up massively in alpha at the same presets (view distance etc). Which makes the comparison void. Metal is already faster.

 

[wubsylol]

Metal is already faster.

My experience, using equivalent settings, is the opposite.