OS X Metal vs OpenGL benchmark

[tipoo]

Just a quick update to show the progress of the Metal drivers since my first test. Here are Metal results under macOS 10.12.4, in the same order.
34.6
85.1
78
72.3
5496

Metal now faster than El Cap OpenGL. The difference isn't big, but again, we don't expect to see much difference on a weak GPU. At least the GPU is not used less efficiently now.

Still seems very case by case for me, on my 27" iMac with a 755M I ran both this morning, everything except driver overhead was still better on the OpenGL test then the Metal 2 test. Odd. I wonder how much is API side and how much is optimizing for the API, as I recon these low level APIs are more like a lot of rope for a developer to use as they will, not always an instant performance uplift.

Wonder if this is using Direct to Display too, that's a big deal, lack of it always hurt MacOS GPU performance over windows since it was doing compositing and colour correction when it didn't need too, and compositing even when set to lower resolutions.


https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sierra-the-ars-technica-review/7/

And compute is even more different than graphics scores - the 2017 iMac scoring 115,000 in OpenCL compute was getting 50,000 on Metal compute on High Sierra. Hm.

 

[jeanlain]

I'm not sure whether the openCL and Metal compute test are doing the same things. Anyhow, to me metal is more important for graphics, and it does appear to do better than openGL in that area.
What "Metal 2 test" are you referring to? I've never managed to get GFXBench running on high sierra.

And I don't think window compositing really affects performance that much. I ran benchmarks in windowed mode, which does not require scaling the game content, to fullscreen mode, where the rendered image needs to be scaled to 5k (I have a 2017 iMac) and performance has always been exactly the same.
[doublepost=1508879856][/doublepost]Heh. Now GFXBench Metal is working again. Maybe it's the High Sierra security update that fixed a few things.
The T-Rex offscreen test has got a big boost compared to my test on Sierra (835 vs 710 fps). The others show similar results, or slightly lower fps.
OpenGL is way behind in all tests. Radeon pro 580.

 

[Janichsan]

And I don't think window compositing really affects performance that much. I ran benchmarks in windowed mode, which does not require scaling the game content, to fullscreen mode, where the rendered image needs to be scaled to 5k (I have a 2017 iMac) and performance has always been exactly the same.

Which is pretty much the point: if macOS were allowing Direct to Display rendering, (real) fullscreen would be noticeably faster than windowed mode at the same rendering resolution, pretty much regardless of scaling.

 

[jeanlain]

That's not a given. I ran benchmarks in windowed mode and fullscreen mode under Windows and I don't recall seeing a difference.

Compositing might add a 1-frame delay between render and final display (I don't know), which should be avoided in VR, but I really don't think it reduces fps much. It's only a tiny portion of the GPU time.

Unfortunately, I don't see how to measure the benefits of "Direct to display" in Metal 2, since v-sync is forced. Hitman can disable V-Sync, but it's because it uses openGL for the final screen swap, which probably prevents Direct-to-display (a Metal feature).

 

[marksatt]

That's not a given. I ran benchmarks in windowed mode and fullscreen mode under Windows and I don't recall seeing a difference.

Compositing might add a 1-frame delay between render and final display (I don't know), which should be avoided in VR, but I really don't think it reduces fps much. It's only a tiny portion of the GPU time.

Unfortunately, I don't see how to measure the benefits of "Direct to display" in Metal 2, since v-sync is forced. Hitman can disable V-Sync, but it's because it uses openGL for the final screen swap, which probably prevents Direct-to-display (a Metal feature).

Windowed mode will be slower than "exclusive fullscreen" provided the game is GPU bound. If the game is CPU bound there might be enough spare GPU cycles to hide the compositor's work. Modern games on Windows often use a fullscreen window - much like Apple's fullscreen mode - that still runs through the compositor which can make benchmarking tricky.

There is also a tax on the GPU for upscaling a game to fill a Retina display resolution and as a result does affect the frame rate. It is worse the slower the game is running as it eats GPU cycles that would otherwise be used by the game. Changing the physical display resolution to match the game obviously solves this problem.

If Hitman is using Feral's GL display technique then it is rendering directly to the display. Metal's "direct-to-display" technique is in effect a return to the way GL always worked.

 

[jeanlain]

If Hitman is using Feral's GL display technique then it is rendering directly to the display. Metal's "direct-to-display" technique is in effect a return to the way GL always worked.

You mean the way it worked before OS X 10.6, which IIRC introduced the fullscreen window for OpenGL apps (hence compositing)?
You said it yourself, Apple's fullscreen mode runs through the compositor. Metal's "direct to display" bypasses it, so it's different.

If Metal's "direct to display" is effectively a return to pre-Snow Leopard, will it change the monitor native resolution and make cmd-tab clunky? That's not something that a game developer may want to happen, but if I understand that WWDC session video, it's supposed to be automatic if the Metal game does the right things (fullscreen opaque render at a resolution that is not higher than the monitor and in a colour space that is compatible).

EDIT: the only "full" Metal apps I can test are War Thunder and GFXBench. They don't seem to behave differently from how they did Sierra in fullscreen mode, except that now fps is capped to 60 (was 120 before).

 

[ikir]

^ Where did you find more information? I can only find copies of Apple's PR.

Regarding F1 2016, I wonder if Metal matching or even beating DX11 will be the rule now. I've read at several places that WoW is still better under DX11 than Metal, same fore Obduction. Maybe the DX11 version of F1 2016 is just poorly optimised (could be a bad console port).
Anyway, this shouldn't undermine Feral's work.

As I know and from my test Metal with lastest games is on par with DX11. Tested with Radeon Pro 460, same numbers in DX 11 and Metal which is a big achievement.
On eGPU.io i’ve read F12016 was running Faster than DX11

 

[jeanlain]

On eGPU.io i’ve read F12016 was running Faster than DX11

A french website came to the same results, but for some users here the Metal version was slower than DX11.

Back to "direct-to-display". I've just tested both WarThunder and GFXBench and they behave just as if they had full screen windows. The game content "flies" over the desktop when I enable mission control with the gesture. I don't know if "direct-to-display" automatically kicks in when the game takes the whole screen and on-the-fly enables compositing when it doesn't. That'd be neat.

 

[Ritsuka]

If I remember correctly the windows server does some kind of optimisations if there is a full screen window.

 

[marksatt]

You mean the way it worked before OS X 10.6, which IIRC introduced the fullscreen window for OpenGL apps (hence compositing)?
You said it yourself, Apple's fullscreen mode runs through the compositor. Metal's "direct to display" bypasses it, so it's different.

Fullscreen windows do allow the compositor to run, but even with OpenGL it'd silently start bypassing it when the compositor wasn't doing anything. Then you'd get a lovely hitch whenever a notification wanted to display over the game and brought the compositor back in. (See Apple's GL Docs: https://developer.apple.com/library...acProgGuide/opengl_fullscreen/opengl_cgl.html).

Metal's direct-to-display is effectively just the same thing as the GL behaviour documented above, the implementation details are different but the result is the same. Previously in El Cap & Sierra a CAMetalLayer would *always* run through the compositor, even when nothing is being rendered above it, but on High Sierra when the compositor isn't required it can bypass it.

If Metal's "direct to display" is effectively a return to pre-Snow Leopard, will it change the monitor native resolution and make cmd-tab clunky?

No. You can get Direct-To-Display with a fullscreen window without capturing the display or changing resolution. Again, the GL tech note says the same thing .

*However* on Retina Displays to avoid paying a GPU performance penalty from having to upscale from a lower-resolution to the full Retina resolution it is still better to switch the display resolution down to match the game. Engines that choose to do this resolution switching might end up behaving a lot more like pre-Snow Leopard games.

EDIT: the only "full" Metal apps I can test are War Thunder and GFXBench. They don't seem to behave differently from how they did Sierra in fullscreen mode, except that now fps is capped to 60 (was 120 before).

Apple changed the behaviour of the convenience methods for Metal frame presentation in High Sierra to prevent rendering more frames than the display could actually physically show. UE4 never used these and always decided when to present frames itself, so it can still run at 120fps on High Sierra.

 

[jeanlain]

Thanks for the explanations.
So there is little benefit of Windows' "exclusive fullscreen" over what we have on macOS? Exclusive full screen "just" let rescaling be done by the display hardware instead of the GPU.
On a retina Mac, rescaling should be very fast if you play at exactly half the native resolution, as it merely quadruples the pixels and doesn't do any interpolation.
[doublepost=1509173846][/doublepost]

Metal's direct-to-display is effectively just the same thing as the GL behaviour documented above, the implementation details are different but the result is the same. Previously in El Cap & Sierra a CAMetalLayer would *always* run through the compositor, even when nothing is being rendered above it, but on High Sierra when the compositor isn't required it can bypass it.

Does it mean one may now disable V-Sync on a Metal app, or is this unrelated?

 

[marksatt]

Thanks for the explanations.
So there is little benefit of Windows' "exclusive fullscreen" over what we have on macOS? Exclusive full screen "just" let rescaling be done by the display hardware instead of the GPU.
On a retina Mac, rescaling should be very fast if you play at exactly half the native resolution, as it merely quadruples the pixels and doesn't do any interpolation.

You can still get an exclusive fullscreen mode on both macOS & Windows and it allows you to change the physical display resolution and also completely disable the compositor - you don't get any OS notification windows etc. Apple just don't recommend it and don't like to see it in the App Store.

Retina scaling isn't free. It used to be that the game was CPU bound and so there was spare GPU time to do the scaling and not affect the frame-rate. That is no longer the case and Metal exposes the reality that the scaling can result in a ~10% performance hit (give-or-take depending on final resolution). So it is better for the game to change the display resolution when entering fullscreen mode (regardless of whether it is Windowed Fullscreen or Exclusive Fullscreen) to better match the game's resolution and avoid this upscaling penalty.

Does it mean one may now disable V-Sync on a Metal app, or is this unrelated?

Yep, developers can disable V-Sync for Metal in High Sierra.

 

[jeanlain]

ep, developers can disable V-Sync for Metal in High Sierra.

Excellent.

Retina scaling isn't free. It used to be that the game was CPU bound and so there was spare GPU time to do the scaling and not affect the frame-rate. That is no longer the case and Metal exposes the reality that the scaling can result in a ~10% performance hit (give-or-take depending on final resolution). So it is better for the game to change the display resolution when entering fullscreen mode (regardless of whether it is Windowed Fullscreen or Exclusive Fullscreen) to better match the game's resolution and avoid this upscaling penalty.

Does it also occur when the game runs at exactly half the native monitor resolution? My testings suggest that the system is clever enough to just quadruple the pixels. I thought it shouldn't take much GPU time.

 

[jeanlain]

Yep, developers can disable V-Sync for Metal in High Sierra.

Does/will UE4 allow that?

 

[marksatt]

Does/will UE4 allow that?

Yes, UE4 4.18 lets you toggle V-Sync in full screen mode but not windowed or windowed full screen.

 

[jeanlain]

Cool. But does the fullscreen mode disable compositing entirely (like pre- OS X 10.6)?

 

[marksatt]

Cool. But does the fullscreen mode disable compositing entirely (like pre- OS X 10.6)?

Yep.

 

[jeanlain]

Yes, UE4 4.18 lets you toggle V-Sync in full screen mode but not windowed or windowed full screen.

War thunder can now disable V-Sync in fullscreen window (I haven't tested standard windowed mode). I saw this on my radeon 580 iMac. On my MacBook Air with intel HD6000, V-Sync still appears to be enforced and fps limited to 60.
Anyhow, it does appears that on certain setups, V-Sync can be disabled in composited Metal apps.

 

[imacken]

On eGPU.io i’ve read F12016 was running Faster than DX11

They are wrong!

 

[JordanNZ]

They are wrong!

I really think it depends on which graphics card you have... On certain AMD cards it seems to be on par.

 

[Janichsan]

I really think it depends on which graphics card you have... On certain AMD cards it seems to be on par.

"On par" is a far cry from "faster". The reports that F1 2016 could run faster with Metal than with DX11 are exactly what iMacken claimed: wrong.

There is only one single published benchmark backing this claim, which is vague on its methodology to say the least, and which no one was ever able to reproduce.