Metal in El Capitan - Gaming Performance Prediction Thread

[Irishman]

So, September is rapidly approaching, and with it should come Apple's latest and greatest OS X upgrade. We know the basics of Metal (Apple's new graphics API), but specifics and particulars are shrouded beneath layers and layers of NDA. As usual, those who know can't speak.

But this is Mac Rumors and this community has never let a paucity of facts stop us before! And, let's be honest, there's a certain personal pride to be had for getting a prediction correct, right?

So, what kinds of performance gains do you think we'll see once Metal hits its stride? By this I mean the time between now and when developers have been able to write their games to it. Aspyr has already indicated some confidence in their ability to match or exceed Windows performance of games they port to OS X. What do you think that means for individual games?

10%? 20%? More?

Will we see more games that can do 60 fps in older Macs?

I'll go on record and predict an average of 25% faster than OpenGL Yosemite. Some will be greater, some will be less.

 

[jeanlain]

I expect the biggest performance gains to be seen in situations where a game performs much better on Windows, because the difference points to problems in the openGL code of the game or the openGL implementation in OS X.
We can expect Metal to solve the latter. Now, it's not impossible to produce poor Metal code if the developer wants to simply translate DirectX coding principles that don't work well with Metal. A different approach may be needed.

From a more theoretical standpoint (and from what I understand, because I'm not a developer), the biggest gains are to be seen when the application CPU thread is waiting on the drivers' thread, which should occur when the GPU is not the limiting factor. Those situations would occur when the GPU is much faster than the CPU and when they are a lot of objects to draw (lots of "draw calls").

Now, what kind of performance increase to except, I would say 30-40% faster than an openGL equivalent at the very best, (+10-20% typical), but that's a guess.

 

[koyoot]

So far, what weve seen is that Mantle, and othe low-level APIs on Windows 10 side give over 20% increase in performance in sub 1600p resolutions. However, at 4K, according to golem.de, and depending on the game we can see even decrease in performance(which completely contradicts logic), to 10% increase in performance. That is on Windows side.

Potentially Metal should equal the performance of Windows 10. It should be within at max 10% of Windows 10 performance, depending how good are developers at Metal optimization. It has potential to bring more performance than Windows side, but... that will only time show.

 

[Dirtyharry50]

I predict that it is unpredictable!

 

[Huntn]

I expect the biggest performance gains to be seen in situations where a game performs much better on Windows, because the difference points to problems in the openGL code of the game or the openGL implementation in OS X.
We can expect Metal to solve the latter. Now, it's not impossible to produce poor Metal code if the developer wants to simply translate DirectX coding principles that don't work well with Metal. A different approach may be needed.

From a more theoretical standpoint (and from what I understand, because I'm not a developer), the biggest gains are to be seen when the application CPU thread is waiting on the drivers' thread, which should occur when the GPU is not the limiting factor. Those situations would occur when the GPU is much faster than the CPU and when they are a lot of objects to draw (lots of "draw calls").

Now, what kind of performance increase to except, I would say 30-40% faster than an openGL equivalent at the very best, (+10-20% typical), but that's a guess.

Does Metal replace OpenGL or update it?

 

[marksatt]

Does Metal replace OpenGL or update it?

Metal is a replacement. Despite the protestations from the Khronos Group, OpenGL's time is up.

 

[koyoot]

As a whole, or only on OSX?

 

[marksatt]

As a whole, or only on OSX?

On the Mac the future is clearly Metal. Legacy OpenGL may persist for years & years, but it will have been superseded by the low-overhead APIs (Metal, D3D, Vulkan).

 

[marksatt]

Metal on the Mac will be brand new at El Capitan's release, whereas the current incarnation of D3D has had a decade or so of work by Microsoft & GPU vendors, so expecting that the Mac will suddenly be equal or faster than Windows is somewhat unrealistic. It should be better than GL but in games performance improvements are dependent on a lot of factors, so don't set yourselves up for disappointment when your favourite game doesn't see the improvement you have assumed Metal will offer.

 

[teagls]

Metal on the Mac will be brand new at El Capitan's release, whereas the current incarnation of D3D has had a decade or so of work by Microsoft & GPU vendors, so expecting that the Mac will suddenly be equal or faster than Windows is somewhat unrealistic. It should be better than GL but in games performance improvements are dependent on a lot of factors, so don't set yourselves up for disappointment when your favourite game doesn't see the improvement you have assumed Metal will offer.

There are still a lot of factors to performance like the drivers, the quality of programming for a particular game, etc. Based on Apple's track record of writing god awful drivers for OpenGL I don't hold much hope for their metal drivers. But, since the metal api is very clean and modern and standard in OS X it might make maintaining and writing drivers easier. Secondly, you can do a ****** job programming a game on Windows with D3D and it will still run like crap. It's up to the developer to allocate the labor and money to develop the game in metal. As mentioned before since the metal api is much more simplified and does not have a lot of the extension and bulk of OpenGL it might take less development time.

 

[Huntn]

Metal is a replacement. Despite the protestations from the Khronos Group, OpenGL's time is up.

Excellent. If you are third rate, can't you expect to be replaced?

 

[Ferazel]

I don't know how much Khronos is weeping for OpenGL as they decided to usurp it's entire functionality with Vulkan. As previously mentioned, OpenGL will continue to live on in backwards compatibility for quite awhile. OpenGL will remain the defacto graphics API on Linux and Android for many years to come. The hardware fragmentation on those platforms makes it difficult for developers to jump ship to learn and target Vulkan as their primary rendering path in the near-term future (I'm guessing a 2-3 years). Also another thing to think about is that for teaching high-level 3D, OpenGL and DirectX 11 may be preferable in an educational environment due not needing to teach lower-level functionality.

On the Mac side I think that OpenGL will be dropped much more quickly. Just looking at Steam stats, over 70% of Mac users are running some version of Yosemite which was only released last year. Like iOS adoption rates, it makes it very easy for developers to decide which platforms to target and what technology to support. Obviously, with Metal there is also a hardware driver component which unfortunately leaves a lot of Radeon 5000-6000 cards in the dust, but overall you're probably looking over 60% of the OS X market that will be Metal compatible in 1 year's time. To me, that is an easy decision to make. The remaining users are unlikely to be upgrading to buying new software.

Back on topic... there are no benchmark tools for Metal, so no one has any idea on the exact performance. The only people who do, are likely bound by developer NDAs that prevent them from discussing it. Plus the developers just started re-engineering their games for Metal and may not represent final speeds. I hope that we can get UniEngine or some other well-engineered engine (*cough*) to have a benchmark to the public by the time the public beta rolls around. However, from what I was seeing at the WWDC presentations, in order for Metal to really give us that boost of performance it needs to be put into a CPU-limited environment with multi-threaded command buffers with some hand-wavy engineering to cut down on the number of render submissions to the GPU.

I think the biggest immediate effect of Metal OS X games will not be from performance but in power efficiency. Laptop users will get longer battery times and less heat from their computers as the CPU will be paused waiting on the GPU more. Until game devs start taking advantage of low-overhead APIs on the Windows side and filling that CPU with some gameplay changes (AI, Physics, or more buffer submissions) the most immediate effect will be cooler computers.

Going even further into speculation land, and where Apple wants to take Metal on OS X. I could see Apple doing something really cool and creating one of the first high-speed unified system memory PC. Not the rumored low-end ARM Macs mind you, but a high-end Mac Pro level of computing (at least to start). Currently, Intel-GPU only Macs are basically doing this in a kind of bastardized process. The memory sanctioned by the GPU can not be directly accessed by the CPU (although I wonder if Metal drivers will allow this or if it is a hardware limitation). Copies of the same data must be communicated through the memory controller to communicate between the GPU and CPU. This type of communication is slow. Also, system memory currently is a relatively low bandwidth DDR3 memory (15GB/s) and the Intel GPUs are "meh". However, This might explain Apple's coziness with AMD (as their GPUs are appearing in Macs again). AMD is an early adopter of High-Bandwidth Memory (512 GB/s) and HBM unified memory might make for a compelling pro computer experience for software that can take advantage of it. Imagine a pro computer that used 1 quad-core CPU and 2 GPUs to grind away at the same memory addresses each with 512 GB/s access to the same memory.

 

[Huntn]

So will El Capitan kill every Mac game that came before it? Or will there be backwards compatibility? And will developers be supporting both standards for a while?

 

[antonis]

Guys, the openGL is present in El Capitan. And it will be. Even if we forget about the games for a moment, there is a big number of other applications still using it.

 

[Ferazel]

Yeah I didn't mean to imply that OpenGL will be removed entirely in future versions of OS X. Only that developers would target the new API a lot quicker than on other platforms where OpenGL is currently being used.

 

[Irishman]

So, now that El Cap is out in the wild, and we're upgrading, how long do you think it will be before we see the first Metal-optimized games for OSX??

 

[Yaboze]

Blizzard said by the end of the year for WoW. That's the only one I know of.

 

[jeanlain]

There are already some results! (that was unexpected)
http://arstechnica.com/apple/2015/1...-os-x-el-capitan-sometimes-great-often-mixed/
Disclaimer: this is on a unfinished version of a benchmark tool.
There are some interesting gains on Nvidia. As well as some losses. The gains matter more, as we can expect performance issues to be fixed in the future.
There not much to expect for intel GPUs since Metal mostly helps when the CPU is the limiting factor, which is not the case when the GPU is slow.

 

[Dirtyharry50]

There not much to expect for intel GPUs since Metal mostly helps when the CPU is the limiting factor, which is not the case when the GPU is slow.

Whether the CPU or the GPU is the limiting factor is related to how the game is coded not the hardware it runs on.

I don't think it is correct that Metal mostly helps when the CPU is the limiting factor. It helps by reducing driver overhead and it helps by allowing for processing to be offloaded from the GPU to the CPU so the GPU can be focused on such tasks as physics and A.I. according to what the linked article states. In this way it makes use of the CPU to a greater extent than was previously the norm. For most games currently not using an API like Metal (which at the moment is everything) the CPU is not usually that much of a factor to begin with compared to the all important GPU. There are exceptions to every rule of course but CPU is rarely an issue with current games compared to GPU when it comes to performance. Most systems now have very capable CPUs when compared to GPUs just in general and I'm sure that is why new APIs like Metal are designed to capitalize on that CPU processing power that is currently going to waste while GPUs are overworked. I would expect this had to be the situation with iOS devices where Metal first appeared and brought substantial gains.

While much less impressive gains are had with integrated GPUs in the testing they conducted I think that is more a function of the processing they are capable of than anything else. After all, it isn't news that integrated GPUs are not nearly as powerful as GPUs on graphics cards although fortunately they are making strides with each new iteration.

I think if we were to see a set of benchmarks that just tested performance of physics and A.I. calculations on integrated GPUs vs those on graphics cards the results would closely resemble the differences noted in the more general tests performed by the benchmarking software used in the linked article.

The thing is, this benchmark while interesting does not reflect accurately what is going to happen in the real world as games are developed. I would expect game developers to take all of the options into account and use the features in a more balanced way that strives to provide optimal performance across the board to the greatest extent possible in real world scenarios. There will be more at play than what is measured by these tests not the least of which is the resourcefulness of skilled software engineers who will work hard to provide the best gains possible across current hardware and that is going to include in the Mac OS X world a lot of integrated GPUs going forward.

Of course, I might just be misunderstanding this but what I note above is what I get out of what I just skimmed in that article and previous stuff I've read on the subject.

I hope that perhaps Mark from Unreal may see this and provide some clarification based on his own direct experience working with Metal. I would like to understand it better if I do not understand it correctly now.

 

[Eric5h5]

You can't just offload physics, or any calculations, to the GPU; it would require a complete rewrite. The CPU is often a big factor in performance with games due to how OpenGL works with driver overhead. Metal reduces that overhead, so games with, for example, lots of draw calls, will see benefits.

--Eric

 

[marksatt]

You can't just offload physics, or any calculations, to the GPU; it would require a complete rewrite. The CPU is often a big factor in performance with games due to how OpenGL works with driver overhead. Metal reduces that overhead, so games with, for example, lots of draw calls, will see benefits.

Eric5h5 is totally correct. Metal, Vulcan & DX12 are about exporting driver complexity to the application, thus allowing us as developers to avoid unnecessary CPU time validating driver state that we can ensure is always correct. So, in theory we should be able to do more graphics work in less CPU time. Offloading work to the GPU requires rewriting existing code from C++ to Compute shaders - which only works well for certain problems, is very labour intensive and prone to finding all the unfortunate errors in graphics drivers and shader compilers.

While much less impressive gains are had with integrated GPUs in the testing they conducted I think that is more a function of the processing they are capable of than anything else. After all, it isn't news that integrated GPUs are not nearly as powerful as GPUs on graphics cards although fortunately they are making strides with each new iteration.

Correct.

I think if we were to see a set of benchmarks that just tested performance of physics and A.I. calculations on integrated GPUs vs those on graphics cards the results would closely resemble the differences noted in the more general tests performed by the benchmarking software used in the linked article.

That's all CPU work, so Metal/GPU won't have a direct affect.

The thing is, this benchmark while interesting does not reflect accurately what is going to happen in the real world as games are developed. I would expect game developers to take all of the options into account and use the features in a more balanced way that strives to provide optimal performance across the board to the greatest extent possible in real world scenarios.

GFxBench isn't a great example as they are just rolling demos rather than games & there will be different engineering decisions taken for a game designed for Mac/PC to ensure scaling. However...

There will be more at play than what is measured by these tests not the least of which is the resourcefulness of skilled software engineers who will work hard to provide the best gains possible across current hardware and that is going to include in the Mac OS X world a lot of integrated GPUs going forward.

Unfortunately I have to disagree with this. I expect to see traditional developers focusing on making the very best use of the graphics features of DX12 (& its PS4 equivalent) to maximise rendering perf./quality *for console*, but that won't be balanced and will skew *heavily* toward modern, powerful discrete GPUs when brought to desktop.

There aren't many traditional PC games that officially support Intel GPUs (most don't even support mobility variants of AMD & NV GPUs!) because these machines often aren't fast enough and their owners haven't historically made up much of the audience for traditional games. This may be changing by default on Mac, where Apple don't ship fast GPUs in anything but the 5k iMac, but AFAIK isn't happening amongst the WinPC gamers targeted by traditional developers. Any computer company would asphyxiate support for gaming on their machines if they went Intel integrated only because traditional games developers *have* to focus on the market that makes them (nearly) all their money - which remains consoles and gaming PCs where avg. GPU power is many times greater than the fastest Intel part.

Clearly that doesn't apply to indie or mobile developers - but that's not my professional background.

 

[Ferazel]

Interesting that they posted an update to that ArsTechnica article that has a claim from the benchmark dev.

Kishonti Ltd, creators of GFXBench, have contacted us to say that a new build of GFXBench Metal for OS X will be available soon and that it may change performance. We'll run the new version of the test and update this article as soon as it's available.

One of the golden nuggets in the article is that the Manhattan bench performed so much better under Metal than OpenGL, especially on the 680MX (50% improvement!). However, that is the exception and not the norm. I have a feeling that Apple knows that the state of OS X Metal is not ready for primetime. That is why Blizzard is waiting for some El Capitan point releases to fix up some remaining issues and maybe improve performance above OpenGL. Unity has also updated their engine to support OS X metal in their latest beta with the claim that performance may be worse in some situations.

@marksatt - I think your Unreal engine bias is showing (which you admit to). I agree with you that the "traditional" game devs are Windows/Console focused and they want to target the high profile console market. However, I feel gaming in the last 5 years has changed and/or expanded. I would argue a large percentage of of game devs today target mobile as their primary platform then port that game to desktop. I think the type of gaming that Apple is interested in pursuing is at this more mainstream mobile/tablet level. Apple is a huge force in the mobile market and is synergistically tying it into the OS X ecosystem. I'm guessing that as long as their GPUs remain faster than their current A# SoC parts they're OK with that level of 3D performance. I don't necessarily like the direction they're taking, but that would be my guess on Apple's OS X priorities.

 

[jeanlain]

The gains on the "Manhattan" scene are interesting. It is the most complex scene, but it's still relatively simple compared to many games. The other scenes, especially the static ones, should have low driver overhead even in openGL.
On of the big advantage of Metal is the possibility to use multiple CPU cores for graphics tasks (submit commands from multiple threads), something that is harder to achieve in openGL. I suppose the GFXBench doesn't especially take advantage of that.
In a session about Vulkan at Siggraph, they showed huge improvements over openGL, but they used demoes with thousands of objects, lots of particles, and optimisation for multiple cores. Quite different from the current GFXBench.

They also said that Metal cannot enable the performance that DX12 and Vulkan can reach because it retains the openGL binding model (I don't know that that means specifically), but that also makes Metal simpler to use.

 

[Dirtyharry50]

Thank you Eric and Mark for the clarifications. I somehow managed to get this in reverse when I read it I think, "since one of the API’s goals is to process more on the GPU so that the CPU is free to handle physics or AI" in the linked article. Oops!

As for the work of skilled engineers I mentioned, I was thinking primarily about the guys at Aspyr and Feral but also other companies developing for Macs that take into account the large numbers of them with integrated GPUs.

 

[marksatt]

I think your Unreal engine bias is showing (which you admit to),

Not just Epic, but Feral and Frontier too, where AAA is the name of the game.

I agree with you that the "traditional" game devs are Windows/Console focused and they want to target the high profile console market. However, I feel gaming in the last 5 years has changed and/or expanded. I would argue a large percentage of of game devs today target mobile as their primary platform then port that game to desktop.

Games is an always changing market and you are right that there are a lot of mobile developers porting their games or even outright switching focus to PC/console. They are beginning to fill the space in the market that the defunct mid-tier AAA developers once occupied. These developers will face the same perennial problems of having to compete with the big-hitters on fractions of the budget & struggling for market share.

I think the type of gaming that Apple is interested in pursuing is at this more mainstream mobile/tablet level. Apple is a huge force in the mobile market and is synergistically tying it into the OS X ecosystem. I'm guessing that as long as their GPUs remain faster than their current A# SoC parts they're OK with that level of 3D performance. I don't necessarily like the direction they're taking, but that would be my guess on Apple's OS X priorities.

Certainly on iOS/tvOS that makes a great deal of sense, as it plays well with the audience and is acceptable to developers. Integrated GPUs on desktop make AAA support very, very tricky but the real issue would be the knock on effects for developing games targeting mobile platforms: game development tools, like UE4Editor, perform far better on discrete GPUs than integrated ones (even under Windows) and they can be even more demanding than AAA games*.

* This statement is definitely subject to Unreal bias - my last prolonged experience with Unity dates back to this article I wrote for IMG back in the day. I might work for Epic, but Unity's success still makes me smile knowing it started on the Mac.