2018 Mac mini ± Hyperthreading

[Spectrum]

I've not seen anyone make this test before: Comparing i7 ± Hyperthreading

i7/8GB/1TB with Hyperthreading: https://browser.geekbench.com/v4/cpu/11297676

---
i7/8GB/1TB without Hyperthreading: https://browser.geekbench.com/v4/cpu/11298898

So...at least for Geekbench tests, hyper threading added by the i7 makes no difference (probably within margin of error).
I'm surprised that is for sure. This means that—at least for some tasks—the only real benefit of the i7 may be the slightly higher turbo speeds.

---
EDIT: For completeness, here is the i7 limited to 4 cores (no HT): https://browser.geekbench.com/v4/cpu/11299126

[doublepost=1545140328][/doublepost]

---
EDIT2: And here are the respective Cinebench scores: (a much bigger bonus from effect of HT):
6C/12T: ~1170 (stable @ ~3.5Ghz)
6C/6T: ~875 (stable @ ~3.8Ghz)
4C/4T: ~615 (stable @ ~4.2Ghz)

 

[Spectrum]

Is no-one else surprised that turning off hyperthreading has no effect on the Geekbench score?

 

[macsforme]

It seems to me that HyperThreading would facilitate a smoother allocation of resources to each thread when the number of threads running exceeds the number of available processor cores, but would not be expected to increase the overall maximum performance of the CPU. Perhaps someone with more knowledge on the matter could clarify this for us, though.

 

[pl1984]

Is no-one else surprised that turning off hyperthreading has no effect on the Geekbench score?

I'm surprised how prevalent Geekbench scores are used despite it being a very poor indicator of real world performance.

 

[FreakinEurekan]

Is no-one else surprised that turning off hyperthreading has no effect on the Geekbench score?

Maybe you aren't properly understanding what hyperthreading is doing? Each core can do a certain amount of work, you can do that work in one thread or split it into two, which can certainly benefit certain apps which use many threads but not to full capacity. But benchmarking is going to show raw processing, I don't expect to see much of a difference with or without HT.

 

[pl1984]

Maybe you aren't properly understanding what hyperthreading is doing? Each core can do a certain amount of work, you can do that work in one thread or split it into two, which can certainly benefit certain apps which use many threads but not to full capacity. But benchmarking is going to show raw processing, I don't expect to see much of a difference with or without HT.

Without understanding the details of how Geekbench performs its CPU benchmarks I think it is completely reasonable to expect hyperthreading to increase the multi CPU score. If it doesn't then I consider that a flaw in Geekbench as a hyperthreading processor offers more performance than a non-hyperthreaded processor in many situations. If Geekbench doesn't reflect that additional performance then one has to question the value of that score.

 

[FreakinEurekan]

If Geekbench doesn't reflect that additional performance then one has to question the value of that score.

Geekbench pushes the CPU to its limits in all available modes, which normal usage doesn't typically mimic. I don't trust Geekbench score to linearly mimic the differences I'd see in running a specific app or process on two different systems but ¯\_(ツ)_/¯ what would you propose they do differently? No "Benchmark" is going to perfectly match real-world use, just like one real-world use case doesn't match a different real-world use case.

 

[pl1984]

Geekbench pushes the CPU to its limits in all available modes, which normal usage doesn't typically mimic. I don't trust Geekbench score to linearly mimic the differences I'd see in running a specific app or process on two different systems but ¯\_(ツ)_/¯ what would you propose they do differently? No "Benchmark" is going to perfectly match real-world use, just like one real-world use case doesn't match a different real-world use case.

If you follow my comments about Geekbench you'll see I'm no fan of the benchmark (see post #4 in this thread).

Having said that I have to question the value of a score if it is not representative of the additional performance hyperthreading offers in many workloads. Based on this metric alone the Core i5 and Core i7 CPUs used in the Mini would be identical to one another (after accounting for the clock difference). For several tasks Hyperthreading can add significant performance gains. If Geekbench cannot account for that then it's even worse than I first thought.
[doublepost=1545233539][/doublepost]

I've not seen anyone make this test before: Comparing i7 ± Hyperthreading

i7/8GB/1TB with Hyperthreading: https://browser.geekbench.com/v4/cpu/11297676


---
i7/8GB/1TB without Hyperthreading: https://browser.geekbench.com/v4/cpu/11298898


So...at least for Geekbench tests, hyper threading added by the i7 makes no difference (probably within margin of error).
I'm surprised that is for sure. This means that—at least for some tasks—the only real benefit of the i7 may be the slightly higher turbo speeds.

---
EDIT: For completeness, here is the i7 limited to 4 cores: https://browser.geekbench.com/v4/cpu/11299126

[doublepost=1545140328][/doublepost]

---
EDIT2: And here are the respective Cinebench scores: (a much bigger bonus from effect of HT):
6C/12T: ~1170 (stable @ ~3.5Ghz)
6C/6T: ~875 (stable @ ~3.8Ghz)
4C/4T: ~615 (stable @ ~4.2Ghz)

What does CPU monitor show for each test? Does it show 12 threads with hyperthreading and only 6 without hyperthreading?

 

[SpacemanSpiffed]

As a AAA game developer, I can chime in with my direct observations. Overall Intel's hyper threading can provide a nice boost (35-45%) for the right kind of threaded workload. But, if there's a bottleneck such as completely saturating memory bandwidth in either direction (can happen when doing a lot of CPU based rendering for example), that becomes your limiting factor and more hardware threads barely moves the needle.

I don't know what GeekBench is doing, but it likely is maxing something out, or is designed 'oddly', because apps that use lots of threads, like most modern games (game engines went that way when consoles went multi-core in the PS3/XBox 360 era) see a measurable improvement with HT available.

 

[FreakinEurekan]

If you follow my comments about Geekbench you'll see I'm no fan of the benchmark (see post #4 in this thread).

Okay. I don't actually follow your comments, you replied to me. But be that as it may, does it invalidate anything I've said? If you're pushing a CPU to its maximum capability, with or without hyperthreading won't show a significant difference. Each core can do a certain amount of work if pushed to the max, whether it's all in one thread or split into two.

If someone is, let's say, using Handbrake to convert a 4k video to 1080p, then it's probably going to be pretty close to what GB shows. In that case, yes, a CPU without HT will perform about as well as one with, if they're the same clock speed. In many other cases, gaming as @SpacemanSpiffed mentions, HT will give a good performance boost. So should Geekbench dumb down the results if a CPU doesn't have HT? It's a benchmark, they have to draw the line somewhere and "All out" seems as good a place as any to me.

 

[jtara]

Hyperthreading works by allowing unallocated core resources to run a second thread.

Most/many instructions don't fully utilize the CPU. For example, if a thread is running an instruction that just fetches a value from memory to a register. You have a compute unit that is not being used. If another thread HAPPENS to have it's instruction pointer set such that the next instruction is e.g. to add the value of one register to another, that thread can proceed in parallel. One thread only needs to fetch from memory. The other only needs to add, and needs no memory access.

That's hyper-simplified (see what I did there?) but the gist of it. There are some duplicated functional units. There is out-of-order execution. (e.g. is there an "add" in the next few instructions, that does not depend on not-yet-executed instructions for it's data? It can be opportunistically executed out-of-order.

How useful hyper-threading is depends on the workload and particular instructions.

To test the theoretical gain of hyper-threading, you need a benchmark designed for this purpose.

It's more useful to use real workloads - and the kind of real workloads that interest you.

 

[Spectrum]

What does CPU monitor show for each test? Does it show 12 threads with hyperthreading and only 6 without hyperthreading?

Yes. Here with HT:

Here without:

 

[pl1984]

Okay. I don't actually follow your comments, you replied to me.

It was a figure of speech, a means to alert you that I am not a fan of Geekbench. Does this really need explanation?

But be that as it may, does it invalidate anything I've said? If you're pushing a CPU to its maximum capability, with or without hyperthreading won't show a significant difference. Each core can do a certain amount of work if pushed to the max, whether it's all in one thread or split into two.

If that's the case then I would posit Geekbench is even worse a benchmark than I first thought.

If someone is, let's say, using Handbrake to convert a 4k video to 1080p, then it's probably going to be pretty close to what GB shows. In that case, yes, a CPU without HT will perform about as well as one with, if they're the same clock speed. In many other cases, gaming as @SpacemanSpiffed mentions, HT will give a good performance boost. So should Geekbench dumb down the results if a CPU doesn't have HT? It's a benchmark, they have to draw the line somewhere and "All out" seems as good a place as any to me.

Handbrake does benefit from HT.
[doublepost=1545236815][/doublepost]

Yes. Here with HT:

Here without:

That's even worse than I thought. This shows GB doesn't stress the CPU much at all but is utilizing the additional threads available. What does it look like in the "CPU Usage" view (command+2)?

 

[Spectrum]

It was a figure of speech, a means to alert you that I am not a fan of Geekbench. Does this really need explanation?

If that's the case then I would posit Geekbench is even worse a benchmark than I first thought.


Handbrake does benefit from HT.
[doublepost=1545236815][/doublepost]
That's even worse than I thought. This shows GB doesn't stress the CPU much at all but is utilizing the additional threads available. What does it look like in the "CPU Usage" view (command+2)?

My test above also clearly show that Cinebench also benefits from HT (1170/875=+33% gain from Hyperthreading, despite the average core speed dropping).
I presumed that the tests within Geekbench were designed so as to give an average across all sort of workloads, including those that benefit from hyper threading.
I notice that there is a tiny increase in Geekbench score +HT. So maybe Geekbench is not (additionally) flawed, but the result simply means that there are very few workloads included in Geekbench that benefit from +HT.

I'd be interested in someone doing some heavy encoding to compare the i7 mini ± HT turned on.

 

[pl1984]

My test above also clearly show that Cinebench also benefits from HT (1170/875=+33% gain from Hyperthreading, despite the average core speed dropping).
I presumed that the tests within Geekbench were designed so as to give an average across all sort of workloads, including those that benefit from hyper threading.
I notice that there is a tiny increase in Geekbench score +HT. So maybe Geekbench is not (additionally) flawed, but the result simply means that there are very few workloads included in Geekbench that benefit from +HT.

If true it makes me question the multicore results even more. The purpose of a benchmark is to give a general idea of the computing power between different systems. This would appear to be a glaring oversight if it cannot account for HT.

I'd be interested in someone doing some heavy encoding to compare the i7 mini ± HT turned on.

I'd be willing to do it on my 2010 or 2013 Mac Pro's but it appears you need the developer tools installed in order to disable HT (if someone knows otherwise please let me know). I can perform the test on my Z440 system where I can disable HT in the UEFI. I may do that later this afternoon but it wouldn't be a macOS result.

 

[Spectrum]

If true it makes me question the multicore results even more. The purpose of a benchmark is to give a general idea of the computing power between different systems. This would appear to be a glaring oversight if it cannot account for HT.


I'd be willing to do it on my 2010 or 2013 Mac Pro's but it appears you need the developer tools installed in order to disable HT (if someone knows otherwise please let me know). I can perform the test on my Z440 system where I can disable HT in the UEFI. I may do that later this afternoon but it wouldn't be a macOS result.

If you have an AppleID you can download Xcode and developer tools.

 

[GrumpyCoder]

Is no-one else surprised that turning off hyperthreading has no effect on the Geekbench score?

Why would anyone be surprised? This type of benchmark isn't designed to take advantage of hyperthreading (or to step away from Intel, simultaneous multithreading in general). In some situations HT doesn't do anything, in others there's a nice benefit. You will see a benefit for thread-level-parallelism, simultaneously issuing and execution, proving multiple hardware contexts, scheduling and switching capability. And of course hiding latency, branch misprediction, memory, etc.

This has been discussed so many times, that I can say, just use the right tool for the job and Geekbench isn't the right tool for benchmarking HT performance. If anyone is interested in this, I always recommend to start with Tanenbaum's Modern Operating System, then Hennessey's Computer Architecture: A Quantitative Approach, Shen's Modern Processor Design: Fundamentals of Superscalar Processors. Those are for the basics, then do some research and then look at some more recent resources such as: https://www.agner.org/optimize/.

All of this will keep you busy for a while and it's not for the end user checking emails and surfing the web on his Mini or any other computer. But it will help to understand why Geekbench isn't the right tool for this specific job.

 

[Spectrum]

I am no computer scientist - just a user.

1. It is empirically true that some workloads benefit from HT
2. Geekbench advertises itself as benchmarking tool that simulates average workloads across a range of tasks and platforms. It is not unreasonable to assume that this includes tasks that benefit from HT (unless they are so rare as to be irrelevant).
3. Given 1&2, I expected Geekbench to reveal the performance delta provided by HT.

The fact that Geekbench doesn't show this, is, arguably, rather important.
It may mean that ±HT is really pretty specialist and unimportant for the average user.
(But may be very important to some specialist users nonetheless.)

Thus, for the average user, there may be little or no benefit of a BTO i7 over a stock i5 (or i3) for example.

 

[pl1984]

This has been discussed so many times, that I can say, just use the right tool for the job and Geekbench isn't the right tool for benchmarking HT performance. If anyone is interested in this, I always recommend to start with Tanenbaum's Modern Operating System, then Hennessey's Computer Architecture: A Quantitative Approach, Shen's Modern Processor Design: Fundamentals of Superscalar Processors. Those are for the basics, then do some research and then look at some more recent resources such as: https://www.agner.org/optimize/.

Your average Geekbench "user" isn't going to know this and they're not going to read up on processor design. I'm very familiar with operating system designs (and coding practices, I have a degree in programming) and I wasn't aware the multiprocessor GB score didn't reflect at least some benefit from HT. The fact it does not makes it an even worse benchmarking tool than I first thought was.

 

[Spectrum]

Your average Geekbench "user" isn't going to know this and they're not going to read up on processor design. I'm very familiar with operating system designs (and coding practices, I have a degree in programming) and I wasn't aware the multiprocessor GB score didn't reflect at least some benefit from HT. The fact it does not makes it an even worse benchmarking tool than I first thought was.

It is even more relevant for the 2018 mac mini (and iMacs too) where the choice between i5 and i7 is largely driven by whether the user wants ±HT (plus ~+10% maximum boost speed).

There have been many comments elsewhere suggesting that either i3 or i7 are the best choices.
My testing here makes me think differently - that, for the average user, an i5 without HT is a far better choice if funds are limited. Put the savings towards more RAM, a bigger SSD, an external drive, an eGPU, or a better monitor...

This decision is not the same for laptop parts, however, which all somewhat inexplicably include HT. In the past this was also the case for the mac mini - because it too used a laptop CPU.

e.g. I am sure many users chose the i7 quad core 2011/2012 models, not for HT, but for the quad cores. It just also came with HT, so they look like a great thing to have. It certainly influenced my decision to order the 2018 i7 mac mini rather than the stock i5 2018 model.

 

[GrumpyCoder]

1. It is empirically true that some workloads benefit from HT
2. Geekbench advertises itself as benchmarking tool that simulates average workloads across a range of tasks and platforms. It is not unreasonable to assume that this includes tasks that benefit from HT (unless they are so rare as to be irrelevant).
3. Given 1&2, I expected Geekbench to reveal the performance delta provided by HT.

1. True.
2. That's something you have to talk about with those making Geekbench.
3. If that's how they're advertising, it's to be expected yes. I can't tell you if they mean that it includes HT. As it seems, that's not the case.

It may mean that ±HT is really pretty specialist and unimportant for the average user.

I would not say that. The average user can still benefit from HT, but it depends on the application. If you run a thread in a CPU core and that thread takes up 100% space (let's just call it "space" to simplify, even if it's not correct) of that core, there's no benefit from HT. Switching between threads would then mean performance will go down due to the overhead introduced by HT. On the other hand, if two threads take less space, there can be a benefit because some of the space can hold information from thread A and B and switch quickly. It really depends on how optimised the threads are and what your application is doing in the first place. In some cases there's no benefit from HT, in others there is.

Your average Geekbench "user" isn't going to know this and they're not going to read up on processor design.

Yes, no doubt about it. The problem is when these tools are used for benchmarks and the results are misinterpreted. This is not a user fault, the creators of the benchmark tools are to be blamed. Reminds me a lot of the MHz war we had about 20 years or so ago. Double the MHz for twice the performance, or so they said back then.

 

[pl1984]

Yes, no doubt about it. The problem is when these tools are used for benchmarks and the results are misinterpreted. This is not a user fault, the creators of the benchmark tools are to be blamed. Reminds me a lot of the MHz war we had about 20 years or so ago. Double the MHz for twice the performance, or so they said back then.

These are in conflict with one anther. It's the user who interprets the results.

 

[jtara]

If Geekbench were designed to measure the benefit of hyperthreading, it would report hyperthreaded/non-hyperthreaded results.

It was not. OP used Geekbench for a purpose for which it was not designed, and invented his own benchmark measurement.

 

[GrumpyCoder]

These are in conflict with one anther. It's the user who interprets the results.

While it is the user that interprets the results, it's the creators of benchmark tools that suggest you can just download, let it run and read some number. Truth is, and this might sound a little harsh, those that create benchmark tools shouldn't put them in the hands of users or at least tell them to keep their hands away and not use something they don't understand. Neither should someone who can flip a switch to turn a lightbulb on/off be put into a nuclear power plant and push a few buttons.

Edit: To clarify, I'm not blaming anyone here, but I think it's dangerous what happens in the IT world these days. Everyone is told "you can do it, it's so easy". Just look at all the youtube videos of people telling others how easy it is to upgrade memory in the MacMini. Most of these guys lack the proper tools such as anti-static mats and wrist straps, yet they're telling others to do it. It's all fun and games until someone loses and eye... or in this case, break something.

 

[Spectrum]

If Geekbench were designed to measure the benefit of hyperthreading, it would report hyperthreaded/non-hyperthreaded results.

It was not. OP used Geekbench for a purpose for which it was not designed, and invented his own benchmark measurement.

In what way did I invent my own benchmark measurement?
I was intrigued by two things:
1. The potential benefits of HT
2. The fact that, in Geekbench tables, the i5 and i7 mini are not so far apart.

I then used Xcode Instrument to perform an experiment.
I got a result I found interesting and surprising.
I posted it here for discussion.


EDIT. Interestingly, someone way back in 2014 was able to observe HT increase in Geekbench of about 20%
https://forums.macrumors.com/threads/small-speedup-from-hyper-threading.1809392/
So...maybe the test has changed, or I did something wrong?