Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Big Favor Needed for the Chess Community

H

h1a8

macrumors newbie
Original poster
Jan 13, 2010
4
0
Hi everyone. Apparently, the chess community believes that macs are handicapped when it comes chess calculation (in comparison to windows computers). They believe that macs are not optimized for chess as well as windows computers are. This is because macs are allegedly getting bad benchmark results in comparison to current windows intel i7/i9 and amd(ryzen) machines. Therefore, most chess professionals and enthusiasts only use a windows computer for chess analysis.

My theory is that the reason the macs are getting bad results is due to allocating more hash than macOS is willing to give as physical memory, resulting in virtual memory being used for hash, which in turn causes very bad performance. I believe that decreasing the size of the hash until only physical memory is used improves performance drastically.

I ask that members here run a couple of stockfish benchmark tests using their m3 max (14 and 16 core) and m2 max/pro (12 core) macbooks and then post the results here. I will report the findings to all the major chess sites.

The benchmark file (Stockfish M1 pop-neon) can be found here:
under Apple/Mac to the top right
You can also view the ranking list for many different processor/OS setups shown.

Once you download the file run the following commands for the 14-core (m3 max) machine
"bench 512 14 26 default depth nnue"

and then run the test again with this command
"bench 1024 14 26 default depth nnue"

The latter command uses more hash memory (1024MB instead of 512MB). I want to see if there is a drastic difference between the two.

For 16-core machines (m3 max) or for 12-core machines (m2 max/pro) just change the "14" in the command to a "16" or to a "12", respectively.

Thank you!
 
Hi h1a8.
I tried ... but there is nothing "running" or happening.
  1. After downloading, there is "Stockfish 14.1 M1 pop-neon" in my download directory.
    --> What can/shall/must I do with this?
  2. Where do I have to enter the "bench 512 14 26 default depth nnue"
    --> I tried in Terminal ... resulting in the error message "bench: command not found"
 
MichaelAC said:
Hi h1a8.
I tried ... but there is nothing "running" or happening.
  1. After downloading, there is "Stockfish 14.1 M1 pop-neon" in my download directory.
    --> What can/shall/must I do with this?
  2. Where do I have to enter the "bench 512 14 26 default depth nnue"
    --> I tried in Terminal ... resulting in the error message "bench: command not found"
Click to expand...
Maybe move the file to your desktop? You click on the file and a command prompt should appear (that's how it works in windows). Then you type the command (without the "" marks of course) and hit enter.
 
SjoukeW said:
I have a M2 Max for work. Happy to run your benchmarks. The file downloaded wanted to open in a text editor…
Click to expand...

I had the same issue myself, but I found a newer version (Stockfish 16, the aforementioned link pulls from version 14.1) via Homebrew. I installed it via Terminal, then ran the following to run the benchmark:

Code: 
stockfish bench 1024 12 26 default depth nnue
 
Last edited:
Results running 1024MB:

Code: 
===========================
Total time (ms) : 148303
Nodes searched  : 986525637
Nodes/second    : 6652094

Results running 512MB:

Code: 
===========================
Total time (ms) : 151782
Nodes searched  : 1080479934
Nodes/second    : 7118630

So in my case, the 1024MB test completed ~3400 faster than the 512 MB test, although the latter test did search through more nodes overall and per second than the 1024MB test.

Now for fun, I decided to rerun the 1024MB test, but I swapped out the number of CPU cores for the number of GPU cores (on my machine) just to see if that would affect the results. Turned out that almost doubled the time from the 12 cores test, but it also beat the nodes per second results of both the initial tests using 12 cores.

Code: 
===========================
Total time (ms) : 288705
Nodes searched  : 2101599484
Nodes/second    : 7279401
 
I installed stockfish with brew :)
Using Stockfish 16
Code: 
  Chip:    Apple M2 Max
  Total Number of Cores:    12 (8 performance and 4 efficiency)
  Memory:    32 GB


Code: 
stockfish bench 512 12 26 default depth nnue

Total time (ms) : 110002
Nodes searched  : 1016341622
Nodes/second    : 9239301

Code: 
stockfish bench 1024 12 26 default depth nnue

Total time (ms) : 101730
Nodes searched  : 929432670
Nodes/second    : 9136269

Code: 
stockfish bench 2048 12 26 default depth nnue
 
Total time (ms) : 121228
Nodes searched  : 1086621797
Nodes/second    : 8963455

Code: 
stockfish bench 2048 12 26 default depth nnue


Total time (ms) : 119606
Nodes searched  : 1055641504
Nodes/second    : 8825991

Code: 
stockfish bench 8192 12 26 default depth nnue

Total time (ms) : 115477
Nodes searched  : 1051961539
Nodes/second    : 9109706
 
I know that you explicitly asked for M2 and M3 results, but I couldn't help trying it on my 8 core M1 Pro 14" MacBook Pro. Here are the results for Stockfish 14.1:

Code: 
bench 1024 8 26 default depth nnue

Total time (ms) : 180226
Nodes searched  : 1682432528
Nodes/second    : 9335126

Code: 
bench 512 8 26 default depth nnue

Total time (ms) : 182392
Nodes searched  : 1714745054
Nodes/second    : 9401426

The nodes per second seems pretty high compared to some of the results with newer CPUs here.


Edit:

I also tried compiling the latest version of Stockfish (dev-20240126-nogit) on my own machine and got different results:

Code: 
bench 1024 8 26 default depth nnue

Total time (ms) : 101629
Nodes searched  : 865702755
Nodes/second    : 8518265

Code: 
bench 512 8 26 default depth nnue

Total time (ms) : 90512
Nodes searched  : 795864255
Nodes/second    : 8792914
 
Last edited:
MichaelAC said:
Hi h1a8.
I tried ... but there is nothing "running" or happening.
  1. After downloading, there is "Stockfish 14.1 M1 pop-neon" in my download directory.
    --> What can/shall/must I do with this?
  2. Where do I have to enter the "bench 512 14 26 default depth nnue"
    --> I tried in Terminal ... resulting in the error message "bench: command not found"
Click to expand...
SjoukeW said:
I have a M2 Max for work. Happy to run your benchmarks. The file downloaded wanted to open in a text editor…
Click to expand...

To run the downloaded file, two things need to be done. I'm assuming that the file is in the Downloads folder here, but your path may vary. This is all done from the terminal.

First, you need to allow the application to bypass the Gatekeeper check for downloaded files:
Code: 
xattr -rd com.apple.quarantine ~/Downloads/Stockfish\ 14.1\ M1\ pop-neon

Then you need to change the permissions on it so it can be executed (u+x allows all users on the computer to run it):
Code: 
chmod u+x ~/Downloads/Stockfish\ 14.1\ M1\ pop-neon

Now the script can be run from the terminal:
Code: 
~/Downloads/Stockfish\ 14.1\ M1\ pop-neon

Once there you can type in the command to run the benchmark (changing 8 to match the number of cores on your CPU):
Code: 
bench 1024 8 26 default depth nnue
 
Last edited:
  • Like
Reactions: MichaelAC
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back