Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

32 GB of RAM in a Mac Pro

ChrisA said:
If you are building something like that you really need to be on a different platform than Apple. Do the user interface on Apple if you like, but run the back-end on server class hardware. Applications like these tend to grow and with Apple if you are already on a MP and you need more, like 8 times more. There is nothing as you are already on the top of the line. But if you were running on 4 or 8 CPU Sun server you are running ona "low-end" server and there is room to grow.

One of the things the higher end server class hardware has is "fault tolerance". The Sun equipment will continue to run even with a failed CPU and failed RAM and with a failed disk controller. It will re-configure itself. Also you have multiple power supplys that can be swaped without bringing the system down. Also the Sun stuff is comparably priced to Apple. Sun as a company is very much like Apple. Solaris and Mac OS X share much in common.

Do you really need this much physical RAM? What is the algorithm you are implementing
Click to expand...

I already have implemented this, on a smaller scale, on another platform, I want to switch to OS X and foresake Windows because it is an abomination.

There is no "growth" that needs to be budgetted for, this is a one-shot deal. I need the RAM to solve prohibitively large chess-like endgames, and there is no way to get to the next level without grinding a disk into pulp if using a paging schema, which would take about 10 computer-years to finish. For example:

http://www.gothicchess.com/javascript_endings.html

You can see this checkmate in 268 moves that was already solved:

1. Qe2! Qg8+ 2. Ka7! Qg7+ 3. Ka6! Qa1+ 4. Kb7 Qb2+ 5. Kc6 Qf6+ 6. Kc5 Qf5+ 7. Kd6 Qf8+ 8. Kd7! Qf7+ 9. Qe7 Qd5+ 10. Qd6! Qb7+ 11. Ke6 Qe4+ 12. Kf6 Qf3+ 13. Ke5 Qe2+ 14. Kf5 Qf3+ 15. Qf4 Qd5+ 16. Kf6 Qc6+ 17. Ke5 Qb5+ 18. Ke4 Qb1+ 19. Ke3 Qe1+ 20. Kd3 Qc1 21. Ke2 Qc2 22. Ke3 Qc1 23. Kf2 Qc2 24. Kg3 Qg6+ 25. Kf3 Qd3+ 26. Qe3 Qf5+ 27. Kg3 Qi5+ 28. Kg4 Qi6+ 29. Kf4 Qf6+ 30. Kg3! Qj6+ 31. Kf3! Qi6+ 32. Kf2 Qi5+ 33. Ke2 Qb5+ 34. Ke1 Qb1+ 35. Kf2 Qa2 36. Kg3 Qg8+ 37. Kf4 Qj8+ 38. Kf5 Qf8+ 39. Ke6 Qc8+ 40. Kd6 Qd8+ 41. Kc6 Qa8+ 42. Kb5 Qb7+ 43. Ka5 Qa8+ 44. Kb4 Qb7+ 45. Ka3 Qa6+ 46. Kb2 Qb5+ 47. Qb3 Qe5+ 48. Qc3! Qb5+ 49. Ka3 Qa6+ 50. Kb4 Qb6+ 51. Kc4 Qc6+ 52. Kd4 Qd6+ 53. Ke4 Qg6+ 54. Kd5 Qg8+ 55. Kc5 Qc8+ 56. Kd4 Qg8 57. Qc6+ Ki2 58. Qd7+! Ki3 59. Kc3 Qa2 60. Qd6 Qa1+ 61. Kc4 Qc1+ 62. Kb5 Qb1+ 63. Qb4 Qd3+ 64. Kc6! Ki2 65. Qi4+ Kh2 66. Qh5+! Kg1 67. Qd5! Qa6+ 68. Kd7! Qa7+ 69. Ke6! Qb6+ 70. Kf5 Qf2+ 71. Ke5 Qe2+ 72. Kf6 Qf2+ 73. Ke7 Qa7+ 74. Qd7 Qc5+ 75. Qd6! Qg5+ 76. Ke6 Qg6+ 77. Kd7! Qg4+ 78. Kc6 Qa4+ 79. Kb7 Qe4+ 80. Kb6 Qb1+ 81. Ka6 Qa2+ 82. Kb7 Qf7+ 83. Kc6 Qf3+ 84. Qd5 Qf6+ 85. Kc5 Qa6 86. Kb4! Kf2 87. Qb5! Qe6 88. Kc3! Kf3 89. Qd3+ Kg2 90. Qc4! Qe1 91. Qg4+ Kh2 92. Qf4+ Kg2 93. Kb2 Qe2 94. Qi4+ Kh2 95. Qh4+ Ki2 96. Kc2 Kj3 97. Kc3 Qf3+ 98. d3 Qe3 99. Qd4 Qh6 100. Qd5 Qe3 101. Kc4 Qa7 102. Qc5 Qa2+ 103. Kc3! Qa1+ 104. Kd2 Qa2+ 105. Ke3 Qe6+ 106. Kd4 Qg4+ 107. Kc3! Qg3 108. Qf5 Qc7+ 109. Kd2 Qd6 110. Ke3 Qh6+ 111. Ke4 Qc6+ 112. Qd5 Qg6+ 113. Kf3 Qf6+ 114. Ke2 Qj2+ 115. Ke3 Qe7+ 116. Kd4 Qh4+ 117. Kc5 Qe7+ 118. Kb6 Qf6+ 119. Kb5 Qb2+ 120. Kc6 Qc3+ 121. Kd7 Qg7+ 122. Ke8 Qh8+ 123. Ke7 Qh4+ 124. Kd7 Qh3+ 125. Kc7 Qg3+ 126. Kc6 Qg6+ 127. Kc5 Qa6 128. Qf3+ Ki4 129. Qg4+ Ki5 130. Qd7 Qa5+ 131. Qb5! Qc7+ 132. Kd5 Qd8+ 133. Ke6+! Ki4 134. Qd5 Qg8+ 135. Ke5 Qg5+ 136. Kd4 Qg7+ 137. Kc4 Qc7+ 138. Qc5 Qf7+ 139. Kc3 Qf3 140. Qd6 Kh3 141. Qe6+! Kg3 142. Kc4 Qf1 143. Qg8+! Kh4 144. Qd5! Kg4 145. Kb5! Kg3 146. Qg5+ Kh2 147. Qe3 Kg2 148. Kb4 Qb1+ 149. Kc5! Qc2+ 150. Kd6! Qc8 151. d4 Qd8+ 152. Kc5 Qc7+ 153. Kb4 Qb6+ 154. Kc3 Qa5+ 155. Kc4 Qa6+ 156. Kc5 Qa5+ 157. Kc6 Qa8+ 158. Kd7 Qd5+ 159. Ke7 Qb7+ 160. Kf8 Qb8+ 161. Kg7 Qc7+ 162. Kg6 Qc6+ 163. Kg5 Qa8 164. Qe2+ Kg1 165. Qe5 Qj8+ 166. Kg6 Qj6+ 167. Kf7 Qj3+ 168. Ke7 Qi3+ 169. Kd7 Qh3+ 170. Kc7 Qc3+ 171. Qc5 Qg3+ 172. Qd6 Qc3+ 173. Kd7 Qh3+ 174. Kc6 Kh1 175. d5 Qh8 176. Kb5 Qh7 177. Kb6 Qh8 178. Qe7 Qb2+ 179. Kc6 Qc2+ 180. Qc5 Qe4 181. Kb6 Qb1+ 182. Qb5 Qg6+ 183. Kc5 Qf5 184. Qe2 Ki1 185. Qe1+ Kh2 186. Qh4+ Ki1 187. Kd4 Qf1 188. Qg3+ Kj2 189. Qj6+ Ki1 190. Qi6+ Kj2 191. d6 Qa1+ 192. Kd5 Qa8+ 193. Ke5 Qb8 194. Qj5+ Ki2 195. Qg2+ Ki1 196. Qc6 Qh8+ 197. Ke6 Qh3+ 198. Ke7 Qh4+ 199. Kd7 Qh7+ 200. Kc8 Qg8+ 201. Kc7 Qg3 202. Qc4 Ki2 203. Qb4 Qe5 204. Kc6 Qe6 205. Qb2+ Ki1 206. Qc1+ Ki2 207. Qi7+ Kj3 208. Qj7+ Ki4 209. Kc7 Qc4+ 210. Kd8! Qg8+ 211. Kd7 Qg3 212. Qe7 Kj4 213. Qj2+ Ki5 214. Qd2 Ki6 215. Kc6 Qf3+ 216. Kc7 Qf5 217. Qe2+ Kj6 218. Qc4 Qa5+ 219. Kb7 Qf5 220. Qc6 Qb1+ 221. Kc7 Qi1 222. Qd5 Ki6 223. Qf5 Kj6 224. Kd7 Qg3 225. Ke6 Qb3+ 226. Kf6 Qi3+ 227. Kf7 Qi7+ 228. Kf8 Qh8+ 229. Ke7 Qi7+ 230. Qf7 Qc1 231. Qf6+ Ki7 232. d7 Qc5+ 233. Ke8 Qb5 234. Qc3 Ki8 235. Ke7 Qg5+ 236. Kd6 Qh6+ 237. Kc7 Qf4+ 238. Kb7 Qe4+ 239. Kb6 Qb1+ 240. Kc7 Qi1+ 241. Kc8 Qi2 242. Qd3+ Kj8 243. Qg6 Qh3 244. Qg8+ Ki7 245. Qg7+ Kj6 246. Qf6+ Ki5 247. Kc7 Qh7 248. Qd4 Ki6 249. Kb7 Kj7 250. Ka8 Qh1+ 251. Ka7 Qh7 252. Qd1+ Ki7 253. Qc1+ Kj6 254. Qc7 Qh8 255. d8Q Qa1+ 256. Kb8 Qb1+ 257. Qb6+ Qxb6+ 258. Qxb6+! Ki5 259. Qb4 Ki6 260. Qi4+ Kh6 261. Kc7 Kg5 262. Kc6 Kf6 263. Qh5 Kg7 264. Kd5 Kf6 265. Kd6 Kg7 266. Ke6 Kg8 267. Kf6 Kf8 268. Qh8# 1-0

That was just a 5-piece ending on the 80-square board featuring King + Queen + Pawn vs. King + Queen.

"How big" is this database?

There are 60 legal placements for the white pawn.
There are 79 squares remaining for the white queen.
There are then 78, 77, and 76 squares remaining where you can place the black queen, white king, and black king. Some of these are not allowed due to the rules of who can move when one side is in check, but disregard this for the moment. And, lastly, either side can move from any of these arrangements, so you have to mutiply the count by 2.

For position count, you get: 60 x 79 x 78 x 77 x 76 x 2 = 4,327,202,880

But, since the longest win is 268 moves = 268 + 267 = 535 "plies" if you count moves for both sides, you need 2 bytes per position to store the distance to win.

So 4,327,202,880 x 2 = 8,654,405,760 bytes = more than 8 GB of RAM.

Now, I want to solve the 6-piece database, meaning you can basically multiply this by 75, which would be 600 GB. It is really not that "harsh", with symmetry tricks I can reduce this by a factor of 4 meaning 150 GB of RAM would be needed.

It just so happens, with 32 GB of RAM, I could load about 20% of the DB in RAM at a time, and this would greatly reduce the disk access if I implement a most-recently-seen-position buffer scheme.
 
Blue Velvet said:
Why do you want or need to calculate all these permutations and end-games? Just curious.
Click to expand...

Because it's cool man :)

Actually, as the program searches, if it encounters any of the 300 billion positions I have already solved, it can return perfect information back down to the root of the tree.

There are billions of King + Queen + Pawn vs. King + Queen positions where the side that is up a pawn CANNOT win -- the weak side can draw.

So, in a "jam", if the program was down a pawn, it could head for such a position since the score "0" would be returned. Other programs would still think they were ahead by 1.0 pawns, but Gothic Vortex would know better.

Also, some endgames are so complex, they CANNOT be won with a search. Gothic Vortex would be able to play the correct move for any position INSTANTLY, just looking it up in the database, and making the best possible play.

Even with 15 pieces on the board, the 5-piece database is probed often enough that it can make aanouncements such as "Mate in 276" since the program can force a series of trades getting it down to the reduced material in the endgame database.

Any other computer program would think it was "just down a pawn" and have no idea its impending doom was inevitable.

In the 2004 World Championship, our program went 14-0 to take all the marbles. You can read about it here:

http://www.chessville.com/GothicChess/ComputerWorldChampionships.htm
 
And I forgot to mention, in that huge move list showing the 268 move checkmate, you see some moves ending in !

Every ! means that is the only move to win.

So, if you play ANY other move, you have just handed the draw to your opponent, or even worse, though less likely, you can still lose such an ending.
 
orangemacapple said:
would one of the new flash hdd's work and use virtual memory?
Click to expand...

I'd imagine the hard drive bandwidth is still to low. For example, a MacBook Pro can retrieve information from 2 gb of RAM in 1 second, wheras no matter what speed the hard drive is, you'll be limited to something 'slightly' less than that.
 
If you've got that much money to burn to simulate some chess variant well good on you but I think you need to re-evaluate your programming approach bigtime, more ram is always nice for simulation work but you shouldn't *need* it under any circumstances. If you do, you've got one huge memory leak somewhere. Trust me, fluid dynamics, software x-ray optics and economic modelling are way more taxing than chess simulation and they can be done on far lower-grade hardware.

Killyp - you can ping google in well under a second on a broadband connection, you're off on your latency by a few factors there.
 
jaguarx said:
If you've got that much money to burn to simulate some chess variant well good on you but I think you need to re-evaluate your programming approach bigtime, more ram is always nice for simulation work but you shouldn't *need* it under any circumstances. If you do, you've got one huge memory leak somewhere. Trust me, fluid dynamics, software x-ray optics and economic modelling are way more taxing than chess simulation and they can be done on far lower-grade hardware.

Killyp - you can ping google in well under a second on a broadband connection, you're off on your latency by a few factors there.
Click to expand...

Well, if you were right, I would agree with you :)

There is no memory leak.

There is no other way to solve these endings unless they are in a huge RAM buffer, because trillions of positions are being solved, and, as everyone knows, RAM access is order of magnitudes faster than paging off of the disk.

I have published a few papers for the International Computer Games Association Journal, one of which appeared in an artificial intelligence book.

You can download them here:

http://www.GothicChess.com/7_piece.pdf

http://www.GothicChess.com/80.pdf

The first paper is about the longest win ever solved in the game of checkers: 253 moves if you count the play for both sides. I had to solve roughly 23 billion checkers positions (in RAM) to compute that.

The second paper is about the chess variant played on 80 squares, and how chess ALMOST evolved into this in the 1920's. Also, I figured out how to compute the value of the chess pieces on boards of any size (the "Holy Grail" of chess mathematics) and this is one of the reasons they were eager to publish the paper.

NOTE: The program only needs the large amount of RAM to generate the databases. I can run the Gothic Chess program that reads them with a mere fraction of the RAM.

Regarding "money to burn": Gothic Chess has been around since 1998. Since that time I have sold over 50,000 of these boards to many different countries. If you google "Gothic Chess" you can find stuff in German, French, Russian, Italian, etc. etc.

Maybe it also helps that I have a 6'5" tall former hooters waitress as a Media Spokesperson :)

alexis_10.jpg
 
GothicChess.Com said:
There is no other way to solve these endings unless they are in a huge RAM buffer, because trillions of positions are being solved, and, as everyone knows, RAM access is order of magnitudes faster than paging off of the disk.
Click to expand...

Have you considered a huge RAID array? i know it won't be as fast as ram... but could it provide an alternative? With a dedicate raid box, you should be able to get better than 600MB/sec with a quad channel fiber connection... and all the storage space you could ever use.

just a thought... i've got raid on the brain lately as i config my system.
 
brooker said:
Have you considered a huge RAID array? i know it won't be as fast as ram... but could it provide an alternative? With a dedicate raid box, you should be able to get better than 600MB/sec with a quad channel fiber connection... and all the storage space you could ever use.

just a thought... i've got raid on the brain lately as i config my system.
Click to expand...

Well, the computation will take 1.5 computer years if it is 100% RAM resident. With my buffering solution, it will take about 2.2 computer years. Since I have a limit to my patience, I may just lease 4 machines (the task is parallelizable) and have everything in about 8 months' time.

Now, scale that by the speed ratio of a RAID disk to RAM, and you see why it becomes impossible. Even with a 10:1 ratio (and it will be over 1000:1, way over) you are talking about 22 computer years with a RAID-only approach.

I just have a boatload of code to crank. The project has over 200,000 lines of source code as is, and I am about 65,000 lines away from completion.
 
You don't need the ram, it'd just be nice, thus my point. Take a look at Amazon's EC2 service, you're better off buying cycles than you are leasing hardware for something like this and it doesn't look like something that's too hard to paralellize. Of course that doesn't generate you threads on major sites with your product splattered all over them. If you do want to lease hardware you're better off with something like a small cluster of Sun Fire X4200s, more cycles for your buck. I'm not sure how you're coming up with those 1.5/2 year figures though.

This thing written in c? perl?
 
jaguarx said:
You don't need the ram, it'd just be nice, thus my point. Take a look at Amazon's EC2 service, you're better off buying cycles than you are leasing hardware for something like this and it doesn't look like something that's too hard to paralellize. Of course that doesn't generate you threads on major sites with your product splattered all over them. If you do want to lease hardware you're better off with something like a small cluster of Sun Fire X4200s, more cycles for your buck. I'm not sure how you're coming up with those 1.5/2 year figures though.

This thing written in c? perl?
Click to expand...

Will all due respect, you don't understand the process :)

If the database is not probed in RAM, it turns a "2-computer-year" solution into one that requires decades. You can have the fastest computer in the world cranking on it, it will do you no good if you have to read the database off of disk.
 
There's a 25-year anniversary sale at Sun:
http://www.sun.com/emrkt/25sale/index.jsp

I would go for an semi entry-level Sun Fire X4600 M2 with 8 dual-core Opterons for now, and swap the CPUs later in the year for quad-cores (it uses CPU modules, so one cannot just get the cheapest version with 4 CPUs).
It takes a maximum of 128GB RAM.

But that only if one would only need to change the CPUs later, not the whole modules, otherwise I imagine the upgrade could be expensive.
You'll have to ask Sun.
 
32GB for the Mac will cost you $8000
64GB (8 CPUs) for the Sun will cost you $10400 (forget about 128GB, it would cost you $40000). But the base 8-CPU configuration already uses 2GB DIMMs and leaves 16 slots free, so you would just have to add 32GB for $5200.

8-core Mac Pro with 32GB and bad memory bandwidth (21.3 GB/s): $12000
16-core Sun Fire with 64GB and excellent memory bandwidth (85.6 GB/s): $25200
(upgradeable to 32-core or 128GB).
 
cube said:
32GB for the Mac will cost you $8000
64GB (8 CPUs) for the Sun will cost you $10400 (forget about 128GB, it would cost you $40000). But the base 8-CPU configuration already uses 2GB DIMMs and leaves 16 slots free, so you would just have to add 32GB for $5200.

8-core Mac Pro with 32GB and bad memory bandwidth (21.3 GB/s): $12000
16-core Sun Fire with 64GB and excellent memory bandwidth (856 GB/s): $25200
(upgradeable to 32-core or 128GB).
Click to expand...

Are Suns and the new Macs both "Big endian"? If so, I can use the sun to crank out the databases then use the Mac to play the game.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back