Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

32 GB of RAM in a Mac Pro

No problem. x86 and x86-64 (aka x64) systems are always little endian.

If you want big endian you need Power, SPARC (Itanium only sometimes big endian, like with HP-UX).
 
This Sun configuration has a 60 day free trial so you might want to check if they would fix the price at the special promotion level, which ends very soon, and is what makes this system attractive.
 
Downside of this Sun is that you would have to get some external storage, like this:

http://www.sun.com/storagetek/disk_systems/workgroup/2500/

But at least it uses 15000 RPM drives, and it also has RAID 3 and 1+0 (not just the dangerous 0, or the slow 1 and 5).


I don't know, maybe you can apply your symmetry tricks also to the file, so that it would fit in a couple of little internal disks.
 
What the F*%@?

ChrisA said:
OK so it's 1600 pounds or $3200. the price is nothing compared to the cost to write the application. What does a programer make per hour? Add payrol taxes and overhead. rent for the office space, insurance and so on and you can't get buy for under $100/hour Figure it takes at least 1,000 hours (half a man year) to get even something simple done and you are spending $100K. $3K for Ram is pocket change for a software development project. And believe me you "don't get nothing" for only 0.5 man years of development effort - maybe a prototype, proof of concept demo or rev 1.0 of a very simple application.

Look at some numbers. a very productive team can write about 1000 lines of code per person per month. That is fully debugged and documented and run through QC, a bug tracking system and in configuration control. Of course anyone can type faster but we are talking about production quality delved, non-beta code. Figure $100/hour and 160 hours per month to $16000/1000 means $16 per line. Now count the lines of code in project that are like yours. The PostgreSQL DBMS has about 800,000 lines My telemetry system is at about 100K lines now. You can look at some open source projects and count lines there. Bottom line is that $5K or $10K for a computer is trivial, you don't even see that in the "rolled up" budget. that 1000/month figure assumes a top talent motivated team.
Click to expand...


They weren't kidding when they said, "Men Are From Mars". :p
 
cube said:
There was a typo in the aggregated memory bandwidth of the Sun Fire: it's 85.6 GB/s (still 4 times bigger than the Mac Pro, so twice as much per core).

There will be some memory bandwidth improvements in the quad cores, among other things:
http://www.channelinsider.com/print_article/AMD+Unveils+Barcelona+QuadCore+Details/191008.aspx
Click to expand...

Yeah I was looking at that 856 GB/sec and thinking.... "whoah, how is that possible?" :)

As long as they are "same-endian" (can't believe we use Jonathan Swift to describe a computer standard!) I might go with the HP to crank the databases, but I would have to get familair with a good compiler for that environment.

And, since the thing has so much horsepower, I would have to make a version of the program for that platform too :)

I think with some clever #define statements I can make one source file for either OS, but now I would have to consult with an HP Gui expert.

Thanks for all of your informative posts, very, very helpful.
 
GothicChess.Com said:
Maybe it also helps that I have a 6'5" tall former hooters waitress as a Media Spokesperson :)

alexis_10.jpg
Click to expand...

giggity
 
cant you do something like the Folding@HOME program does. send out work units to computers to make the processes for you?

obviously that means making a program to talk to your server to retrieve work, and send it back when its done.
 
Willis said:
cant you do something like the Folding@HOME program does. send out work units to computers to make the processes for you?

obviously that means making a program to talk to your server to retrieve work, and send it back when its done.
Click to expand...

I have given that a great deal of thought over the past 6 months or so. I would LOVE to be able to do something like that! Chopping the database up into so many pieces would be a monumental undertaking.

Ultimately, I suppose, for me to compute the 7-piece database, I will have to do something like that.

But just half a year with some kick-butt hardware is way way too tempting to rewrite the project. :)
 
GothicChess.Com said:
As long as they are "same-endian" (can't believe we use Jonathan Swift to describe a computer standard!) I might go with the HP to crank the databases, but I would have to get familair with a good compiler for that environment.
Click to expand...

HP? OK, let's clarify (ignoring the old hardware):

Little endian:
PC
Mac/Intel
Linux/x86|Itanium
Sun/Intel|AMD

Big endian:
Mac/PowerPC
IBM/Power
Sun/SPARC
HP-UX/Itanium (I think)

If you have an old PowerPC Mac now, I think it makes more sense to get a cheap Intel Mac and convert your program to little endian, than to buy a big endian machine. The Mac Pro is little endian, so it's the same situation as with the Sun/Opteron we're talking about.

If you're just talking about the endianness of the file, a bit of the abstraction in the I/O code can make you endian-neutral for that.

My recommendation for a compiler would be to go with GCC, which you can use on basically anything (on some embedded projects it's even the only option). You can even use it on Windows, and it's the standard compiler for the Mac.
 
The batch program has a GUI? Unless you based it on some portable toolkit, the work is much more than just some #define's.
If you have a Windows GUI, you could compile your application with Wine on Solaris/x86, Linux/x86, Mac/Intel and other x86 Unixes.
 
Why is endian that important that it should decide what hardware you use? Surely once you've created your database you can convert it to whatever endian you want in a simple pass. Also can't you make your calls into the db endian independant?

b e n
 
BTW, the lowest retail price I've seen for an Opteron 8218 CPU (like on this Sun) is $1029.
 
To maximize memory bandwidth with the Opteron, you'll have to take some care.

There are some interesting links on NUMA at the end of this page (not just Windows, like the article):
http://developer.amd.com/articlex.jsp?id=106

This could also be useful:
http://www.digit-life.com/articles2/cpu/rmma-numa2.html

For Solaris, here is some information on MPO:
http://www.sun.com/software/solaris/performance.jsp
http://www.opensolaris.org/os/community/performance/numa/

There's an X4600 architecture diagram on page 14 here:
http://www.voltaire.com/download/casestudies/Sun%20TiTech%20Blueprint_Final.pdf
 
For your application I think one would have to look more at the bisection bandwidth (bandwidth of the minimum number of links one has to remove to partition the network into two equal halves). You can compute this by looking at the architecture diagram above. I think this would be something like 32 or 40 GB/s.
 
Regarding "endian-ness" -- I compute and write data in 4 byte chunks malloced as unsigned long integers, therefore the byte order would be different on thte different endian machines.

Converting 8.5 terrabytes of written data from one endian format to another is very time consuming and highly probable that an error would be introduced.

So, given that the Sun box and the new Macs are "same endian", I can crank the databases on the Sun and use it on a new Mac (both have yet to be purchased.)

The database generation phase is gui-less, but the game and the engine that uses the databases has a gui. So, since the HP would vastly outperform the Mac at playing the game (faster CPU, faster RAM, faster disk speed, etc.) I was thinking I might as well play it on the better hardware for even more amazing results.

This is "90% wishful thinking" at this point though, since I have not used the compiler nor ever written a single GUI for that hardware.
 
OK, I see. Sometimes you say HP when referring to the Sun.

Just remember this amazing price is valid only until the 7th May.
 
I think in the terabyte database he has the whole sequences of checkmate moves from each position, not just the next move in that direction.

I think he will have to wait a bit before he can distribute Gothic Vortex VI on a hard disk, or would the database be compressible to a couple TB, and then require the people to have a little array to decompress and play? Or can just the immediate move database be distributed and the terabyte db regenerated by the customer?
 
cube said:
I think in the terabyte database he has the whole sequences of checkmate moves from each position, not just the next move in that direction.

I think he will have to wait a bit before he can distribute Gothic Vortex VI on a hard disk, or would the database be compressible to a couple TB, and then require the people to have a little array to decompress and play? Or can just the immediate move database be distributed and the terabyte db regenerated by the customer?
Click to expand...

He has already stated that this is not for distribution to clients and that the actual program would be a much smaller size. I assume the reason he needs all this data is so his chess program can stand up well to playing against a human opponent.
 
so let me get this straight you need to permutate 6 chess pieces in every possible position on the board.... and you need 32 gb or ram? each record must be frickn huge!! id imagine you can trim some fat somewhere.
 
kellah said:
so let me get this straight you need to permutate 6 chess pieces in every possible position on the board.... and you need 32 gb or ram? each record must be frickn huge!! id imagine you can trim some fat somewhere.
Click to expand...

I don't think you understand.

To compute the entire 6-piece set you need to solve:

King + Queen + Queen vs. King + Queen + Queen
King + Queen + Queen vs. King + Queen + Chancellor
King + Queen + Queen vs. King + Queen + Archbishop
King + Queen + Queen vs. King + Queen + Rook
King + Queen + Queen vs. King + Queen + Bishop
King + Queen + Queen vs. King + Queen + Knight
King + Queen + Queen vs. King + Queen + Pawn

...

King + Queen + Chancellor vs. King + Queen + Chancellor
King + Queen + Chancellor vs. King + Queen + Archbishop
King + Queen + Chancellor vs. King + Queen + Rook
King + Queen + Chancellor vs. King + Queen + Bishop
King + Queen + Chancellor vs. King + Queen + Knight
King + Queen + Chancellor vs. King + Queen + Pawn

...

King + Queen + Archbishop vs. King + Queen + Archbishop
King + Queen + Archbishop vs. King + Queen + Rook
King + Queen + Archbishop vs. King + Queen + Bishop
King + Queen + Archbishop vs. King + Queen + Knight
King + Queen + Archbishop vs. King + Queen + Pawn


...

etc., etc., etc.,

And, there are 80 x 79 x 78 x 76 x 75 x 74 x 2 x 2 bytes for each database slice that does not involve pawns, and 80 x 79 x 78 x 76 x 75 x 60 x 2 x 2 if there is one pawn.

That is 64,042,602,624,000 bytes per non-pawn database.

Why do you think these databases are so small?
 
GothicChess.Com said:
Well, I have this chess-like program and it can announce Mate in 268 moves right now.
For me to go to the next stage, and solve every possible position from any permutation involving 6 pieces, I would need that boatload of RAM.
Click to expand...

This is a "classic" problem that has been beaten to death in the text books.
You seriously need to re-think and research the algorithm you are using. It
can be done without so much RAM.

Basically you are searching a large "tree". Like i said this problem has been
solved and analysed and re-hash and writen about more times then anyone
can count You do not need so much RAM.

In the worst case you can implement your own memory paging system and
move parts of the search tree in to RAM as required. The good thing is that
RAM access is 100% predictable so custom written paging can work well.
If you use asynchronus I/O you don't block while waiting on a the disk.

There is no need for so much RAM this has been done decades ago using
computers that were not able to address so much RAM

The reason you might need so much RAM is for some task were you can
not predict what memory is needed to by accessed next. For example a
DBMS can'r know what the next query will be. But this a a very predictable
"tree walk" and you should be able to swap in sub trees before they
are needed.

In fact if you have a fixed budget you would be best to spend it on CPU
power before you spend it on RAM.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back