llvm and ld64 for ppc64 on Leopard

[unilock]

I managed to get llvm-powerpc, ld64-97 +llvmppc, and cctools +llvmppc installed from your MacPorts repository. So there's that, at least.

Unfortunately, many ports (or their dependencies) still want gcc7, which appears to be impossible to get working... is there a way to force all ports to use gcc10-bootstrap as their compiler by default?
Is that even a good way to go about it...?

 

[barracuda156]

I managed to get llvm-powerpc, ld64-97 +llvmppc, and cctools +llvmppc installed from your MacPorts repository. So there's that, at least.

Unfortunately, many ports (or their dependencies) still want gcc7, which appears to be impossible to get working... is there a way to force all ports to use gcc10-bootstrap as their compiler by default?
Is that even a good way to go about it...?

I think gcc7 should be completely avoided, especially on ppc64, since it has unfixed bugs plus does not support some features which are needed too often now (say, filesystem, or C++20).

In principle, gcc10-bootstrap is simply the “normal” gcc, like it is built outside of MacPorts (without extracting libgcc into a standalone thing). So there is no reason why it should not work as the main compiler.
Now, MacPorts environment assumes a different setup, so here and there some changes will be needed.

To really force it MacPorts-wide, you probably need to insert related code into tlc files of the base itself. Less intrusive approach would be to make a portgroup and use that.
Take a look where MacPorts picks compilers according to C/C++ standards. We only need modern gcc with C/C++11 and higher. So there the choice should be tweaked to use gcc10-bootstrap.

However, at the moment at least I believe the correct thing is to fix building gcc14 for ppc64, and then let MacPorts pick compilers normally. (Well, 10.5 needs minimal changes to use gcc14 instead of gcc7, but those are trivial, just pick them from my repo branch.)

 

[barracuda156]

I managed to build gcc10-bootstrap for ppc64 overnight, but it doesn't look like I can use it to compile llvm-7.0 directly, since it (or one of its dependencies) depends on ld64-97, which in turn requires LLVM in the first place...

Would my best bet be trying to use your llvm-powerpc port at this point?

Alternatively, for going the "circular dependency" route, I would need to:

1. Compile all of llvm-3.3 and ld64-97's dependencies as +universal
2. Install those two packages for ppc
3. Install gcc7 as +universal
4. Install llvm-3.7 for ppc64
5. Re-install ld64-97 for ppc64

Correct?

(as an aside, it's sudo port install <package> build_arch="ppc", not sudo port install <package> build.arch="ppc")

This is all very confusing 😅

Any success ever since, what is the current status?

 

[unilock]

Any success ever since, what is the current status?

Unfortunately, I haven't done much more with my PPC system since my last update. It's only a hobby for me, but it's quickly becoming rather overwhelming 😅

Maybe I could just symlink gcc10-bootstrap to wherever MacPorts expects gcc7 to be? Though I feel like that may have unexpected consequences...

 

[barracuda156]

Unfortunately, I haven't done much more with my PPC system since my last update. It's only a hobby for me, but it's quickly becoming rather overwhelming 😅

Maybe I could just symlink gcc10-bootstrap to wherever MacPorts expects gcc7 to be? Though I feel like that may have unexpected consequences...

I don’t think symlinking gonna work (and certainly will leave fortran broken, since gcc10-bootstrap does not build it at all), but you could clone the system and make an experiment
Keep in mind, MacPorts base expects libgcc port to provide the runtime for gcc compilers. gcc10-bootstrap is not built that way, it has everything inside it. So while compiler should work, you will probably need to replace libgcc with a stub port (to cheat MacPorts). Then just try and see what happens.

I think we should rather fix it properly. That worked, as a matter of fact, when gcc11 was the main compiler. Unfortunately, I nuked everything libgcc* I had, and apparently also cctools and ld64. I have a prebuilt llvm5 and gcc11, both as ppc+ppc64, but without libgcc it is unusable. I might have old portfiles somewhere, but I wouldn't really count on that.
From a quick look into gcc11 portfile which I used to build it for ppc+ppc64 it seems that the only two significant changes are a block to use gcc10-bootstrap and disabling jit (this may be important; for ppc it builds fine, perhaps not for ppc64 and therefore universal). Here is the portfile extracted from the tarball:

 

[pc297]

Quick question, can ppc64 and ppc ports co-exist in a ppc-only macports installation (built with build_arch ppc) or would said installation would have to be started from scratch with build_arch ppc64 ? Is universal_archs ppc64 ppc then required to allow ppc ports?

And, finally, does anyone here have an unofficial repo for ppc64 ports ?

 

[barracuda156]

Quick question, can ppc64 and ppc ports co-exist in a ppc-only macports installation (built with build_arch ppc) or would said installation would have to be started from scratch with build_arch ppc64 ? Is universal_archs ppc64 ppc then required to allow ppc ports?

And, finally, does anyone here have an unofficial repo for ppc64 ports ?

1. Yes, in fact any combo of archs can co-exist, MacPorts do not confuse those. You can also switch the main build arch on the go (either on command line or in settings), turn +universal on/off etc. To use one arch port for a build for another arch use `depends_skip_archcheck` option (for example, it does not matter what arch xz or cmake are, as long as they are build dependencies and nothing links to them).

What cannot co-exist is C++ runtime: it should either be `libstdc++` or `libc++`, but not a mixture of the two.

2. You can start one
I could probably sort out the toolchain once my 10.6 ppc fork is finalized, but got no time to build 10k ports for ppc64 in addition to maintaining them on ppc.

 

[pc297]

1. Yes, in fact any combo of archs can co-exist, MacPorts do not confuse those. You can also switch the main build arch on the go (either on command line or in settings), turn +universal on/off etc. To use one arch port for a build for another arch use `depends_skip_archcheck` option (for example, it does not matter what arch xz or cmake are, as long as they are build dependencies and nothing links to them).

What cannot co-exist is C++ runtime: it should either be `libstdc++` or `libc++`, but not a mixture of the two.

2. You can start one
I could probably sort out the toolchain once my 10.6 ppc fork is finalized, but got no time to build 10k ports for ppc64 in addition to maintaining them on ppc.

Thank you very much, I did find your repo for darwin10 btw, great to have the binaries for SL! If I do start to compile for ppc64 I might do quick repo for binaries like @doctor_dog did for darwin9 at http://143.198.135.172:80 (great work).

For compilation purposes, rather than my Quad, I do have an 8-core power8 machine at home running Debian (with qemu and a leopard image installed), could try 10A190 or educovas's 10.6.8, should be much faster than the Quad let alone qemu under intel, (it's running ppc64el Debian but the hypervisor does support running qemu natively in big endian via the MSR bit so it will basically be native ppc and ppc64 compilation) will let you know

 

[pc297]

Would anyone happen to have darwin9 tar.bz2s of openal-soft by any chance? It's one of the few packages that I cannot compile for the life of me (compilation fails at 90%, even using the latest port)...

Thanks!

 

[barracuda156]

Would anyone happen to have darwin9 tar.bz2s of openal-soft by any chance? It's one of the few packages that I cannot compile for the life of me (compilation fails at 90%, even using the latest port)...

Thanks!

Could you share the log? It should be quicker just to fix it.

 

[pc297]

Could you share the log? It should be quicker just to fix it.

There it is, I looked at it more closely and it unfortunately appears to be Leopard-related, it's from coreaudio.cpp and it's mostly about AudioComponent and related classes, which I think you tried to address in miniaudio (issue 898)

It stops at 78% (latest commit), I think it had gone further with a previous version; tried with gcc-11, 12, 13 and 7

Thanks a lot in any case!

 

[barracuda156]

There it is, I looked at it more closely and it unfortunately appears to be Leopard-related, it's from coreaudio.cpp and it's mostly about AudioComponent and related classes, which I think you tried to address in miniaudio (issue 898)

It stops at 78% (latest commit), I think it had gone further with a previous version; tried with gcc-11, 12, 13 and 7

Thanks a lot in any case!

A usual way to address such issues, as long as an immediate fallback is unavailable (some of Core Audio stuff was just renamed), is to look at commit history with git blame, find where a problematic code was introduced and conditionally revert that commit. Often it is about some bells and whistles which are unnecessary anyway.
Another possibility is to see if Core Audio is actually necessary is the first place (it may be, but if openal can use portaudio, for example, then Core Audio could be disabled).
In the worst case you can always revert to some older version which builds fine. In most cases you do not really need the latest version of a given port for its dependents to work.

P. S. I have noticed that older macOS are pegged to 1.23.1 now, so I will see what can be done re 10.5 along with getting 1.24.1 building.

UPD. Core Audio was added in 1.21.0: https://github.com/kcat/openal-soft/commit/d6c8bb35b4f9af60cd1dd43c35d530937c95a3ef
And the breakage for 10.5 added in https://github.com/kcat/openal-soft/commit/21c84bcd96a2e457b446e53d3287878e32fc77e0
So reverting that may fix it.

 

[barracuda156]

There it is, I looked at it more closely and it unfortunately appears to be Leopard-related, it's from coreaudio.cpp and it's mostly about AudioComponent and related classes, which I think you tried to address in miniaudio (issue 898)

It stops at 78% (latest commit), I think it had gone further with a previous version; tried with gcc-11, 12, 13 and 7

Thanks a lot in any case!

Could you try this and then share a log? https://github.com/macports/macports-ports/pull/27031
It will probably fail, but I wanna see what else needed.

 

[doctor_dog]

Could you try this and then share a log? https://github.com/macports/macports-ports/pull/27031
It will probably fail, but I wanna see what else needed.

You and @pc297 may have gotten farther than I, but for what it's worth the last time I tried to build openal-soft it was throwing errors related to libdispatch (which I believe is only available on 10.6+). I don't know if there is a specific version you are looking for, but I generally have luck with audio ports going back to ~2013/2014 era commits.

 

[barracuda156]

You and @pc297 may have gotten farther than I, but for what it's worth the last time I tried to build openal-soft it was throwing errors related to libdispatch (which I believe is only available on 10.6+). I don't know if there is a specific version you are looking for, but I generally have luck with audio ports going back to ~2013/2014 era commits.

That issue I fixed long back, there is a patch in the port.

 

[doctor_dog]

Ah! @barracuda156 you are too quick for me, lol. It had been a while since I tried to build it (I can't even recall what it was a dependency for I was trying to build), thank you for fixing that. It seems I have a lot of ports to re-try, you've been busy my friend.

 

[barracuda156]

Just bumped into my comment here: https://trac.macports.org/ticket/46801#comment:16

I guess we need to try ld64 with llvm34 build for ppc (32-bit) to build gcc for ppc64. MacPorts has an option to skip archcheck, so that the build won’t complain on mismatch.

 

[unilock]

What a confusing situation this is for the layman...!

Perhaps it's too much to ask, but would it be feasible to create some sort of simple guide to set up a functional GCC, ld64, LLVM, etc. from scratch on ppc / ppc64 with MacPorts, such that all (or most) ports can be compiled without having to muck about with portfiles to switch between compilers and the likes? Or is something like that completely out-of-reach with the current state of MacPorts on PowerPC?

My original end goal was to compile `openjdk8-powerpc` and a few other more modern utilities, but the path to get there has proven to be rather complicated and difficult to follow...

 

[barracuda156]

What a confusing situation this is for the layman...!

Perhaps it's too much to ask, but would it be feasible to create some sort of simple guide to set up a functional GCC, ld64, LLVM, etc. from scratch on ppc / ppc64 with MacPorts, such that all (or most) ports can be compiled without having to muck about with portfiles to switch between compilers and the likes? Or is something like that completely out-of-reach with the current state of MacPorts on PowerPC?

My original end goal was to compile `openjdk8-powerpc` and a few other more modern utilities, but the path to get there has proven to be rather complicated and difficult to follow...

To begin with, is ppc64 a hard requirement or not? If yes, then there is no easy way, AFAIK. I can make a blind guess that forcing ld64 to build for ppc32 and then using it to compile for ppc64 may work, but that is untested by anyone, as of now.

If you are fine with ppc, install 10.6.8 and my PPCPorts, and mostly everything just works.
If you need Leopard, then for ppc more or less it should be functional with standard MacPorts. If something fails, please report that on Trac, and you may get help.

P. S. By the way, I have no idea if openjdk will build for ppc64, and I would rather assume that it won’t. It has been only built and tested on ppc, AFAIK.

 

[unilock]

@barracuda156 Thank you for the insight! I always assumed that ppc64 had some performance benefit over ppc, but it certainly seems like whatever benefit there may be isn't worth the trouble.

Thank you for all your work on keeping these old systems maintained

 

[barracuda156]

@barracuda156 Thank you for the insight! I always assumed that ppc64 had some performance benefit over ppc, but it certainly seems like whatever benefit there may be isn't worth the trouble.

Performance benefits may or may not be there (I never tried to run comparative tests, but this is what I have read at least). There should be some benefits in terms of some specific software compiling which does not compile for 32-bit, but the reverse will be more prevalent, I believe (not due to any inherent deficiency of ppc64, of course, but simply because it was assumed that builds are for 32-bit powerpc). Some ports, including languages, explicitly support only 32-bit version, so to make them work on ppc64 non-trivial effort will be needed. 10.6.8 does not support ppc64, so that is limited to 10.5.8, which is not great. Finally, nothing of ppc64 stuff can work on portable hardware, so perhaps it is a very small niche which can benefit from it. Overall, yes, I think it is a hardly justifiable amount of work with uncertain gains, even in principle (i.e. if the work will be successful to begin with).

 

[GA204]

One huge benefit of having 64-bit binaries (at least on G5) is that you can build projects that need more than 3 GB memory for the linker stage. The standard XCode LD and LD64 on OS X 10.4 seem to be 32-bit and both have given me a vm_alloc error when I recently tried to link the existing TenFourKit (with some code modifications to make it compile with the standard XCode gcc-4.0).

Currently, I'm working on manually building a 64-bit PPC LD binary for OS X 10.4 (using MacPorts patches and Portfile as a guide but changing the source code where needed to get it to compile in XCode on Tiger) so I can get TenFourKit to link, and then possibly look at the 64-bit PPC LD64. Hopefully, between all of us, we can get ppc64 LD64 working.

 

[barracuda156]

One huge benefit of having 64-bit binaries (at least on G5) is that you can build projects that need more than 3 GB memory for the linker stage. The standard XCode LD and LD64 on OS X 10.4 seem to be 32-bit and both have given me a vm_alloc error when I recently tried to link the existing TenFourKit (with some code modifications to make it compile with the standard XCode gcc-4.0).

Currently, I'm working on manually building a 64-bit PPC LD binary for OS X 10.4 (using MacPorts patches and Portfile as a guide but changing the source code where needed to get it to compile in XCode on Tiger) so I can get TenFourKit to link, and then possibly look at the 64-bit PPC LD64. Hopefully, between all of us, we can get ppc64 LD64 working.

I am not sure this is the correct conclusion. I you mean 32-bit builds, they are supposed to use ppc slice of the binary, so whether ppc64 is present or not is as much relevant as presence of x86_64. If you mean ppc64 builds, then it seems that ld does not always work correctly if built for ppc64, and the issue is different.

It should be testable with NodeJS build, which fails at linking with ppc32. (Can’t do it now though since it gonna require 10.5.)

Why don’t you just use gcc-4.2 though?

 

[GA204]

I am not sure this is the correct conclusion. I you mean 32-bit builds, they are supposed to use ppc slice of the binary, so whether ppc64 is present or not is as much relevant as presence of x86_64. If you mean ppc64 builds, then it seems that ld does not always work correctly if built for ppc64, and the issue is different.

It should be testable with NodeJS build, which fails at linking with ppc32. (Can’t do it now though since it gonna require 10.5.)

Why don’t you just use gcc-4.2 though?

The bug I'm encountering is that stock XCode LD fails to allocate enough memory to link the WebCore component of TenFourKit. The error message indicates that this is happenening at precisely the 3 GB mark. This is despite there still being plenty of free physical RAM (and virtual memory).

So, I'm assuming this may be a Mac OS X 32-bit memory space limitation. Even though the maximum addressable space for 32-bit should be 4 GB, there is often a general memory limit for programs before this. For instance, in 32-bit Windows programs, such a memory limit occurs at 2 GB for most user-land programs, except those few which are compiled to be "Large address aware".

Since Tiger is supposed to support 64-bit user-land address space and I'm compiling on a G5, I'm assuming the problem is with LD being 32-bit. However, I'm only in the early stages and it could just as easily be a 32-bit dynamic library it's using that's causing the problem or even something completely different. Nonetheless, my current plan is to try and get a 64-bit version of LD working to see if that may fix the problem or improve my workflow.

As for GCC-4.2, I did try to install apple-gcc42 +gpl3 with MacPorts a couple of months ago, however, MacPorts failed to find one of the patches in the repositories. I may try apple-gcc-40 or tinkering around with trying to manually compile apple-gcc-42, but for now I've just been lazy and stuck with the stock version of GCC in Xcode.

 

[barracuda156]

The bug I'm encountering is that stock XCode LD fails to allocate enough memory to link the WebCore component of TenFourKit. The error message indicates that this is happenening at precisely the 3 GB mark. This is despite there still being plenty of free physical RAM (and virtual memory).

So, I'm assuming this may be a Mac OS X 32-bit memory space limitation. Even though the maximum addressable space for 32-bit should be 4 GB, there is often a general memory limit for programs before this. For instance, in 32-bit Windows programs, such a memory limit occurs at 2 GB for most user-land programs, except those few which are compiled to be "Large address aware".

Since Tiger is supposed to support 64-bit user-land address space and I'm compiling on a G5, I'm assuming the problem is with LD being 32-bit. However, I'm only in the early stages and it could just as easily be a 32-bit dynamic library it's using that's causing the problem or even something completely different. Nonetheless, my current plan is to try and get a 64-bit version of LD working to see if that may fix the problem or improve my workflow.

As for GCC-4.2, I did try to install apple-gcc42 +gpl3 with MacPorts a couple of months ago, however, MacPorts failed to find one of the patches in the repositories. I may try apple-gcc-40 or tinkering around with trying to manually compile apple-gcc-42, but for now I've just been lazy and stuck with the stock version of GCC in Xcode.

The ld64 itself should build for ppc64 with no issues, as long as you do not build it against LLVM (which becomes less trivial, since no version of LLVM which can build with gcc-4.0/4.2 builds for ppc64).