Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Software Engineer Runs Generative AI on 20-Year-Old PowerBook G4

AltiVec!!!!!!!!!!!!

My god there’s a word I haven’t hear in a long time.
 
Apple!Fre@k said:
Okay, cool, but are there any market implications for this? Does this change anything?
Click to expand...
It changed nothing indeed. PowerPC Macs are obsoleted. But that does not stop people from playing with those hardware. That's why vintage phone, classic iPod, handheld camcorder have a market.
 
I can only get Jan.ai to run on my Chromebook. any other Chromebook users have suggestions / alternatives to Jan.ai? its good, but its a bit slow, considering its 8gb ram, core i3. just curious what other options there are.
 
mdriftmeyer said:
No you won't. You'll realize one known fact: a stripped down windows manager, non-hdr, zero airplay and hundreds of other services in OS X if possible to implement would show how efficient XNU is to the Linux Kernel.

I won't even mention the Core Graphics stack and Display PDF that Linux will never have alone would dump all over Xorg and Wayland.
Click to expand...
Those “hundreds of other services” are what makes an OS bloated. The point of Linux is that you choose exactly what you want to install to make the most of your machine. For example, if I have a server I don’t need any sort of GUI - it only slows things down.

If MacOS offered that level of customizability such that you could install just the kernel and bare essentials then it would be similar. But it doesn’t and never has - you get what you get.
 
If they can achieve this, Apple should be able to do the same for all older Macs. Honestly, I've never understood why Apple restricts the 2013 MacBook Pro from running the latest operating system. I had to resort to using a Legacy Patch to get it functioning. Surprisingly, the OS performed smoothly, even better than Big Sur, which is the latest version available for older models.
 
  • Like
Reactions: redheeler
That's extremely impressive and all... but can we talk about how GORGEOUS this PowerBook G4 is? And WTF, the post owner has one in mint condition too!
 
Altivec would be the equivalent of am64's AVX extensions or ARM NEON. I remember vector extensions being a big argument for PowerPC and POWER back in the day.
 
  • Like
Reactions: DefNotAnLLM
supremedesigner said:
If they can achieve this, Apple should be able to do the same for all older Macs. Honestly, I've never understood why Apple restricts the 2013 MacBook Pro from running the latest operating system. I had to resort to using a Legacy Patch to get it functioning. Surprisingly, the OS performed smoothly, even better than Big Sur, which is the latest version available for older models.
Click to expand...
Planned obsolescence.

There are examples of unsupported MacOS versions running better than supported ones. Mountain Lion ran better than supported Lion on a lot of 2006-2007 Macs.
 
computerwiz222 said:
This is quite the response! I am really enjoying the discussion here.

This is nothing more than a tech demo, but a fun one nonetheless. Many of you have correctly pointed out that this 110M model is puny in comparison to some of the larger multi-billion parameters. This setup would be lucky to produce a token in an hour, if we could even somehow fit the model into a 32-bit address space.
Click to expand...

One thing that'd be interesting to see is if perf gets any better building with GCC 14 instead of 4. I have a cross-compilation setup in a docker image here if you're interested:

github.com

gcc-powerpc-apple-darwin8/.github/workflows/Dockerfile at build/gcc-14.2+MacOSXSDK10.4u · VariantXYZ/gcc-powerpc-apple-darwin8

Some docker files and notes for getting a modern cross-compiling toolchain for OSX 10.4 Tiger on PPC Macs. Each build/ branch contains a different toolchain + SDK combination. - VariantXYZ/gcc-powe...
github.com github.com

I think MacPorts could give you a host toolchain of 14 if you wanted to run the compilation on the PowerBook instead for science...
 
VariantXYZ said:
I have a cross-compilation setup in a docker image here if you're interested
Click to expand...

That's handy. I left this poor PowerBook G4 to build gcc 8.5 overnight when I started this project! It did finish, eventually. Changing from gcc 4.x to 8.x didn't yield any significant performance improvements.

I did spend some time digging through the code to find other places where AltiVec could be employed. When I was testing with the 15M model, inference took roughly 40 seconds total. Of this time, around 30 seconds is spent in the matmul function. I tried to improve some of the other functions like softmax and rmsnorm but it always made things slower by about 100-250ms.

Interesting stuff!
 
  • Like
Reactions: VariantXYZ and redheeler
computerwiz222 said:
That's handy. I left this poor PowerBook G4 to build gcc 8.5 overnight when I started this project! It did finish, eventually. Changing from gcc 4.x to 8.x didn't yield any significant performance improvements.

I did spend some time digging through the code to find other places where AltiVec could be employed. When I was testing with the 15M model, inference took roughly 40 seconds total. Of this time, around 30 seconds is spent in the matmul function. I tried to improve some of the other functions like softmax and rmsnorm but it always made things slower by about 100-250ms.

Interesting stuff!
Click to expand...

Matmul taking the longest makes sense, but I wonder if GCC 14's auto-vectorization with -maltivec might work...

I might try on my own old G4 running Tiger. It's the 867 MHz version but it's still a trooper.
 
There is a little diffrence about 2B Llama model slowly prompting pancake recipe and what's apple trying to do with little to none markup on system speed.
 
bousozoku said:
I suspect that everyone running a Mac was glad that the renewed Apple was going to use NeXTStep instead of BeOS as a new basis.
Click to expand...
Aye. NeXTStep was an established OS, although niche at the time. It also boasted the best spreadsheet app at the time, Lotus Improv.

Apple wanted NeXTStep. They didn't care for Jobs at the time; he returned as an "adviser."😐 They had to take him as a condition to obtain the OS.
 
  • Like
Reactions: bousozoku
coolfactor said:
Properly optimized.... that's the ticket! So many people have demanded that Apple add in tons of RAM, but in doing so, developers get lazy and don't optimize, and then tons of RAM still isn't enough over time. I've appreciated Apple's conservative approach to RAM over the years, as it's forced optimizations, which is critical to software being great.
Click to expand...
I completely agree. More RAM has led to lazier developers...Apple included. iOS is so bloated and inefficient.
 
Mousse said:
Aye. NeXTStep was an established OS, although niche at the time. It also boasted the best spreadsheet app at the time, Lotus Improv.

Apple wanted NeXTStep. They didn't care for Jobs at the time; he returned as an "adviser."😐 They had to take him as a condition to obtain the OS.
Click to expand...
I'm not sure everything went the right way and it was three major versions (10.3.4 FTW) before it was useful, but it turned out quite a bit better than if they had used a Linux-style monolithic kernel.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back