I wanted to share my story about setting up my Mac Mini to boot with macOS, Windows 10, and Ubuntu Linux with an eGPU. This will not be an exhaustive guide on setting up an eGPU with a mac (many such guides exist, which I benefitted from in creating this setup), but rather I hope that discussing some of the intricate details of what was required to make it work will be helpful to others.
As a cross-platform developer and a casual gamer, I had been searching for a single-machine solution that could accommodate each of these pursuits. I know I'm not alone in appreciating Apple for their positive traits, while being frustrated by the crippled desktops that they have been offering for the last few years. It took a tremendous amount of research to get this all set up (perhaps more than I would invest if I had to do it over again), but now that it's working, it is a stable setup for the most part.
Computer
I used my 2014 Mac Mini 2.6GHz 8GB. I was glad to be able to take advantage of Thunderbolt 2, due to bandwidth limitations that you will run into. If Apple ever gets around to creating a headless desktop machine with Thunderbolt 3, I will appreciate the even greater performance.
Thunderbolt Enclosure
I picked up a secondhand BizonBox 2 on eBay. It came in conjunction with a GPU that I ended up being unable to use, but once I resold that, I think I ended up shelling out about $350 for the enclosure and 180W power supply. While this was a bit of cash, it's still a significant savings over the $600 that Bizon wants for their current Thunderbolt 2 enclosure. I was expecting a notch in the top of the enclosure for the power supply wires right behind the front cover (like I'd seen in pictures), but it turned out that this version had neither a notch nor a front cover, as it appears it was meant to accommodate full-length cards which stick out a little bit. The result is a rather unappealing look, but at least there appears to be plenty of air circulation.
GPU
From what I have read, EVGA is a good card manufacturer to go with for a stable eGPU setup. The eGPU enclosure came with an EVGA NVIDIA GTX 980 4GB. Suffice it to say, this was a fantastic card and I was blown away by the performance. Unfortunately, I ran into very strange stability issues under Windows and Linux. When booting into Windows, the machine would crash at various times during boot about half the time, and when it did boot successfully, sometimes the GPU would simply fail to come up and would appear disconnected. I tried literally everything I could think of as far as software fixes, and even bought a new Thunderbolt 2 cable to try, but I was unable to solve the issue. When booting into Linux, the GPU would also occasionally fail to come up. After being worried that my GPU was defective, I tried plugging the enclosure into two other machines (both Early 2011 15" MacBook Pros with Thunderbolt 1), and the eGPU was completely stable. I eventually suspected a firmware issue with my Mac Mini, but I knew my luck would be slim getting any help from Apple. I inquired with Bizon's tech support, and they stated they had never received any similar reports before, and suggested I try hotplugging the eGPU enclosure at various times during the bootup stage. This was not an acceptable solution for me.
I didn't want to invest a whole lot of time or money into trying a bunch of different GPUs, so I decided to go with the safest bet I could. I was limited by the 180W power supply, and I speculated that my problem may have been related to using a PCIe 3 card in a PCIe 2 slot (notwithstanding that the former is supposed to be backwards compatible). Since pretty much anything would have been an improvement over the integrated Intel Iris, I decided to try the latest PCIe 2 card from NVIDIA that was within my power requirements, so I bought an EVGA NVIDIA GTX 560Ti 2GB for about $60 on eBay (it took me a while to find the 2GB model, since the 1GB model was more prevalent). While this card isn't comparable to the GTX 980, it's still a huge improvement over the integrated card, and it has been sufficient for the casual gaming I like to do. As an added bonus, when running Windows or Linux, my Mac Mini appears to disable the integrated card when the eGPU is plugged in (see "HDMI Switch" below), which didn't happen with the GTX 980.
Drivers
On Windows, I used the drivers I downloaded from NVIDIA's web site. On Linux, I enabled the proprietary drivers in the system settings. On macOS, I disabled System Integrity Protection, then ran the automate-eGPU tool from GitHub, which has been fantastic in my experience. Every time there is a system update to macOS, I have to wait a few days for NVIDIA to update their drivers, then install the system update and re-run the automate-eGPU tool to update the NVIDIA drivers.
Partitioning & rEFInd
The Disk Utility in macOS since El Capitan is severely crippled from its former version, so I ended up doing the majority of the partitioning in Linux, then I used the diskpart tool in the Windows installer to set up the Windows partitions. I had to make the EFI partition a little bit bigger (I think 300MB) due to having three operating systems storing stuff on it. I'm not sure which operating system is doing it (I suspect macOS), but the partition table occasionally reverts to a hybrid rather than a protective MBR. It's pretty easy to boot into Linux and change it back using the parted tool.
rEFnd is a great tool, and nearly all the setup was automatic. Occasionally I need re-bless the rEFInd directory in the EFI partition (generally after a system update for Windows or macOS).
HDMI Switch
One of the issues with using an eGPU on macs is the lack of a boot screen (and the corresponding lack of FileVault support). It wasn't acceptable to me to abandon FileVault, so I needed another solution. The obvious solution is to plug your display(s) into your eGPU and then run a secondary connection between one display and your mac's built-in video output, so you get the boot screen/FileVault prompt through the built-in output and then it switches over to the eGPU. The issue with this is that your computer then may think you have three displays, depending on how it handles the two video cards and three display connections. On Windows, it's easy enough to disable the superfluous display, and I believe it isn't difficult on Linux either. However, on macOS, I was not able to find a solution that allowed me to fully disable the superfluous display. I tried SwitchResX as well as this tool but there were a lot of glitches, settings changing around by themselves, etc.
Ultimately, what I needed was something that would physically break the HDMI connection between my mac's built-in video output and the display. Ultimately, I went with this mechanical switch which appears to do just that. The options seem to be limited as far as switches go, which may be because such switches technically violate the HDMI spec (at least this is what people were saying on various forum posts on the topic). I run HDMI from my GPU to one of the switch inputs, and HDMI from my mac's built-in video output to the other switch input, and I run the switch output to the display. When I boot into Windows or Linux, I toggle the switch as soon as I select the OS I want with reFIND. When I boot into macOS, I wait until the system boots up completely, then toggle the switch.
macOS
This is where I spend the majority of my time. Other than having to wait a few days to install a system update (so the NVIDIA drivers get updated as well), I really don't have any complaints as it's quite a stable system. Everything graphical is quite snappy, and gaming (what limited selection there is for macOS... bring me Overwatch, Blizzard!) is quite an improvement over the integrated graphics. All in all, I'm quite pleased with how everything is working.
Windows
Windows is probably the most problematic of the three operating systems I run. While the issues aren't frequent, occasionally I've ended up with an unusable Windows installation and have had to re-install it. The issues seem to occur when Windows tries to install updates, and I suspect it has to do with Windows trying to install over my NVIDIA drivers. I think I've mitigated the issue by updating the NVIDIA drivers straight from the vendor first, and then allowing Windows do run the rest of its updates.
Linux
I had some initial challenges getting the eGPU to work under Linux. When I reviewed the kernel log, I saw a lot of messages like these:
These were resolved when I followed a suggestion I found online and added "pci=realloc" to my kernel arguments. Since that was fixed, stability has been great.
Conclusion
I owe a debt of gratitude to the community of eGPU enthusiasts who have created a wealth of information on how to get these devices working. As those of you who've tried it have experienced, the path to getting an eGPU working is often far from straightforward. Hopefully someone will benefit from my journey as well. Please feel free to post questions about my experience if you have any.
As a cross-platform developer and a casual gamer, I had been searching for a single-machine solution that could accommodate each of these pursuits. I know I'm not alone in appreciating Apple for their positive traits, while being frustrated by the crippled desktops that they have been offering for the last few years. It took a tremendous amount of research to get this all set up (perhaps more than I would invest if I had to do it over again), but now that it's working, it is a stable setup for the most part.
Computer
I used my 2014 Mac Mini 2.6GHz 8GB. I was glad to be able to take advantage of Thunderbolt 2, due to bandwidth limitations that you will run into. If Apple ever gets around to creating a headless desktop machine with Thunderbolt 3, I will appreciate the even greater performance.
Thunderbolt Enclosure
I picked up a secondhand BizonBox 2 on eBay. It came in conjunction with a GPU that I ended up being unable to use, but once I resold that, I think I ended up shelling out about $350 for the enclosure and 180W power supply. While this was a bit of cash, it's still a significant savings over the $600 that Bizon wants for their current Thunderbolt 2 enclosure. I was expecting a notch in the top of the enclosure for the power supply wires right behind the front cover (like I'd seen in pictures), but it turned out that this version had neither a notch nor a front cover, as it appears it was meant to accommodate full-length cards which stick out a little bit. The result is a rather unappealing look, but at least there appears to be plenty of air circulation.
GPU
From what I have read, EVGA is a good card manufacturer to go with for a stable eGPU setup. The eGPU enclosure came with an EVGA NVIDIA GTX 980 4GB. Suffice it to say, this was a fantastic card and I was blown away by the performance. Unfortunately, I ran into very strange stability issues under Windows and Linux. When booting into Windows, the machine would crash at various times during boot about half the time, and when it did boot successfully, sometimes the GPU would simply fail to come up and would appear disconnected. I tried literally everything I could think of as far as software fixes, and even bought a new Thunderbolt 2 cable to try, but I was unable to solve the issue. When booting into Linux, the GPU would also occasionally fail to come up. After being worried that my GPU was defective, I tried plugging the enclosure into two other machines (both Early 2011 15" MacBook Pros with Thunderbolt 1), and the eGPU was completely stable. I eventually suspected a firmware issue with my Mac Mini, but I knew my luck would be slim getting any help from Apple. I inquired with Bizon's tech support, and they stated they had never received any similar reports before, and suggested I try hotplugging the eGPU enclosure at various times during the bootup stage. This was not an acceptable solution for me.
I didn't want to invest a whole lot of time or money into trying a bunch of different GPUs, so I decided to go with the safest bet I could. I was limited by the 180W power supply, and I speculated that my problem may have been related to using a PCIe 3 card in a PCIe 2 slot (notwithstanding that the former is supposed to be backwards compatible). Since pretty much anything would have been an improvement over the integrated Intel Iris, I decided to try the latest PCIe 2 card from NVIDIA that was within my power requirements, so I bought an EVGA NVIDIA GTX 560Ti 2GB for about $60 on eBay (it took me a while to find the 2GB model, since the 1GB model was more prevalent). While this card isn't comparable to the GTX 980, it's still a huge improvement over the integrated card, and it has been sufficient for the casual gaming I like to do. As an added bonus, when running Windows or Linux, my Mac Mini appears to disable the integrated card when the eGPU is plugged in (see "HDMI Switch" below), which didn't happen with the GTX 980.
Drivers
On Windows, I used the drivers I downloaded from NVIDIA's web site. On Linux, I enabled the proprietary drivers in the system settings. On macOS, I disabled System Integrity Protection, then ran the automate-eGPU tool from GitHub, which has been fantastic in my experience. Every time there is a system update to macOS, I have to wait a few days for NVIDIA to update their drivers, then install the system update and re-run the automate-eGPU tool to update the NVIDIA drivers.
Partitioning & rEFInd
The Disk Utility in macOS since El Capitan is severely crippled from its former version, so I ended up doing the majority of the partitioning in Linux, then I used the diskpart tool in the Windows installer to set up the Windows partitions. I had to make the EFI partition a little bit bigger (I think 300MB) due to having three operating systems storing stuff on it. I'm not sure which operating system is doing it (I suspect macOS), but the partition table occasionally reverts to a hybrid rather than a protective MBR. It's pretty easy to boot into Linux and change it back using the parted tool.
rEFnd is a great tool, and nearly all the setup was automatic. Occasionally I need re-bless the rEFInd directory in the EFI partition (generally after a system update for Windows or macOS).
HDMI Switch
One of the issues with using an eGPU on macs is the lack of a boot screen (and the corresponding lack of FileVault support). It wasn't acceptable to me to abandon FileVault, so I needed another solution. The obvious solution is to plug your display(s) into your eGPU and then run a secondary connection between one display and your mac's built-in video output, so you get the boot screen/FileVault prompt through the built-in output and then it switches over to the eGPU. The issue with this is that your computer then may think you have three displays, depending on how it handles the two video cards and three display connections. On Windows, it's easy enough to disable the superfluous display, and I believe it isn't difficult on Linux either. However, on macOS, I was not able to find a solution that allowed me to fully disable the superfluous display. I tried SwitchResX as well as this tool but there were a lot of glitches, settings changing around by themselves, etc.
Ultimately, what I needed was something that would physically break the HDMI connection between my mac's built-in video output and the display. Ultimately, I went with this mechanical switch which appears to do just that. The options seem to be limited as far as switches go, which may be because such switches technically violate the HDMI spec (at least this is what people were saying on various forum posts on the topic). I run HDMI from my GPU to one of the switch inputs, and HDMI from my mac's built-in video output to the other switch input, and I run the switch output to the display. When I boot into Windows or Linux, I toggle the switch as soon as I select the OS I want with reFIND. When I boot into macOS, I wait until the system boots up completely, then toggle the switch.
macOS
This is where I spend the majority of my time. Other than having to wait a few days to install a system update (so the NVIDIA drivers get updated as well), I really don't have any complaints as it's quite a stable system. Everything graphical is quite snappy, and gaming (what limited selection there is for macOS... bring me Overwatch, Blizzard!) is quite an improvement over the integrated graphics. All in all, I'm quite pleased with how everything is working.
Windows
Windows is probably the most problematic of the three operating systems I run. While the issues aren't frequent, occasionally I've ended up with an unusable Windows installation and have had to re-install it. The issues seem to occur when Windows tries to install updates, and I suspect it has to do with Windows trying to install over my NVIDIA drivers. I think I've mitigated the issue by updating the NVIDIA drivers straight from the vendor first, and then allowing Windows do run the rest of its updates.
Linux
I had some initial challenges getting the eGPU to work under Linux. When I reviewed the kernel log, I saw a lot of messages like these:
Apr 23 16:34:47 **** kernel: [ 162.941067] pci 0000:07:00.0: BAR 15: no space for [mem size 0x0e000000 64bit pref]
Apr 23 16:34:47 **** kernel: [ 162.941069] pci 0000:07:00.0: BAR 15: failed to assign [mem size 0x0e000000 64bit pref]
...
Apr 23 16:34:47 **** kernel: [ 162.996768] nvidia 0000:09:00.0: enabling device (0000 -> 0002)
Apr 23 16:34:47 **** kernel: [ 162.996969] NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:
Apr 23 16:34:47 **** kernel: [ 162.996969] NVRM: BAR1 is 0M @ 0x0 (PCI:0000:09:00.0)
Apr 23 16:34:47 **** kernel: [ 162.996971] NVRM: The system BIOS may have misconfigured your GPU.
Apr 23 16:34:47 **** kernel: [ 162.996982] nvidia: probe of 0000:09:00.0 failed with error -1
Apr 23 16:34:47 **** kernel: [ 162.997197] nvidia-nvlink: Nvlink Core is being initialized, major device number 242
Apr 23 16:34:47 **** kernel: [ 162.997390] NVRM: The NVIDIA probe routine failed for 1 device(s).
Apr 23 16:34:47 **** kernel: [ 162.997391] NVRM: None of the NVIDIA graphics adapters were initialized!
Apr 23 16:34:47 **** kernel: [ 162.997396] nvidia-nvlink: Unregistered the Nvlink Core, major device number 242
Apr 23 16:34:47 **** kernel: [ 162.941069] pci 0000:07:00.0: BAR 15: failed to assign [mem size 0x0e000000 64bit pref]
...
Apr 23 16:34:47 **** kernel: [ 162.996768] nvidia 0000:09:00.0: enabling device (0000 -> 0002)
Apr 23 16:34:47 **** kernel: [ 162.996969] NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:
Apr 23 16:34:47 **** kernel: [ 162.996969] NVRM: BAR1 is 0M @ 0x0 (PCI:0000:09:00.0)
Apr 23 16:34:47 **** kernel: [ 162.996971] NVRM: The system BIOS may have misconfigured your GPU.
Apr 23 16:34:47 **** kernel: [ 162.996982] nvidia: probe of 0000:09:00.0 failed with error -1
Apr 23 16:34:47 **** kernel: [ 162.997197] nvidia-nvlink: Nvlink Core is being initialized, major device number 242
Apr 23 16:34:47 **** kernel: [ 162.997390] NVRM: The NVIDIA probe routine failed for 1 device(s).
Apr 23 16:34:47 **** kernel: [ 162.997391] NVRM: None of the NVIDIA graphics adapters were initialized!
Apr 23 16:34:47 **** kernel: [ 162.997396] nvidia-nvlink: Unregistered the Nvlink Core, major device number 242
These were resolved when I followed a suggestion I found online and added "pci=realloc" to my kernel arguments. Since that was fixed, stability has been great.
Conclusion
I owe a debt of gratitude to the community of eGPU enthusiasts who have created a wealth of information on how to get these devices working. As those of you who've tried it have experienced, the path to getting an eGPU working is often far from straightforward. Hopefully someone will benefit from my journey as well. Please feel free to post questions about my experience if you have any.
