Big Data Management on MacOS?

[YoitsTmac]

How is everyone handling big data through MacOS? I originally started with a virtualized environment on ESXi with a SATA passthrough to retain health monitoring, but I found SATA PCI cards unreliable and would stop working.

Then I got a QNAP NAS, but it peaked at 20MB/s when encrypted.

Then I thought I’d try OpenNAS, but I found it difficult. There was another hurdle I ran into but I can’t recall what it was.

Then I went to Xpenology and its encryption mode didn’t support seem to support file names as long as MacOS, and caused copying issues for backups. So that was a dead end.

At this point I was pretty frustrated, so I converted to bare metal and bought a Mac Mini. Then I purchased a 4-bay DAS from QNAP TR-004 and left it in passthrough mode to retain all my drives as individual drives. Now I can’t see my drive health status, which kind of makes deflates the confidence I have in the rig.

Here’s what I demand from my remote storage:

• Full encryption without massive speed sacrifices (Apple’s Encrypted APFS does this excellently)
• Plex server
• Remote backup for all my Macs and my work drive (video editor)
• …fast enough to back up my work drive in a half reasonable amount of time
• A backup of all my files in the event of a drive failure (Carbon Copy Cloner)
• Proper account management? Apple’s shared file tools are kind of crap…I think this is an open secret

I find myself frustrated again. Apple’s management and permissions seem to be whack. I use one folder as a giant network USB. I can view it, delete from it and copy from it no problem. I can create new folders, but I can’t name them. Sometimes a folder will be read-only until I revisit it, but I’m logged in remotely as the admin. I have to screen share to do my folder management, and it’s a bit work-heavy for some basic file management.

Now I feel I should make a Hackintosh so that all the drives are connected via SATA and appear internal, and therefore will be able to be read by Drivedx while retaining all other functions including sleep. Any tips?

 

[zarmanto]

I'm not particularly knowledgeable specifically on big data... but to address your last comment: personally, I think that anyone with the time, money and technical expertise should attempt to build their own hackintosh at least once. At the very minimum, you're likely to come away with a greater appreciation for what Apple brings to the table. (I certainly did.)

That said: I would never, ever recommend to anyone that they use a hackintosh for actual business purposes. If that's your objective, I suggest that you go a little bit more "bare-bones," and setup a Darwin server instead.