CrowdStrike Says Global IT Outage Impacting Windows PCs, But Mac and Linux Hosts Not Affected

[klasma]

Not sure why people are crapping on Microsoft here. The issue is crowdstrike. They have Linux and Mac versions of Falcon sensor too. They just happened to screw up the windows version today and push it to production past everyone’s security controls.

This could be Apple or Linux tomorrow.

It actually was Linux recently: https://news.ycombinator.com/item?id=41005936

 

[kazmac]

..but one can possibly expect them to roll out the update by testing it on a representative and limited array of devices before going global. That`s pretty valid all down the custody chain. One could easily argue that Crowdstrike must have failed to execute that kind of due dilligence, as would end customer sys admins and the lot.

That limited test roll out was exactly what NYC did earlier today and why critical agencies/systems/hospitals are online there.

Thanks to all the awesome IT folks working to resolve this.

 

[jedimasterkyle]

Mankind WITH computers:

Mankind WITHOUT computers:

 

[00001000bit]

This is what happens when businesses want to do things on the cheap.

It's not about cheap, it's about deflecting blame. The C-suite execs are accountable to the board. The board is populated by people who aren't tech experts.

If you are a CIO who chooses to manage your own security, and something goes wrong, you're the one responsible, you're the one getting fired. If you just adopt an "industry best practice" and it goes wrong ... "well, what can you do? We did everything we were expected to do."

The board would demand action on a homegrown security failure. But, one that affects you and a bunch of other companies (a handful of whose boards they ALSO sit in on) - then that's just an infrastructure failure. It's the same reason you see so many companies buy into solutions by SAP or Oracle when they could get by much cheaper. When something goes wrong, it's the vendor's fault.

 

[jakey rolling]

If the product itself is vulnerable to massive global outages because some coder at a random 3rd party software company inserted rogue code, then we've just seen how vulnerable the entirety of the enterprise system that relies on MS Windows is.

Crowdstrike is used on macOS and Linux machines as well, and in order to work properly and be an effective security tool, it would require the same kernel-level access and update schedule that it has on Windows. If you don't think that macOS and Linux are just as vulnerable to the same coding mistake, you are incredibly naive.

The only reason this is newsworthy is because it affected the operating system version that is most widely used on large corporate systems. If this mistake had happened on the macOS version of the update instead, it would barely register a blip outside of a few market departments.

If you're a hacker, you now have a blueprint for taking out much of the global computing infrastructure. You don't have to hack microsoft, you just have to hack any developer who can push an update to a large portion of the installed base that runs windows.

Or you can infiltrate a commonly-used library and become a kernel contributor. Or did we already forget about that incident?

 

[Tago]

Or the topic is that you don't spot the difference between open and closed systems.

You have options for an open system. I prefer a closed system. You want to take my option away from me. I'm fine that you have the option you want.

As many bordering geriatric users, I have used a good many variations from *nix, Dos, most iterations of Windows, Mac, Ios, Android and so on. Believe Apple delivers an ok balance for their Macs, and ios is as closed as I want it. I am not a fan of opening up ios to all these 3rd party shacks at all. That stuff isn`t driven by the bulk of ios users and certainly not by Apple. It is driven by competitors who wants to weaken the ecosystem to get a free ride and grab more of the profits, paired with direct competitors to the ecosystem.

Any user who wants that crap would be better served elsewhere. I prefer the Apple approach to phones and pads.

 

[VulchR]

I'm surprised no one has blamed Tim Cook for this.

I blame the teachers.

 

[Timo_Existencia]

Crowdstrike is used on macOS and Linux machines as well, and in order to work properly and be an effective security tool, it would require the same kernel-level access and update schedule that it has on Windows. If you don't think that macOS and Linux are just as vulnerable to the same coding mistake, you are incredibly naive.

I’d blame Apple as well if such an outage occurred on MacOS due to granting kernel-level access to random 3rd party developers. If MacOS is vulnerable in the same way then I’d expect Apple to plug that security hole. Quick.

 

[aloysiusfreeman]

..but one can possibly expect them to roll out the update by testing it on a representative and limited array of devices before going global. That`s pretty valid all down the custody chain. One could easily argue that Crowdstrike must have failed to execute that kind of due dilligence, as would end customer sys admins and the lot.

I'm not 100% sure if this was the case but appears that the update rollout was controlled by CrowdStrike?

 

[ProbablyDylan]

Maybe it's not so bad that my company is too cheap to take computer security seriously.

 

[bzgnyc2]

I’d blame Apple as well if such an outage occurred on MacOS due to granting kernel-level access to random 3rd party developers. If MacOS is vulnerable in the same way then I’d expect Apple to plug that security hole. Quick.

Apple doesn't control whether 3rd party apps get kernel-level access on MacOS. You/your system administrator/IT controls that. There are a lot more controls for this under more recent MacOS -- as well as a shift towards "system extensions" -- but in the end you control what 3rd parties can do on your Mac.

 

[jedimasterkyle]

 

[WarmWinterHat]

I work for a big company that is down because of this. My windows laptop was down. And for all the talk of getting into safe mode... A lot of people get the BSOD as soon as they start to go into safe mode. I was able to fix it by going into windows repair and then to the command prompt.

All our machines have Bitlocker enabled and they are locked. Only IT has the codes to unlock, before the fix can be done.

I don't work in IT and don't have Admin access.

 

[addamas]

The problem is autoupdates generally.

apple and microsoft have pushed updates which have caused problems in the past - so no point saying trash the windows pc and get mac instead. apple do autoupdates too.

- like ios 8.0.1 which disabled cellular network, not technically bricking but made the iphones inoperable as phones

- and Windows 10 October 2018 Update which deleted files from computer

apple and microsoft pulled them but the damage was done

lots of 3rd party apps do autoupdates too, so it isn’t just crowdstrike

but you need to take control of your devices, disable autoupdates as much as possible and do updates manually, and only when it suits you and not them - and only when you know it is safe to do so

Apple learned from mistake - it’s not deployed to every single device at once, automatic updates takes time (but its microsecond compared to Android).
But totally agree - safest way is to disable auto update of OS and if you use crucial apps.

Even Home Assistant had recently crucial issues not mentioned in changelog

 

[splifingate]

Maybe it's not so bad that my company is too cheap to take computer security seriously.

Ha!

 

[bzgnyc2]

I'm not 100% sure if this was the case but appears that the update rollout was controlled by CrowdStrike?

I believe it was pushed out by CrowdStrike but likely on a schedule agreed to by IT (i.e. they decided/accepted an auto-update policy/schedule as part of the enterprise installation/configuration).

 

[kk200]

Not sure why people are crapping on Microsoft here. The issue is crowdstrike. They have Linux and Mac versions of Falcon sensor too. They just happened to screw up the windows version today and push it to production past everyone’s security controls.

This could be Apple or Linux tomorrow.

by following crowdstrike failure on linux I read this recommendation

"Make sure you're running in user mode (eBPF) instead of kernel mode (kernel module), since it has less ability to crash the kernel. This became the default in the latest versions and they say it now offers equivalent protection."

Such kind of user-land infrastructure is exactly what MS needs to provide on windows ....

 

[Timo_Existencia]

Apple doesn't control whether 3rd party apps get kernel-level access on MacOS. You/your system administrator/IT controls that. There are a lot more controls for this under more recent MacOS -- as well as a shift towards "system extensions" -- but in the end you control what 3rd parties can do on your Mac.

And what I’m saying is that to the degree that Apple allows 3rd Party Developers access to core system elements that can brick my entire computer, then that’s a problem that I’d expect Apple to be extremely cognizant of.

And, my sense is that Apple chose to approach IOS much differently than MacOS; which to me, is a benefit. I choose a more closed system when I choose IOS. I’d prefer MacOS move more toward IOS then away from IOS.

 

[TheCrawlingCha0s]

I work for a very small IT firm and this exact thing happened with our RMM/ CRM a few months ago and it was basically 4 of us having to get around 150 PCs back up and running over the course of 4 weeks mostly remote and it was a literal nightmare. God speed everybody who is dealing with this today.

 

[oneMadRssn]

Not sure why people are crapping on Microsoft here. The issue is crowdstrike. They have Linux and Mac versions of Falcon sensor too. They just happened to screw up the windows version today and push it to production past everyone’s security controls.

This could be Apple or Linux tomorrow.

Microsoft deserves a little bit of crap for having a system that cannot even boot in the situation of trying to load a buggy driver - it should be smart enough to isolate and not load a driver that is causing problems. If it could at revert to some bootable state with network access, then the systems would be fixable remotely or automatically.

 

[fyun89]

Seems like a good time to remind everyone MacOS isnt immune to issues.
Just be aware of these kind of news! Update as frequently as possible!

 

[Tago]

That limited test roll out was exactly what NYC did earlier today and why critical agencies/systems/hospitals are online there.

Thanks to all the awesome IT folks working to resolve this.

Yeah, would argue that is a prudent approach that shouldn`t be easily skipped. Glad they did a proper job 👍

 

[jakey rolling]

I’d blame Apple as well if such an outage occurred on MacOS due to granting kernel-level access to random 3rd party developers. If MacOS is vulnerable in the same way then I’d expect Apple to plug that security hole. Quick.

That would be a tough one. Plugging such a "security hole", as you call it, means that boot-sector and kernel-level security mitigations would not be possible, neutering this kind of low-level corporate security software. Apple would essentially be taking a "trust us and only us" approach to security. This may be alright for personal devices and non-critical back-office systems, but it would be an absolute no-go for critical enterprise systems such as the ones affected by the outage today.

 

[jakey rolling]

Seems like a good time to remind everyone MacOS isnt immune to issues.

Thank you for the reminder, early 00's Justin Long.

 

[Timo_Existencia]

That would be a tough one. Plugging such a "security hole", as you call it, means that boot-sector and kernel-level security mitigations would not be possible, neutering this kind of low-level corporate security software. Apple would essentially be taking a "trust us and only us" approach to security. This may be alright for personal devices and non-critical back-office systems, but it would be an absolute no-go for critical enterprise systems such as the ones affected by the outage today.

But how is trusting Crowdstrike not a “trust us and only us” approach to security? I’m not following how you think trusting Crowdsource with your entire system is better than trusting Apple.