One of Crowdstrike's (and ALL the OTHERS, TOO) selling points is immediate response, across the widest possible swath of the internet, to the rapidly changing threatscape. It sounds like Crowdstrike's developers got a tad complacent, failed due diligence.
Crowdstrike is a complex beast, and among its features is an option to TEST code and implementation policy rollouts on your own lab rats, before rolling out to Production. Crowdstrike even offers free lab CIDs (Customer ID - their term for tenant-owned units).
I segregate lab rats in a CID, where we test all updates, no matter the vendor's reputation, or how urgent the update may seem. It takes very little time to get through a basic test plan. Then, if one of 'em sudden'y grows a human ear out of its back, we have a minute to reconsider that rollout, no harm done.
For this week's raging DOS event, there's plenty of blame to share around. There are stages of responsibility that should not be denied out of defensiveness, guilt or possibly relief, in the OCIO's office.
Windows is a choice, albeit a sensible one.
Crowdstrike is a choice, albeit a sensible one.
IT Admin process is a choice, albeit tending to break down under budgetary pressure.
Customers who don't do due diligence get da doo-doo dropped.
For Windows servers, there are recovery modes, typically manually invoked.
@Ethosik id generally describing "System Integrity Protection", which would not interdict legit admin overrides during installations. Generally, one doesn't want auto-rollback, which might merely add a one more problem to a troubleshooting cycle. In fact, auto-rollback introduces another potential attack surface.
Recovery and rollback scenarios are better served by Netboot or PXE boot, to implement scripted repairs from a known-good OS. This can be fiendishly complex to build and maintain. Unfortunately, such COOP (continuity of operations) or DR (disaster recovery) plans are defunded due to budget pressure.
Contempt for Microsoft is real in IT service management, even as it provides job security. Essentially, in terms of finance and sustainability, we all know the only thing worse, overall, than Windows, overall, is everything else, overall. In fact, Microsoft themselves have caused far more Windows DOS events than any other vendor.
In the blink of an eye, it might have been unix/linux. THAT would be the epic perfect **** storm, as the world's genuine, serious, heavy lifting is not done on Windows.
