Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
I was at work last night when this all started. I couldn’t do my work, and there was nothing I could do.

BEST SHIFT EVAAAAA!!!!
 
  • Haha
Reactions: wilhoitm
They just did the same for myself and another in my department (pharmacy). I work in informatics, so I have more tech knowledge than most that I work with..

Boot with a bootable flash drive IT prepared, unlock bitlocker then delete the file. I have about a dozen computers to work on, and it's slllllooooooowwwww. I guess my pharmacy work will have to wait.

View attachment 2398307
Amazing that the endpoints are configured to allow to boot from flash drives 😬 But lucky for you all. Or did they give the keys to the BIOS as well?

I’m surprised, or actually not, how there are so many companies that use a product like Crowdstrike for end point protection but then still leave tbe “backdoors” open. 🤷‍♂️
 
Not sure why people are crapping on Microsoft here. The issue is crowdstrike. They have Linux and Mac versions of Falcon sensor too. They just happened to screw up the windows version today and push it to production past everyone’s security controls.

This could be Apple or Linux tomorrow.
They need to stop allowing third parties injecting code into the kernel. Just like Apple shunned Kexts.
 
IT being a critical technology for all mega corporations, why don't they own their IT, use solutions that they have a good control of? Greed? Greed will always have a cost one day.
LOL...you have to have redundancy in all of your IT solutions. You have to outsource a lot of things because it is not sustainable to do it all in house for any large corporation.
 
A new update for CrowdStrike Falcon is available ! Download it now ?

8xl473.jpg
 
  • Like
Reactions: coolfactor
LOL...you have to have redundancy in all of your IT solutions. You have to outsource a lot of things because it is not sustainable to do it all in house for any large corporation.
I mean...it is sustainable...they just dont want to pay for it which equates to outsourcing things that should remain in house.
 
IT being a critical technology for all mega corporations, why don't they own their IT, use solutions that they have a good control of? Greed? Greed will always have a cost one day.
Just imagine every company having a department to do what a company like Crowdstrike does 🤷‍♂️ It makes sense to outsource that part to a specialist.

But that trust has been broken big time last night / today. Absolutely inexcusable from Crowdstrike based upon what we know now.
 
IT being a critical technology for all mega corporations, why don't they own their IT, use solutions that they have a good control of? Greed? Greed will always have a cost one day.

I work with a few "tech" companies being run by management that has no clue about technology. It's a frustrating comedy show.

The addiction to Windows, over-complicated software designs, lack of forward-thinking design,and gravitating toward the biggest players for products and services is what leads to this mess.

Zoom is a great example. Their software is terrible, but hey, let's all have a Zoom meeting!

Just this morning, the CEO of one company I work with was asking us to "integrate Service X" because she doesn't want to have two browser tabs open. We're all just shaking our heads at the insanity.
 
  • Like
Reactions: TheOldChevy
You make it sound like this is just delayed revenue while the e-commerce front page is down.

Almost any organization today has (Windows) computers intergated into their workflow and operations. Hopefully no plane's onboard computers run Windows but there's still ATC, ticketing systems, baggage handling systems, security checkpoints, ... Similarly for hospitals and various other organizations responsible for people's lives at any givem moment.

And as a company you still have to take care of them. That means rerouting planes around "bricked" airports (but to where if your computers can't get the status of alternate airports?). It's meant canceling non-emergency surgeries and pausing patient visits because doctors can't access patient medical records. But which surgeries to cancel? Get the patient list from the ...oh...just call the patients and ...just get their number from the...oh

Unfortuantely we haven't designed our critical infrastructure/systems (in the broadest sense of the word systems) with resiliency in mind.

The IT leadership who went with a multicloud solution gets it:

Sounds about right, a 135 billion dollar company and it didn’t instigate resilience or backups properly, if at all. Because that costs money and eats into profits, same as airports. As I said to the OP I quoted, why stress about it? Literally all you can do is cancel everything. Planes don’t use Windows, for a reason. Last I checked pilots like to have iPads with them actually.
The great promise of the cloud, use computers run by other companies, services run by other companies, no need for your own infrastructure.. then when it all goes to crap and breaks down and you lose millions, just think those millions could have been spent on resilient backup solutions.

I read a thread on the iPad section on here from a user, he was very sensibly concerned with backing up his photos and videos both locally and in the cloud. Funny how someone considered his private photos and videos highly important, yet companies couldn’t really care less about their data and ability to actually do business, till it all crashes around them.

Its like if AWS went belly up, half the planet would literally stop working.
 
I mean...it is sustainable...they just dont want to pay for it which equates to outsourcing things that should remain in house.
Sometimes...but you can't do everything in house. Then external companies that specialize in software solutions have more specialized talent that in house does not have or can be developed. Agreed that it is a fine line of when to buy a third party solution or do something in-house. But developing an inhouse solution for the things CloudStrike does. Would take a massive $$$ investment upfront and years of fine tuning and find the talent to support it. That is just one part of what it takes for an effective infrastructure and security posture. Now multiple that by 100s of software. It is not sustainable to do all things in-house.
 
Just wait until CrowdStrike integrates real AI into their software...

You think today is scary.

I know, having everything run by cloud services just isn’t enough, we also must as humans throw in AI too.. it’ll be perfectly fine.
 
Sometimes...but you can't do everything in house. Then external companies that specialize in software solutions have more specialized talent that in house does not have or can be developed. Agreed that it is a fine line of when to buy a third party solution or do something in-house. But developing an inhouse solution for the things CloudStrike does. Would take a massive $$$ investment upfront and years of fine tuning and find the talent to support it. That is just one part of what it takes for an effective infrastructure and security posture. Now multiple that by 100s of software. It is not sustainable to do all things in-house.

You‘ve proved my point perfectly. Profits before resilience.
 
  • Disagree
Reactions: cyb3rdud3
Not sure why people are crapping on Microsoft here. The issue is crowdstrike. They have Linux and Mac versions of Falcon sensor too. They just happened to screw up the windows version today and push it to production past everyone’s security controls.

This could be Apple or Linux tomorrow.
Eh, no. The issue is Windows being a giant piece of ****. And it always has been. Had the same issue been deployed on a Mac, it would have isolated the problem to the program, not BSOD the entire machine.
 
  • Love
  • Like
Reactions: com.B and wilhoitm
Not pointless; sure identification has happened, you still need to contain, eradicate, recover. And communicate. Can't just sit back and do nothing. Even in this event there are still plenty of measures that can be taken.

The only true measure is to not rely on a third party in the cloud to literally be the vital component to or run your entire business as some seem to. Although I’m personally stunned MS allowed some third party full deep access to the root of its OS! That’s as proven is just asking for trouble.

My point of sit back and relax was more to the point of, when you chose not to have backup solutions as they are deemed not cost effective, then relax in that knowledge as your business loses millions and millions.
 
Last edited:
Here's the reality of this situation in IT over the past 25+ years: A 3rd party app crashes Windows (or Mac) and the user (whether it's an end user or IT Admin) can't boot. Someone (the IT department Reps) need to physically go to EACH machine and fix the problem so the machine can boot. That takes a massive amount of time and can only be done after the fix is found AND the IT Rep is properly comfortable with performing the steps for the fix.

Now flash forward to the past 10 or so years where so many employees are remote. Their machine can't boot. How is IT going to contact them?...the company better pray that IT has every employee's phone number to a)tell them about the problem and b)later get back to them about the fix. And then when the fix is ready, IT now needs to call each employee and pray they can walk the employee through the fix over the phone. There is the stark reality that over-the-phone cannot be done and now the employee needs to drive 100+ miles to an office or Fedex the machine...either way, days of productivity are lost as well as revenue.

I've worked at several large and small companies and getting ahold of their IT departments on the phone is next to impossible and that's when my machine was working and I had access to the corporate intranet to find the IT phone number. I recall a time when Bitlocker had wacky issues and I couldn't boot and had to call my boss who had to contact IT. What percentage of corporate employees have their boss' phone number?! Very small percentage I would gather.

All I could think off reading that was, why wouldn’t the employee affected follow company policy and actually call IT themselves? Is that beyond them? your experience isn’t that if everyone when trying to contact IT. Most IT teams advertise their contact number for internal and external lines. All you need to do is make a note of it and doubly so if you work remotely.
 
Absolutely, but I was just wondering if sysadmins even have the option. For example, it's my understanding that people who use Jamf Cloud to manage their device that Jamf is in control of the updates - admins do not decide on a schedule.

With how widespread this is I cannot imagine the sysadmins are at even partially at fault here. If so, then I should get my resume ready for some job openings.
Boils down to system owner policy I believe. Let`s face it, if every small/medium business had to run everything IT not to mention security solutions, the overhead would be massive. It`s not feasible and it will never be. "Everyone" have to rely upon services ++ controlled by someone else, and rely upon their reputation. Their reputation tend to be great until it isn`t, at which point it often is too late.

Trust and reputation has a friend - naivite - which tend to crash the party somewhere down the road. A lot of contigency can be had through firm and sensible policies governed by procedures and routines executed by competent people like the sys admins. Believe the important thing is to be realistic - nothing is ever 100% safe/waterproof - and never allow trust and reputation be left alone with their friend naivite. Sceptics are a pain, but letting all of them out when important decisions are made is a mistake.

One could project this upon users of personal devices. Hardly any (in decimal percentages) have competence to safeguard their personal devices properly. Anything from telling tellies, phones, "smart" house systems, cars and what not. Just about everyone have to rely upon the security and privacy provided by the brands. "All you can do" is to be up to scratch with the general threat picture and adjust accordingly.

Some argue they are protected by running their own personal servers, but for those to be secure, the sys adm better be very very good. It`s not for the average bright and competent end user. More than 20 years ago I did that stuff for educational purposes, and I realised the competence lever to keep the stuff safe is way beyond the average Joe`s capabilities. It`s a different ballgame and you need to be rather specialised.

I would argue for smaller organisation unable to carry the overhead of a full size IT department, it is a similar situation. They need to rely a lot upon external competence and services, and to handle that they need to exercise a strict and conscious regime and know what to do when it goes wrong. At one point it often do.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.