CrowdStrike Falcon Worldwide Outage

What it is

CrowdStrike is a network security company. A Falcon Sensor is part of their Cloud-based endpoint protection platform. Think of Endpoint Protection as a firewall on each device that is part of a network. The ‘cloud’ part of it is similar to a conductor in a orchestra, with the endpoints being those playing the instruments, and the instruments are the individual computers/servers.

What caused it

CrowdStrike sent out an update to their software that conflicted with Microsoft Windows, which caused a BSOD (blue screen of death – a ‘crash’ of the software). Following the software crash, the computer gets stuck during reboot and won’t load the operating system, leaving it dead in the water.

How this affects our elections

The idiots that implemented our cloud-based voter registration and poll-book systems have created a HUGE abuse vector in our election ecosystem and I’d be shocked if those election systems weren’t also affected by this. And there is nothing that can prevent their being another accidental (or intentional) abuse! Have you considered that this may just be cover for an election hack just prior to/during an election? It would be perfect cover.

Maricopa County Voting Locations Impacted – and they aren’t alone!

Outages locally have included Maricopa County voting locations, multiple Valley police dispatch centers, several airlines at Phoenix Sky Harbor International Airport and all flights to and from Mesa Gateway Airport.

Gov. Katie Hobbs said on social media that her team is “closely monitoring all services that have been impacted and is working to ensure that we continue delivering the critical services that Arizonans rely on.”

But wait…There’s more! New update from CrowdStrike:

(Source: https://www.crowdstrike.com/blog/falcon-update-for-windows-hosts-technical-details/)

Speaking of Elections…let’s not leave out Dominion just yet…

The idiots at Dominion Voting Systems also leave their election management server database server open to Named Pipes (notice the red box above!):

Is this yet another example of their incredible incompetence? Or is it instead, intentional ‘incompetence’? And we trust them with WHAT? (And yes of course, Named-Pipes is not the only problem showing there.)

According to CISA:

For this system deemed CRITICAL INFRASTRUCTURE, how convenient for Dominion to not even follow standard STIGs. Here’s V-79185:

Did Dominion do anything else wrong?

Lol…oh boy. This is barely even the tip of the iceberg. But in Dominion’s defense, it’s not just Dominion that’s the problem. Every company that electronically centralizes any aspect of our elections creates a superhighway of attack vectors leading into our Nation’s entire foundation, our Election System. The shear incompetence and inability to protect any system in the industry from abuse is the elephant in the room. When you have a bad actor inside the company, you’re done. For instance, look at this very accurate quote from an atsec source code review of Dominion that is applicable to ANY system:

“Backdoors are extremely hard to find because a seasoned programmer can obfuscate code to look benign. The atsec team would like to stress that, when facing a competent and sufficiently motivated maliciousdeveloper, it is extremely difficult to prove that all backdoors in a system have been identified. The famous Turing award lecture by Ken Thompson in 1984 entitled Reflections on Trusting Trust [TRUST] demonstrated how fundamentally easy it is to undermine all security mechanisms when the developers cannot be trusted. This voting system is no exception.”

Yes, I know CISA claims to secure our systems. Unfortunately, the same types of incompetence in these vendors exist in CISA as well. Not to mention, they are also lying right to our face about many things.

So what is the connection between CrowdStrike and Dominion?

Well…it is interesting that CrowdStrike is intercepting Named-Pipes and Dominion also left their database connected to Named-Pipes. Coincidence? Possibly. Convenient as yet another possible attack vector? Absolutely!

So what are you trying to say?

Simply put, the people wielding this technology are wholly irresponsible (at best). At worst, what if there are bad actors at Dominion? Even worse (if that’s even possible), what if there are bad actors at CrowdStrike? What do those bad actors now have access to? How many millions of computers around the world does CrowdStrike have LOW LEVEL control of? (8.5 Million at the last count according to David Weston, Microsoft VP, Enterprise and OS Security in a blog post Saturday). Who owns CrowdStrike? Who works there? THINK ABOUT ALL THAT…

Our election officials are sitting ducks and in no way knowledgeable enough to secure (nonetheless even understand) this threat landscape. How can any election official claim their system is secure when they don’t know it to be such, and they are merely blindly believing what someone they trust tells them? What happens when those that they trust are LYING TO THEM? Our election officials need to accept the reality that is in front of their faces: They cannot control or secure that which they cannot fully see and do not fully understand. The solution is simple…boot all the electronic systems out of our elections and go back to a simple system with a much smaller and controllable threat model, then use technology to add transparency instead of obscurity.

How to fix this current CrowdStrike issue:

The affected file in the update is a particular ‘driver’ that was updated. A ‘driver’ is a program that runs on the computer that performs a task. This driver is the Falcon driver. To repair it, the affected ‘driver’ must be removed in order to allow the operating system to boot up, then the new fixed version of the driver must be installed. The huge complication here is that the driver must be removed MANUALLY. A further complication is for servers that have encrypted hard drives because extra steps must be performed to decrypt the hard drive in order for the repair to be implemented. For companies that didn’t follow best-practices on their encryption passwords, their systems will be permanently locked out and unrecoverable.

Details on repair

The morning of 2024-07-19, a content update was sent to some CrowdStrike Falcon clients on Windows devices which resulted in “Blue Screen” errors for those devices. If you have a Windows device stuck on a blue screen at boot, this issue is almost certainly the cause.

The fix for this issue requires booting the Windows device into Safe Mode or Recovery Mode and deleting a file. Instructions for doing this are below. This post and these instructions may be updated as the situation develops.

FIXING THE WINDOWS DEVICE PROBLEM

Direct link to CrowdStrike instructions: https://www.crowdstrike.com/blog/statement-on-falcon-content-update-for-windows-hosts/

If you are affected by this, we happen to know someone VERY good with solving these types of issues! Contact Mark Cook here.

Workaround Steps for individual hosts:

Workaround Steps for public cloud or similar environment including virtual:

Option 1:

Option 2:

AWS-specific documentation:

Azure environments:

User Access to Recovery Key in the Workspace ONE Portal

When this setting is enabled, users can retrieve the BitLocker Recovery Key from the Workspace ONE portal without the need to contact the HelpDesk for assistance. To turn on the recovery key in the Workspace ONE portal, follow the next steps. Please see this Omnissa article for more information.

Bitlocker recovery-related KBs:

×