Crowdstrike Global Outage: Effective Solutions To Mitigate The Impact

Around 4 AM UTC, CrowdStrike experienced a significant global outage affecting numerous users and critical infrastructures. We suspect with medium confidence a faulty Indicator of Attack (IOA) rule as the root cause, which led to system crashes and operational disruptions.

Our incident response team rushed to help our clients resolving the issue. During the analysis, we found 3 workarounds which could be implemented and at the time of the writing of this blogpost, we are looking to automated the process.

CrowdStrike serves a diverse range of clients across various industries, including Fortune 500 companies, major financial institutions, healthcare organizations, and media companies. With its comprehensive cybersecurity solutions, CrowdStrike protects critical infrastructures, highlighting the widespread reliance on its services. This global outage has emphasized the scale and impact of their client base, affecting essential operations across multiple sectors. For detailed client demographics and more insights, you can visit CrowdStrike's official website or industry reports.

Q&A

I have BSOD, How Do I Fix The Issue?

Written by Threat Researcher, Leo Valentic, who was one of the first to both detect the faulty update and also fix the problem on his own machine - those were the 3 methods he used:

Method 1

Safe Mode and Deleting Specific .sys Files:

1. Boot Windows into Safe Mode or Windows Recovery Environment.

2. Navigate to the directory:

   
   

C:\Windows\System32\drivers\CrowdStrike

1. Delete all .sys files that were modified on 19.7.2024.

2. Ensure only .sys files modified up to 18.7.2024 remain. (This should resolve the issue).

Method 2

Deleting a Specific File:

1. Boot Windows into Safe Mode or Windows Recovery Environment.

2. Navigate to the directory:

   
   

C:\Windows\System32\drivers\CrowdStrike

1. Locate the file named "C-00000291*.sys" and delete it.

2. Restart the system normally.

Method 3

Renaming the Directory:

1. Open Command Prompt with administrative privileges. (In Safe Mode)

2. Enter the following command:

   
   

ren "c:\windows\system32\drivers\crowdstrike" "crowdstrike.bak"

1. Restart the system normally.

Which update is faulty?

It seems that the faulty update was pushed around 4AM UTC.

Was the workarounds successfully tested?

Yes, we've confirmed the workarounds and contacted Crowdstrike to confirm them.

Is there a risk to non-affected machines?

No, Crowdstrike confirmed that they removed the faulty updates and there is no risk to non-affected machines.

I keep getting BSOD, what can I do?

Try all 3 methods. One should be successful with high probability.

I have Bitlocker, how do I get to Safe-Mode?

The issue is unrelated. Ask your IT admin for the Bitlocker key and enter it. Once done, enter Safe-Mode.

As the outage continuous we are researching for ways to overcome the problems.

For more assistance, please contact our team via response@securityjoes.com

Clients of Security Joes were alerted in time and received the update with the above workarounds, prior to this blog being released.

Disclaimer: This blogpost, its content and images are not replacing any formal announcements or updates that may come directly from Crowdstrike or trusted partners of Crowdstrike. Security Joes is an independent incident response company and was not hired nor requested by Crowdstrike to conduct the research around the outage, nor the release of the information publicly. The release comes as aid, to ensure affected companies and organizations recovering from outage as soon as possible.