May 8, 2021 was the day my stepdaughter graduated from medical school. As you can image, it was one of the most important days in my wife’s life, and I was under strict orders: No work this weekend, we are here to celebrate the graduation and nothing else. Things did not work out as planned.
It started with a call from a friend at the Cybersecurity and Infrastructure Security Agency
(CISA) asking me what I knew about the group behind the DarkSide ransomware. Then a call came from a friend at the FBI on the same topic. I spent the rest of the day fielding questions from reporters who were finding out that the disruption at Colonial Pipeline was a ransomware attack—the most infamous ransomware attack to date.
Fortunately for me, my wife is more understanding than I have any right to expect, and, since my pictures of the ceremony turned out well, she forgave me.
We learned some important lessons from the incident, and others we are still learning. Here are some of my biggest takeaways from the Colonial Pipeline attack.
Everyone remembers the long gas lines and gas stations running out of gas up and down the East Coast. But most of the gas shortages, especially in places like Florida (which isn’t serviced by the Colonial Pipeline), were caused by panic—not actual gas shortages.
Most operational technology (OT) systems, like pipelines, have all sorts of built-in redundancies that allow them to function even after a catastrophic failure. In the case of Colonial Pipeline, the OT network was never compromised, but IT-based systems that communicated with the OT network were. This is what led to the shutdown of the pipeline. Even then, however, there were contingency plans in place to get gas delivered.
The problem was that everyone heard the pipeline was shut down, and ran out to get gas immediately. It was the run on gas stations that was primarily, though not entirely, responsible for the temporary gas shortage.
I see this type of panic happen in ransomware attack after ransomware attack. Even organizations that have well-documented and tested Incident Response (IR) and Disaster Recovery (DR) plans often panic during an actual incident. Panic during the initial response after a ransomware attack can lead to mistakes that make recovery more difficult, including a focus on the wrong area of recovery.
If your organization has created and tested strong IR and DR plans, follow those plans, and don’t panic. This will make recovery faster much more effective.
One of the biggest frustrations in trying to fight ransomware is that core members of many ransomware groups reside in Russia, and are thought to be untouchable (they certainly think so). So, while we often see ransomware affiliates get arrested, it is rare to see the leaders behind these groups suffer any consequences for their actions.
That was not the case with DarkSide ransomware after the Colonial Pipeline attack. First, law enforcement appears to have conducted a network intrusion against DarkSide’s infrastructure. Next, the U.S. Department of Justice was able to recover $2.3 million in Bitcoin from one of DarkSide’s wallets. These two actions dealt a serious blow to DarkSide, and spooked the group for at least a period of time. It certainly raised the cost of carrying out future ransomware attacks.
The point for ransomware victims is to call law enforcement. Even if it feels useless, there are a surprisingly large number of tools law enforcement has that you don’t. This may include assisting you with recovery.
This was highlighted last October when it was revealed that law enforcement had access to a decryptor for BlackMatter ransomware, the successor to DarkSide. For months, law enforcement was able to help victims decrypt files without having to pay the ransom.
Within a day of the attack, Colonial Pipeline made the decision to pay a $5 million ransom. Whether or not victims should pay ransom demands is always a hotly debated topic, and one that we aren’t going to rehash here (there is a chapter on this topic in my ransomware book).
But the reality is that sometimes, even with all the resources available to an organization like Colonial Pipeline, paying the ransom is the best option. It is certainly not the best option from a security or law enforcement perspective, but it may be the best option from a busines perspective.
If your organization is in that situation and paying the ransom is necessary, you should document everything as thoroughly as possible and alert law enforcement (as mentioned previously). Ideally, alert law enforcement before paying the ransom in case they may have alternatives that you weren’t aware of.
And although it may seem counterintuitive, do not try to hide the fact that you paid. Ransomware groups may use the fact that you hid paying the ransom to attempt to extort even more money from you at a later date. It is much better to deal with any fines or repercussions up front than endless demands for more ransom payments.
Just over a year after the Colonial Pipeline ransomware attack, the United States Department of Transportation's Pipeline and Hazardous Materials Safety Administration indicated that Colonial Pipeline may be fined almost $1 million for safety violations that likely contributed to the ransomware attack. So, long after the incident response has finished and systems have been fully restored, Colonial Pipeline is still dealing with fallout from the attack.
This is not uncommon. Most organizations don’t realize how long full recovery can take after a ransomware attack. For example, almost a year and a half after a ransomware attack on the Baltimore school system, teachers were still feeling the impact of the attack. It is not just the system recovery that takes a long time—there is the continuing impact on employees or customers who had data leaked during the attack. This can lead to a high rate of attrition among employees, especially those working long hours on recovery.
The Colonial Pipeline incident had far-reaching and long-term effects on the company, including the fact that its name is now nearly synonymous with the threat of ransomware. Learning these lessons can help you avoid becoming the next Colonial Pipeline.