Home » How To Remove Ransomware? » Recovering From A Ransomware Attack » Implementing DR and IR Plans

Implementing DR and IR Plans

After your initial response. You have assessed the damage after containing it, what’s next? It’s time to implement your Disaster Recovery and Initial Response plans.

At This Point in the Ransomware Attack

• The attack has been contained, and the damage has been limited
• Initial triage has been completed and the scope of the attack is known
• Inventory of the infected systems and their data has been completed
• Relevant stakeholders have been notified of key information, including the communication plan going forward
• Incident response (IR) and disaster recovery (DR) plans have been retrieved from their secure location

Now the organization is ready to move from initial response, which is focused more on immediate damage mitigation, to IR, which is more focused on triage, investigation, forensics, and analysis. Here’s where the IR and DR plans put together by the IT and security teams (see: “Creating Disaster Recovery and Incident Response Plans” page) are going to be so important. There’s going to be a lot of pressure from all sides to get different services turned back on quickly, but the plans are there for a reason. Follow them unless an extenuating circumstance requires a departure from them.

Any deviation from the documented plans should be authorized by senior leadership. This rule empowers your team to tell any person or department requesting a change that they have to go through leadership. After all, it’s the leadership of the organization that has to decide the priorities of the organization.

It is possible that after the initial triage, the damage from the ransomware attack might turn out to be minimal and everything will be fully restored in a few days. But that’s rarely the case. As always, organizations should plan for the worst and hope that the thoughtful planning, combined with talented IT and security teams, prevent the worst from coming to pass.

NOTE: This page assumes you have a Disaster Recovery (DR) plan in place. If you need help creating a DR plan, consider reviewing these two resources:

PAY ATTETNTION

Get Some Rest!

The health advice that follows applies to security leadership, as well. The IR and DR plans should have a clearly defined list of leaders for each team, and those leaders should be working on a rotating schedule like the recovery teams. If the recovery process is well-documented, it should be easy to switch out the leadership team so that everyone is able to get some rest. Who’s in charge and at what times should be communicated to employees and senior leadership so that people aren’t getting phone calls while they’re trying to rest.

Take Care of the Basics: Food and Shelter

Security leadership needs to build out a shift schedule for the IR and DR teams indicating who will be working when. The response for the first few days, while critical systems are restored, might be around the clock. That doesn’t mean that everyone from all the teams has to be present. Tired people make more mistakes, so while the hours are going to be long, ensure that all working employees have down time and off time. While getting everything up and running is critical, keeping everyone healthy is more important in the long run. Consider appointing someone from outside the IR and DR teams to be responsible for ensuring the mental health of the response teams.

Unless an organization is lucky enough …

… to have extensive IR and DR teams, some people will be working very long shifts for several weeks. Consider getting a block of rooms at a hotel near the office for people who need to crash, but might live far away or have been brought in from another office. Make sure everyone can get as much rest as possible. Keeping the IR and DR teams safe, by not driving long distances home after a long day, is really looking out for the teams.

Also, as discussed on the “Creating Disaster Recovery and Incident Response Plans” page, start feeding the teams who are expected to be working these long shifts. It seems like a minor thing, but providing food and drinks to everyone, especially if everyone is working around the clock, has three benefits:

Breaks are just as important, too!

Bringing in food and drinks doesn’t mean that people shouldn’t step out of the building and take breaks. Exercise is important during these long days. So, encourage people to take regular breaks, get outside and walk (if permitted by the weather and local environment). If the building has a gym, give everyone access to it. Not only do such breaks help keep people focused on the task at hand, they’re good for the mental health of the IR and DR teams and can help alleviate some of the frustration that’s naturally a part of any IR or DR situation.

It may seem like this page has spent a lot of time on the subject of food and shelter, but a ransomware attack can be incredibly demoralizing to IT and security teams, as well as to companies as a whole. Companies have been forced into bankruptcy or even to shut down after a ransomware attack . Organizations that are actually resilient may have to deal with months of news coverage, depending on the size of the organization and the industry.

Touches like providing food and shelter and watching out for the mental health of the IR and DR teams can improve employee morale and result in a more successful recovery.

PAY ATTENTION

Take a Picture with Your Phone?

Some ransomware response advisories recommend taking a picture of the ransom note on one of the screens with a smartphone. This can be helpful if the IR team is unsure what the ransomware variant is and wants to check with third-party sources such as ID-Ransomware or No More Ransom. But, almost always, it’s easier to deal with the text in the ransom note than a photo of it.

It can’t hurt to take a picture. Just be sure to delete it when the IR ends, so it doesn’t show up as a memory every year on the anniversary of the attack.

Find the Initial Access Vector and Shut It Down

The first priority of the organization is likely to get systems back up and running so that everyone can get back to work. Resist that urge. Hopefully, the IR and DR plans stress that the first priority needs to be finding the initial access vector and shutting it down.

Before jumping into DR, forensic images …

… need to be made of the infected systems. It used to be that IR firms and government agencies wanted the physical hard drives from encrypted machines, but most of the time a forensically sound image created by a tool such as FTK® Imager (from Exterro) will be enough. This procedure should always be verified through the legal team in consultation with IR, though, and whatever process an organization chooses should be well-documented in the IR and DR plans.

Now the IR team can start inspecting the known infected machines to see what they can find out about the attack, while ensuring that it’s fully contained. This process will likely begin within a couple of hours after the attack is fully contained (with the caveat that if the organization needs to bring in an outside IR team—discussed on the “When You Need Outside Help” page—there may be a slight delay).

If infected machines were able to …

… stay powered on and isolated, the IR team can start going through them to extract information needed for the investigation. Some of the items that should be copied and pulled off the machines include:

The ransomware portable executable (PE)
The ransom note
PowerShell scripts left behind on the system, some of these might be difficult to identify, in some cases it might make sense to pull all PowerShell scripts from the infected machine
Third-party tools that may have been part of the attack
Windows event logs
PowerShell logs
Sysmon logs
A sample of an encrypted file
Contents of RAM (assuming that the machine hasn’t been powered down)

Make copies of these files instead of …

… pulling the original files from the encrypted machine. Pulling the original files off can cause the ransomware decryption process to be corrupted, which can make later decryption impossible in the event that a decryptor is available for the ransomware or an organization pays the ransom.

The data collected from the first machine serves two purposes:

It often helps to build out a diagram, as shown in the image below, documenting the process of retracing the ransomware attack. The IR team should try to trace the attack back to the initial access vector as best as they can with the available evidence, realizing that it’s always possible that a script or other indicator was missed.

Retracing the steps of the ransomware attacker back to the initial access vector

Again, as the IR team is retracing the …

… steps of the ransomware actor, they should build a catalog of all the tools used during the attack, as well as any commands that were run by the ransomware actor, including Windows-native commands. If the ransomware actor managed to zero out the local log files, the IR team will need to do its best to match up timestamps with logs from the SIEM. Hopefully, logs from the endpoints are being sent to the SIEM in near-real time.

Another often overlooked source of valuable data for tracking the ransomware actor’s movements is NetFlow logs. Not every organization collects NetFlow data, because NetFlow data, like Windows event logs, requires a lot of storage, and because it can be difficult to filter out meaningful alerts. NetFlow data does have the advantage of being difficult for ransomware groups to tamper with, because it’s collected at the network level rather than the system level (assuming, of course, that the ransomware actor doesn’t encrypt the server hosting the NetFlow data). Organizations that do have NetFlow data might be able to trace the attack back to the initial access vector more quickly, based on how the actor was moving around the network.

IR teams also need to keep an eye out …

… for any administrative accounts that might have been created by the ransomware actor, both local and network administrative accounts. Search for and remove such accounts on any clean systems, along with other indicators.

If at any time the IR team isn’t sure whether they’ve collected everything they should, consider using a known reference such as the SANS SCORE Security Checklist to flag missing information. As with everything else discussed here, known references are meant to be generic, so not every organization can gather all the data suggested. But these references are a great tool for sparking ideas the IR team may have missed.

IR teams should also be on the lookout for files that might have been exfiltrated in the attack. This information can almost always be found in the log files. Things to look for include:

Often, ransomware actors forget to …

… delete the compressed archive they created with the stolen files. Unpacking this archive tells you quickly which files the attackers took.

While one part of the IR team is collecting evidence, another part can start building out the custom detections for the clean network. Test the machines that don’t appear to have been infected by the ransomware attack to ensure that the ransomware actor left no traces.

The indicators from the infected …

… machines can be used to create YARA or Sigma rules or be fed into the endpoint detection and response (EDR) or IR platform directly as indicators (file names, hashes, IP addresses, or domain names). Many EDR platforms can isolate machines on the network so that they can communicate only to the EDR server. Using a platform like an EDR will allow the IR team to quickly scan hundreds or thousands of machines for indicators specific to the attack. As network segments are confirmed to be free of malware, they can be brought back online, allowing employees to begin to get back to work.

That still doesn’t mean that everything will be functional because ransomware actors like to target servers in the network. Endpoints can probably come back online quickly, but many services in the organization will remain offline.

WATCH OUT

Security Operation Center

As each network segment is brought online, the Security Operation Center (SOC) should be monitoring all network traffic closely to look for command-and-control communication by tools that the ransomware actor left behind and went unnoticed. The SOC should also watch for unusual processes running on these endpoints, once network access is restored. As frustrating as it may be, the DR team should bring online only as many endpoints as they can closely monitor until they’re confident that no remnants of the ransomware actor remain on the network. Remember, during the recovery process the role of the IR team is to find and remove all elements of the ransomware attack and set parameters for restoring service to endpoints and servers. The role of the DR team is to actually restore those systems.

Prioritizing Service Restoration

Once the IR team has successfully identified the ransomware used in the attack and understands the tactics, techniques, and procedures (TTPs) of the ransomware actor, it’s time for the DR team to start restoring services.

Restoration should be done in the order …

… outlined in the DR plan. It’s unlikely that the DR plan could account for every possible combination of servers that will get encrypted. There isn’t necessarily a rhyme or reason to the way ransomware actors traverse the network. They act solely on their ability to gain access, and on guessing which servers appear to have the most interesting files and will cause the most disruption by going down.

This may create some conflict with the DR plan as outlined. Each team in the conflict can make their case to leadership, who will then make the decision as to how to proceed. Updates to the DR plan should be carefully documented, like the other steps up to this point. When all the updates are finalized, restoring from backup can begin.

Get More Ransomware Tools Directly In Your Inbox

Don’t worry, we hate spam too

Restoring from Backups

Assuming that the organization has taken the proper steps to secure their backups so they weren’t encrypted by the ransomware actor, the moment of truth has arrived: The first full, post-attack restore from backup. Remember, this will be a restore from the last full backup, not an incremental backup, so these restores will be longer than an incremental restore.

Even though the encrypted servers have …

… been imaged and can successfully be wiped clean, rebuilt, and restored, many IR experts recommend installing and restoring to new hardware. This isn’t always possible, because most organizations don’t have a lot of spare servers in storage—certainly not enough to account for a devastating ransomware attack. However, whenever possible, it’s better to restore to new hardware rather than reusing the old hardware simply because it’s possible that an indicator was missed. There’s no indication, for example, that ransomware actors infect the BIOS of a machine, but other groups do and it’s possible that ransomware actors may adopt these tactics. New hardware helps to ensure that it’s a completely clean system.

The “Tabletop Exercises” and “Backup Strategy” pages discuss testing backups, but this is the real test: How quickly can the DR team conduct a full restore on a critical server and how much data is permanently lost? Despite all the testing of backup systems, this step in recovery is likely to be a nerve-wracking event for even the most experienced DR teams.

Once the first system has been fully …

… restored, run the same IR checks that were run on the systems in the clean network. At this point, the IR team may not know for sure how long the ransomware actor was in the network, and the organization wants to ensure that no remnants from the ransomware attack are re-introduced into the network.

After a restored system has been thoroughly tested and passed the IR checklist, it can be moved to the clean network and employees can use it again. Just as with the other clean systems, it should be closely monitored by the SOC in case something was missed.

Once you’ve successfully redeployed the …

… first server and created a checklist of the steps you took, the DR team can start working on multiple servers simultaneously. The number of servers that can be restored simultaneously depends on the size of the DR team and the amount of bandwidth available to and from the backup servers.

While part of the DR team is restoring the servers, others need to wipe out and rebuild endpoints. As with servers, it’s better to provision new equipment than to wipe and restore the encrypted devices, in case there’s additional malware embedded in the BIOS or other system component. Depending on the number of endpoints encrypted in the ransomware attack, that might not be a viable solution.

Most organizations back up only selected employee desktop systems, if they back up any at all. If the organization doesn’t have backups to restore, the job of provisioning new endpoints could fall to the IT department through their normal process (assuming the IT department hasn’t been recruited to conduct DR). Having the IT department provision new endpoints to affected employees will bring them at least partially online faster.

WATCH OUT

If the Initial Access Vector was a Phishing Email ...

… the IR team should scan employee inboxes before bringing their endpoints online to see whether that same phishing email message is present. Ransomware groups often send the same phishing email messages to multiple employees. Deleting that message from the employees’ inboxes before bringing their endpoints back online could help prevent a re-infection.

Communicate, Communicate, Communicate

While the IR and DR activities are proceeding, the larger response team has a lot of other work to do, starting with communication. Especially during the early stages of the ransomware attack, communicating with important stakeholders helps keep the recovery process running smoothly. People are surprisingly willing to forgive delays from a ransomware attack as long as they’re kept apprised of the situation.

The “Initial Response” page discussed …

… communication with employees and senior management, but there are a number of other people who now probably need to be informed of the attack. The timing and messaging in communication with different groups varies by organization, and is likely decided at least in part by the legal team. But some of the groups who will need to be notified include:

Law enforcement
The U.S. Cybersecurity and Infrastructure Security Association (CISA)
Clients
Partners and vendors
Reporting agencies
The cyber insurance provider
Outside IR sites

There may be other groups that need to be contacted specific to the organization. Again, the list of groups should be determined in advance.

Depending on how disruptive the ransomware attack is to the general public, the organization may start getting calls from the press. The IR team has to come up with a response to press inquiries (approved by senior management), and designate someone to speak officially to the press on behalf of the organization. It generally should fall on the PR team to carry out that task.

Leaked chats between the BlackMatter ransomware group and someone impersonating a victim

There is another way that information …

… about a ransomware attack may leak. The screenshot above shows the chat negotiation between the BlackMatter ransomware group and a farming cooperative from Iowa, called New Cooperative. That’s not an example of the victim being frustrated at having to deal with a criminal organization. Instead, someone else is “trolling” the BlackMatter group.

How did that happen? At that time, the BlackMatter ransom note, shown in the image below, included a link to a “private” section of their portal that had the ransom demand, samples of exfiltrated files, and a chat application the victim could use to chat with the ransomware group.

The ransom note left for New Cooperative after the BlackMatter ransomware attack

The private section turned out not …

… to be all that private. Anyone who had the ransom note could access that portal and the chat, and many did. Either the EDR used by New Cooperative or one of its IR team members uploaded the sample to VirusTotal for analysis. Researchers found the sample, which isn’t uncommon because researchers are always looking for new ransomware samples. Normally, this would all happen fairly quietly, but since New Cooperative is considered critical infrastructure, it became front-page news and brought even more attention to the insecure private portal.

In addition to threatening recovery, the trolling most likely created a communication mess for New Cooperative. It could no longer effectively communicate with the ransomware group, and suddenly reporters from all over the country were reaching out to find out more about the attack.

BlackMatter has since changed the way their portal works, but other ransomware groups have not. If an organization’s IR plan includes uploading a sample of the ransomware PE to VirusTotal or another analysis engine for additional information, it’s important to note that this may result in additional scrutiny. The PR team needs to be prepared in the event that its ransomware attack goes “viral.”

WATCH OUT

Before You Upload Samples!

Please note that uploading samples to public analysis engines, as described here, is risky and should be carefully considered before doing it. Doing so can disrupt both IR and DR processes and generate a lot of unwanted attention. Not only should great thought be given before doing this manually, you should also check to make sure none of your security tools are uploading these files without your knowledge.

Ignore Pressure from the Ransomware Group

At some point, the victim is going to …

… hear from the ransomware group. They encrypted endpoints and perhaps stole files, and now they want the victim to pay their demanded ransom. If the victim organization doesn’t log into the chat because they’re restoring from backups and aren’t worried about the stolen data, the ransomware group will start emailing people within the organization demanding payment. If that doesn’t work, they’ll start emailing third parties, encouraging them to contact the victim to pay the ransom.

The Allen Independent School District (ISD) in Texas learned what it was like for a ransomware group to bring in outside pressure. When the school suffered a ransomware attack, officials had good backups and didn’t feel it was worth negotiating with the ransomware group to get the stolen files deleted. The ransomware group grew frustrated, so they sent an email to staff and parents, a snippet of which is shown in the screenshot below.

Part of an email sent to Allen ISD parents after the school refused to negotiate or pay the ransom

This meant that in addition to …

… trying to recover from the ransomware attack and get services restored, the school had to field queries from concerned parents.

If the victim does engage in the chat with the ransomware group, the negotiator for the ransomware group generally engages in more high-pressure tactics that try to force the victim to make a payment quickly. The image below shows how one of Conti’s ransomware negotiators suggested that they have a buyer lined up for the victim’s data.

Conti ransomware negotiator claiming to have buyers looking to acquire the victim’s data

In the image below, the Conti ransomware negotiator increases the pressure, letting the victim know they need a decision immediately or data will be posted to the extortion site. They also inform the victim that they’ve started to reach out to customers and partners of the victim, informing those parties of the ransomware attack.

More high-pressure tactics from the Conti ransomware negotiator

In addition to the pressure from inside the organization, the response team can expect increasing pressure from the ransomware group either directly or indirectly. That’s why it’s so important to stick to the IR and DR plans as much as possible and to continuously communicate with all stakeholders. If customers and partners don’t receive regular updates from the victim, all they’ll have to go on is what the ransomware group is telling them, even though ransomware groups regularly lie.

Prepare Everyone for a Long Slog

At this point, it’s likely day three or four …

… of the ransomware attack. The initial response, IR, and DR teams have gotten into a rhythm and progress is being made. But it will probably be weeks before all systems are fully up and running, and months before the recovery is complete.

Once again, communication is important at this stage. Letting everyone know what services have been restored and what the timeline is for other services helps to set expectations. There will also likely be unexpected setbacks along the way, which will undoubtedly affect the timeline. If things do go wrong, the organization may need to bring in outside help. Learn when and how to do that on the “Outside Help” page.

Get Your Copy of Ransomware:
Understand. Prevent. Recover

It’s the ransomware resource you can’t afford to be without. 437 Pages of ransomware know-how. Stay ahead of the cybercriminals: get your copy now!

Download The Free 313 Page Book: Ransomware Understand. Prevent. Recover

Download The "How To Recover From Ransomware" Cheat Sheet

Grab this free PDF resource on how to prevent Ransomware

Share This Resource With Others

Embed The “How To Recover a From Ransomware Attack” resource on your site or blog using this code.

<p><strong>Please include attribution to ransomware.org with this graphic.</strong><br /><br /><a href='https://ransomware.org/how-to-remove-ransomware/'><img src='https://ransomware.org/wp-content/uploads/2022/08/how-to-remove-ransomware-infographic.png' alt='How To Recover From Ransomware' width='400' border='0' /></a></p>

Implementing DR and IR Plans

In This Section

Take Care of the Basics: Food and Shelter

Find the Initial Access Point and Shut It Down

Communicate, Communicate, Communicate

Prepare Everyone for a Long Slog

At This Point in the Ransomware Attack

Get Some Rest!

Take Care of the Basics: Food and Shelter

Take a Picture with Your Phone?

Find the Initial Access Vector and Shut It Down

Security Operation Center

Prioritizing Service Restoration

Get More Ransomware Tools Directly In Your Inbox

Restoring from Backups

If the Initial Access Vector was a Phishing Email ...

Communicate, Communicate, Communicate

Before You Upload Samples!

Ignore Pressure from the Ransomware Group

Prepare Everyone for a Long Slog

Get Your Copy of Ransomware:
Understand. Prevent. Recover

Download The Free 313 Page Book: Ransomware Understand. Prevent. Recover

Download The "How To Recover From Ransomware" Cheat Sheet

Share This Resource With Others

Want More?

Need Help?

Ransomware

Partners

Deep Dive

Is This Your Business?
Get In Touch

Before You Go!
Sign up to stay up to date with everything ransomware

Free Download Now &
Stay Ahead In Future

Implementing DR and IR Plans

In This Section

Take Care of the Basics: Food and Shelter

Find the Initial Access Point and Shut It Down

Communicate, Communicate, Communicate

Prepare Everyone for a Long Slog

At This Point in the Ransomware Attack

Get Some Rest!

Take Care of the Basics: Food and Shelter

Take a Picture with Your Phone?

Find the Initial Access Vector and Shut It Down

Security Operation Center

Prioritizing Service Restoration

Get More Ransomware Tools Directly In Your Inbox

Restoring from Backups

If the Initial Access Vector was a Phishing Email ...

Communicate, Communicate, Communicate

Before You Upload Samples!

Ignore Pressure from the Ransomware Group

Prepare Everyone for a Long Slog

Get Your Copy of Ransomware: Understand. Prevent. Recover

Download The Free 313 Page Book: Ransomware Understand. Prevent. Recover

Download The "How To Recover From Ransomware" Cheat Sheet

Share This Resource With Others

Want More?

Need Help?

Ransomware

Partners

Deep Dive

Is This Your Business? Get In Touch

Before You Go! Sign up to stay up to date with everything ransomware

Free Download Now & Stay Ahead In Future

Get Your Copy of Ransomware:
Understand. Prevent. Recover

Is This Your Business?
Get In Touch

Before You Go!
Sign up to stay up to date with everything ransomware

Free Download Now &
Stay Ahead In Future