Label

This Is Your
Last Chance

Sometimes, almost everything goes wrong.
Home » How To Prevent Ransomware? » Threat Hunting » This Is Your Last Chance

In This Section

The "Hail Mary"

•
Sometimes, almost everything goes wrong. An organization doesn’t detect the initial access vector, the Security Operation Center (SOC) doesn’t see the ransomware actor conducting reconnaissance on the network or didn’t notice files being exfiltrated, and the threat hunting missions fall short. With an estimated 65,000 manual ransomware attacks in 2020, unfortunately this scenario happens often. Some ransomware actors are skilled at moving through the network undetected, while understaffed, overworked security teams can’t keep up with alerts, patching schedules, security hardening, as well as keeping on top of new issues that are constantly arising.
•
In American football, when a quarterback throws a long pass to a receiver, generally surrounded by defenders, and almost always in desperation mode with very little time left to play, it’s called a “Hail Mary” pass. That’s what this page is about, a last chance to stop a ransomware attack before files are encrypted.
•
Please note that even if the detections outlined on this page work, and the ransomware attack is stopped before files are encrypted, there's still a lot of work to do. The ransomware actor has been in the network for a while, so a lot of incident response (IR) work needs to be completed quickly to fully remove the attacker, or they'll continue trying to wreck your environment.
•
In addition, it’s likely that even though the ransomware attack was stopped, sensitive files were removed from the network. This means the organization might have to deal with extortion demands and the threat of stolen files being released publicly. Interestingly, it’s probably more difficult to deal with ransomware groups after a botched ransomware attack, because they weren’t able to leave a link to their chat server or email addresses to contact them. That’s not to say that it’s better to let the ransomware attack continue, just that it may take more work if an organization needs to understand what was stolen (assuming the information can’t be determined through log analysis).

Deletion of Shadow Copies

All that being said, there are a few detections that can serve as effective tools for detecting an impending ransomware attack: shadow copy deletion and the start of the encryption process. Deletion of volume shadow copies are a signal of a ransomware attack. Detecting this activity can help an organization avoid the worst effects, if you can act quickly.

A snippet of a batch file taken from a failed  ransomware attack ...

... is shown in the screenshot below. A successful attack would execute this file on a system right before the ransomware is run. In this batch file, the ransomware actor permanently deletes the files in the Recycle Bin on every drive, then forces an update to the Group Policy Object with two commands:
Delete Shadow Volume Copies
Clear out Windows Event logs
Snippet of a .bat file left behind after a failed ransomware attack

Every—or almost every—ransomware ...

... group deletes volume shadow copies before they run the ransomware and have been doing so since at least 2014. Importantly, deleting volume shadow copies happens before the ransomware is deployed, because the ransomware actor doesn’t know whether they’re going to be kicked off the victim’s machine once the ransomware attack starts.
The Volume Shadow Copy Service (VSS) was introduced in Windows Server 2003. It was then added to Windows Vista and has been a part of every Microsoft desktop and server operating system ever since. The VSS searches through the operating system looking for changes to files and folders and indexes those changes. This creates a history of the files or folders that can be used to restore individual files or folders that are accidentally deleted, overwritten, or damaged through some other error.

Ransomware actors learned early on ...

... that having this service running diminished the effectiveness of a ransomware attack. If victims could simply restore the volume shadow copies, there was no need to pay the ransom. As far back as 2015 experts were recommending that organizations rename or remove vssadmin.exe (the built-in Microsoft command-line tool for manipulating volume shadow copies) as a protection against ransomware .
Removing or renaming vssadmin.exe wouldn’t prevent files from being encrypted, but it could make recovery easier. Still, there are some problems with this advice: It precludes the use of a lot of legitimate tools that use vssadmin.exe to manipulate volume shadow copies. Moreover, not all ransomware actors use the vssadmin.exe executable to remove volume shadow copies.
That being said, ransomware is the only program that uses vssadmin.exe to remove all volume shadow copies in a single action. In that way, ransomware is unique and this information can be used to create that Hail Mary alert to stop a ransomware attack.
School house

Why Is It There?

If no legitimate programs use vssaminddmin.exe to delete all volume shadow copies, why not remove the capability? Its intended use is to help administrators who have to manually delete all shadow copies when troubleshooting backup or storage problems. This capability isn’t used as often on earlier versions of Windows, but it’s still used by administrators.

There also may be rare times when some backup software needs to remove all volume shadow copies. This isn’t generally considered a best practice, but it’s occasionally required. In short, this function still provides utility for systems administrators who are troubleshooting storage, backup, and other problems.

Other Ways of Deleting 
Shadow Copies

While deleting shadow copies is common across ransomware variants, the methods of carrying out the deletion varies depending on the ransomware strain. Some ransomware variants, like the one showed in the previous section, rely on PowerShell scripts. Others build the ability to delete shadow copies into the portable executable (PE). The image below is taken from the leaked manual developed by a Conti affiliate, and shows the command options for the Conti ransomware.
Entry in the Conti manual showing the command flags for the Conti ransomware

All but one of the command options ...

...  (-fast) include the deletion of shadow copies. In contrast, the -fast option encrypts files “without terminating processes that use files, and without deleting Shadow copies.” In other words, using the 
-fast option risks leaving some files unencrypted and could allow some victims to recover files, though they would still face many challenges that other ransomware recoveries have.
But how does Conti delete shadow copies? The PE uses a two-step process. First, it runs a WMI Query Language (WQL) query:
SELECT * FROM Win32_ShadowCopy
This pulls a list of all the shadow copies stored on the local machine. After that, Conti calls a cmd.exe shell to delete the list of files retrieved with the first command using WMIC :

cmd.exe /c C:WindowsSystem32wbemWMIC.exe shadowcopy where "ID='%s'" delete

Using WMIC to delete shadow copies ...

... with the shadowcopy command is another common way for ransomware groups to carry out this task. Among other ransomware groups that have used this method are TeslaCrypt, Maze, and Egregor.
One last way that ransomware actors can delete shadow copies is by using PowerShell. Ransomware groups such as DarkSide, Revil, and some versions of BlackMatter (other versions of BlackMatter use WMI calls) run PowerShell commands similar to the following:
Get-WmiObject Win32_Shadowcopy|ForEach-Object {$_.Delete();}

PowerShell makes sense for many ...

... ransomware actors because it’s so ubiquitous in ransomware attacks, which creates a lot of ransomware developers who are comfortable using it to program automated tasks.
A less common method for deleting shadow copies is to resize the shadow copy storage, rather than deleting the shadow copy files. According to Microsoft, “Resizing the storage association may cause shadow copies to disappear.” Older versions of Conti and Ryuk both used this technique, combined with the deletion of shadow copies using the vssadmin.exe command:
cmd.exe /c vssadmin Delete Shadows /all /quiet
cmd.exe /c vssadmin resize shadowstorage /for=c: /on=c: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=c: /on=c: /maxsize=unbounded
cmd.exe /c vssadmin resize shadowstorage /for=d: /on=d: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=d: /on=d: /maxsize=unbounded
cmd.exe /c vssadmin resize shadowstorage /for=e: /on=e: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=e: /on=e: /maxsize=unbounded
cmd.exe /c vssadmin resize shadowstorage /for=f: /on=f: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=f: /on=f: /maxsize=unbounded
cmd.exe /c vssadmin resize shadowstorage /for=g: /on=g: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=g: /on=g: /maxsize=unbounded
cmd.exe /c vssadmin resize shadowstorage /for=h: /on=h: /maxsize=401MB
cmd.exe /c vssadmin resize shadowstorage /for=h: /on=h: /maxsize=unbounded
cmd.exe /c vssadmin Delete Shadows /all /quiet

Conti and Ryuk weren't the only ...

... ransomware variants to use this technique. The Hakbit and MedusaLocker ransomware also ran the same commands prior to encryption.
Although there are slight variations in the commands ransomware groups run, the ones discussed in this section are the most common methods they use to delete or resize shadow copies before deploying the ransomware.
School house

Why 401MB?

Why would ransomware actors reduce the shadow storage size to 401MB? That seems like an oddly specific number, but doesn’t match any of the usual limits on Windows machines. I wasn’t sure if there was a specific reason for this number, or if one ransomware group had picked it and then everyone followed (as often happens). So, I asked the question on Twitter.
The best answer that came back was from Twitter user @lwolive, who found an article on the Picus blog mentioning that the minimum size for ShadowStorage is 320MB. Trying to run the resize command with anything less than 320MB returns the error: “Error: Specified number is invalid,” with a further note, “or byte level specification, MaxSizeSpec must be 320MB or greater …” Why the ransomware actors picked 401MB remains anyone’s guess; it may have been trial and error. But once one group used that value, it’s likely that other groups just copied what the first group did without thinking about it.

Starting the Encryption Process

After shadow copies have been deleted ...

... or effectively deleted by having ShadowStorage reduced, the ransomware PE needs to run through several system checks before starting the encryption process. Some of the checks the ransomware must make include (but aren't limited to):
Enumerating all drives on the local system
Searching for network drives
Closing running processes that might prevent files from being encrypted, especially anti-virus and other security vendors
Importing the public key and generating the private key (some ransomware variants embed the public key in the executable, so that they don’t have to make command-and-control callouts during the deployment stage)
Changing the background image to show the ransom note (some ransomware variants)

When a single PE engages in all of these ...

... activities, especially in rapid succession, it should always generate log entries that should create a critical alert. It’s important for organizations to keep in mind that not all ransomware groups embed this activity into the PE: Some rely on PowerShell scripts or batch files to carry out some of the tasks, leaving the PE just to do the encryption. But, it doesn’t change the fact that these steps happen in quick succession, even if they’re carried out by different executables, so these activities should still generate an alert in the SOC.
The majority of today’s most active ransomware groups—including Conti, LockBit, BlackMatter, and REvil—embed these functions into the PE. On the other hand, both the Pysa and Grief ransomware PEs don’t have built-in functionality to delete shadow copies, instead relying on the affiliates to carry it out with scripts.
This is an important distinction for the next section, which deals with detecting and responding to this activity. Stopping and isolating the process carrying out shadow copy deletion may not be enough to stop the ransomware attack from progressing.

Endpoint Detection and Response 
+ Automation Is Your Friend

Organizations that understand the importance and prevalence of manipulating shadow copies can now put in protections to alert them when this is happening and stop ransomware attacks. Right? Unfortunately, it’s not quite that easy. The diagram below shows the difference in time between the speed of operation ransomware PE operates versus the time it takes to generate an alert.
After shadow copies have been deleted, the difference in time between encryption starting versus the SOC receiving an alert

A number of different log sources can ...

... produce an alert indicating that shadow copies have been deleted. If an organization has endpoint detection and response (EDR) running on their endpoints, a log event is generated from the EDR platform. In addition, there are indicators in PowerShell logs and Sysmon and, to a limited degree, in Windows logs.
The problem, as outlined in the diagram above, is getting log data from the endpoint to the SIEM and then producing an alert in a timely fashion. There are many examples of organizations not processing an alert from an event until well after the damage has been done—specifically, in the case of ransomware, after encryption has happened.
Many types of cyberattacks leave some room for error with alerts that are delayed as logs are processing. Even with earlier stages of a ransomware attack, there’s room for delay in alerting. Because of the tiny time gap between deleting shadow copies and encryption, there’s no room for delay in this stage of a ransomware attack.

Automation

This is one of the areas in which ...

... automation can really help security teams get ahead of a threat. Rather than wait for the logs to generate an alert and the SOC or security team to act on it, you can automate the process of identifying malicious use of shadow copy manipulation and stop that activity immediately. Doing so can possibly stop a ransomware attack in progress.
One way to automate alerts is in a security orchestration, automation, and response (SOAR) platform. Using SOAR, organizations can build playbooks that collect information from different systems and use that information to take action automatically. For example, Splunk has a prebuilt alert for detecting malicious shadow copy manipulation via PowerShell. A snippet of a Splunk alert is shown in the screenshot below.

The first step in stopping the ransomware attack ...

... is shown in the above image. The file generates an alert indicating that the shadow copy manipulation is happening. The next step is to automate the actions that need to be taken.
These actions may include:
Sending instructions to the EDR to kill the process that called the PowerShell script
Blocking the hash of that process on the rest of the endpoints
Temporarily disabling the user who initiated the process
Shutting down the infected machine
The nice thing about a well-configured SOAR platform is that it can conduct all of these operations in a matter of seconds. It may not be enough to prevent the first machine in the attack from being encrypted, but it can possibly save other machines. This makes the ransomware recovery process much more manageable (though the organization will still likely have to deal with the challenges that come with stolen files being used to extort the organization).

Get More Tools To 
Fight Ransomware 

Sign Up To Receive Our 
Monthly Ransomware Newsletter

Don't Worry, We Hate Spam Too

Hitting the Panic Button: Stopping 
a Ransomware Attack NOW!

Not every organization has a SOAR ...

... platform, but there are other ways to generate immediate alerts when shadow copies are manipulated.
Unsurprisingly, there’s a Sigma rule that helps detect this activity. The screenshot below shows a Sigma rule that a half-dozen people have contributed to, looking for all the common ways that ransomware actors manipulate shadow copies.
Sigma rule for detecting common forms of shadow copy manipulation

The rule includes manipulation via ...

... PowerShell, WMIC, and vssadmin.exe, along with many of the common command options that attackers use. Loading this rule into an EDR to take automatic action can allow an organization to stop the shadow copies from being deleted and stop the ransomware attack.
The process is the same as in the previous section. In this case, the EDR is doing all the work, but the actions are still the same:
Killing the process that called the PowerShell script
Blocking the hash of that process on the rest of the endpoints
Temporarily disabling the user who initiated the process
Shutting down the infected machine
Not every organization has an EDR solution in place. With no SOAR and no EDR it’s very difficult to detect, alert, and act on shadow copy manipulation in a timely fashion. EDR and SOAR solutions take a lot of time and effort to properly maintain, but the benefit they provide is automation for times where stopping an attack quickly is critical. It’s not that the logs won’t be sent or that alerts won’t be generated. The problem is being able to act on those alerts.

For smaller organizations, a tool called ...

... Raccine developed by Florian Roth can stop shadow copy manipulation on endpoints. It has the advantage of not being commonly used, so ransomware groups aren’t looking for it. The way Raccine works is by registering a debugger for the common tools used by ransomware groups to manipulate shadow copy files. When one of those methods is detected, Raccine kills the process and generates a log message that alerts the security team to investigate further.
Raccine is a good solution that stops many types of ransomware variants from manipulating shadow copies and hopefully grants security teams the time they need to stop a ransomware attack. As with any other security solution, it should not be the only solution, but one of many working together.
None of these solutions is completely perfect; there’s always potential for failure. However, if all other alerts are missed, these last-ditch solutions may prevent a ransomware attack from destroying an organization. Detecting the deletion of shadow copy files isn’t the only thing that ransomware groups do before deploying ransomware, but it’s the only action that’s consistent across all ransomware groups. There may be different ways of doing it, but they all do it. They also do it in a way that’s almost always indicative of a ransomware attack, making it a unique Hail Mary detection for ransomware.
Pay attetntion

Is EDR Actually Running?

The detections described here work only if the ransomware actor hasn’t terminated the EDR process. It has been mentioned several times on this site that ransomware actors start an attack by shutting down any security solution they can, including EDRs. This is why alerting and acting on those shutdowns is so important. Organizations that are relying on EDR for this type of protection need to ensure that the EDR is actually running.

If You Liked This... 

Get The "Go-To" Ransomware Resource. The 437 Page Book Ransomware: Understand. Prevent. Recover.  
- Second Edition

GET THE BOOK HERE
Ransomware Book 2nd Edition 2023

Download The 
"How To Prevent Ransomware"
Cheat Sheet

Grab this free PDF resource on how to prevent Ransomware
DOWNLOAD THE PDF

Share This Resource With Others

Embed The "How To Prevent Ransomware" resource on your site or blog using this code.

Share this Infographic On Your Site

Want More?

This site is adapted from a book on Ransomware. 
If you would like to learn more keep reading ...
READ MORE ABOUT RECOVERING FROM A RANSOMWARE ATTACK
© Future US LLC, Full 7th Floor, 130 West 42nd Street, New York, NY 10036
envelope linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram