Kaseya Raced to Patch Before Ransomware DisasterDutch Researchers First Notified Kaseya of Vulnerabilities in April
Global software vendor Kaseya worked in earnest for three months to resolve flaws in its VSA monitoring and management software but ultimately lost the race with ransomware attackers, Dutch researchers say.
On Wednesday, the researchers who had found flaws in VSA released a timeline and description of issues that give more context to the engineering challenges Kaseya faced.
The researchers with the Dutch Institute of Vulnerability Disclosure, or DIVD, found seven vulnerabilities, six of which affected the software-as-a-service and on-premises versions of VSA and one of which only affected the on-premises version.
VSA, which is widely used by managed service providers, facilitates making changes and updates to systems remotely, making it a powerful tool.
Unfortunately, those powers were co-opted by attackers affiliated with the REvil ransomware group. On July 2, the attackers used a series of clever tricks to exploit VSA and distribute ransomware to up to 60 of Kaseya's MSP customers. Then, the ransomware was distributed to between 800 and 1,500 of those MSPs' customers, including grocery chains, schools and restaurants (see: Kaseya: Up to 1,500 Organizations Hit in Ransomware Attack).
None of Kaseya's SaaS customers for VSA were affected. Kaseya turned off its SaaS platform shortly after detecting the attack, which was the "nuclear" option, writes Frank Breedijk, one of the DIVD researchers who found the vulnerabilities. But Kaseya didn't have that kind of control for on-premises users, which had to be warned to take VSA offline.
The damage is still being tallied, but reports indicate some victims may be trying to negotiate with attackers. The ransomware group has also offered to sell a decryption tool, which it claims fixes the problems for all victims, for $70 million in bitcoin.
In a Tuesday video, Fred Voccola, CEO of Kaseya, defended how his organization has handled the breach but acknowledged the victims should be "very, very frustrated." He said the number of victims is a small percentage of his company's customer base, which is around 35,000 organizations.
"For the 50 or so customers of Kaseya that have experienced a breach, I hope this message does not sound like we're diminishing it by saying less than .01 percentage of our customers were breached,” Voccola says.
Kaseya's Response 'On Point'
DIVD's timeline shows it started its research into VSA on April 1 and reported its first issue to Kaseya on April 6.
Four days after that, Kaseya issued its first patch. The process continued over the next three months; Kaseya issued fixes on May 8 and June 26 (see timeline below). Breedijk writes that Kaseya allowed DIVD to vet patches it had developed.
"As we stated before, Kaseya’s response to our disclosure has been on point and timely unlike other vendors we have previously disclosed vulnerabilities to," Breedijk writes. "They listened to our findings."
The vulnerabilities disclosed to Kaseya were:
- CVE-2021-30116: A credentials leak and business logic flaw, to be included in the forthcoming 9.5.7 update for on-premises; fixed in SaaS on June 26;
- CVE-2021-30117: A SQL injection vulnerability, resolved in May 8 patch;
- CVE-2021-30118: A remote code execution vulnerability, resolved in April 10 patch (v9.5.6);
- CVE-2021-30119: A cross-site scripting vulnerability, to be included in 9.5.7;
- CVE-2021-30120: 2FA bypass, to be resolved in v9.5.7;
- CVE-2021-30121: A local file inclusion vulnerability, resolved in May 8 patch;
- CVE-2021-30201: A XML external entity vulnerability, resolved in May 8 patch.
At the time of the ransomware attack on July 2, four of the seven vulnerabilities reported by DIVD to Kaseya had been fixed. Breedijk writes that one of the vulnerabilities used in the attack is one that his organization previously reported to Kaseya, but he doesn't say which one. Full details of all of the flaws won't be released until Kaseya has completed rolling out patches, he writes.
"Ever since we released the news that we indeed notified Kaseya of a vulnerability used in the ransomware attack we have been getting requests to release details about these vulnerabilities and the disclosure timeline," Breedijk writes. "And, while we feel it is time to be more open about this process and our decisions regarding this matter, we will still not release the full details."
Kaseya said that as of Wednesday afternoon, it was still working to restore its SaaS systems. After those are restored, Kaseya says it will issue patches for on-premises customers.
Victims in 17 Countries
The cybersecurity firm ESET notes that, according to its telemetry data, Kaseya ransomware attack victims span at least 17 countries, including the U.K., South Africa, Canada, Argentina, Mexico, Kenya and Germany.
The ransomware used in the Kaseya attack contains code that will avoid running on systems set to Russian and other languages in Eastern European countries, according to Trustwave SpiderLabs. This has been noticed in other ransomware programs as well, according to cybersecurity blogger Brian Krebs.
The SpiderLabs researchers analyzed malware found on the system of one of its customers, which was using an on-premises Kaseya VSA server.
The file was a digitally signed DLL with a file named mpsvc.dll, and this dll was the REvil ransomware payload - in this case, version 2.0 of the malware - the researchers say. As was previously noted by Kaspersky, this dll was side-loaded by a legitimate Microsoft executable (MsMpEng.exe). MsMpEng.exe is benign and part of the Microsoft Antimalware Service. An older version was used by the attackers.
"When MsMpEng.exe runs, it picks up the attacker's 'mpsvc.dll' and loads an exported function from the malicious dll called ServiceCrtMain(). This function unpacks and loads the ransomware into the memory and executes it," Trustwave says.
Trustwave also discovered the MsMpEng.exe and mpsvc.dll were both installed in the infected system by a dropper named Agent.exe, which is widely used to update software, so it's difficult to detect when it is doing something malicious.
The ransomware also loaded the legitimate pen-testing tool Cobalt Strike, installing a beacon that created a communication channel between the attacker and victim, Trustwave says. Cobalt Strike can also be used to exfiltrate data and move laterally through a target system (see: Attackers Increasingly Using Cobalt Strike).
Hitting Cybercriminals 'In the Pocketbook'
Attacks by REvil and other ransomware gangs are leading the government to look for ways to disrupt the gang's profit-making efforts.
"There is currently an 'all hands on deck' approach to these ransomware attacks by the U.S. government," says William Callahan, a former Drug Enforcement Administration official who now works at Blockchain Intelligence Group. "However, just like other crimes, the government must continue to attack the financial infrastructure of transnational criminal organizations. Today, it's a must for law enforcement officers to acquire skills to follow the money and 'follow the coin.'"
Brett Callow, threat analyst at the security firm Emsisoft, adds: "This is the first time that for-profit cybercriminals have struck at such a scale, and the incident really highlights the need for the U.S. government to act quickly and decisively. Cybercriminals are more motivated and better resourced than ever before. Unless we find ways to cut off the flow of cash and remove their incentive to attack, the situation will only continue to worsen."
U.S. President Joe Biden met with several federal agencies Wednesday to discuss ways to battle against persistent ransomware attacks. Biden reportedly discussed mitigation strategies with leaders at the departments of State, Justice and Homeland Security and members of the intelligence community, Reuters reports.
News Editor Doug Olenick and Executive Editor Jeremy Kirk contributed to this story.