SolarWinds Describes Attackers' 'Malicious Code Injection'Software Vendor's Infrastructure Penetrated by September 2019
Investigators probing the supply chain attack that hit SolarWinds say attackers successfully hacked the company's Microsoft Visual Studio development tools to add a backdoor into software builds.
The backdoor, dubbed "Sunburst," was added to the company's Orion network monitoring software beginning in March 2020. Up to 18,000 customers installed and ran the Trojanized software. Attackers then used the backdoor to target a subset of customers, perhaps numbering in the hundreds, for second-stage attacks, which could have led to data exfiltration, eavesdropping - including email inbox access - and follow-on attacks against business partners.
On Monday, Austin, Texas-based SolarWinds released an update on its attack investigation, reporting that investigators have successfully reverse-engineered code that attackers injected into its software development tools.
"We believe we have found a highly sophisticated and novel malicious code injection source the perpetrators used to insert the Sunburst malicious code into builds of our Orion platform software," SolarWinds CEO Sudhakar Ramakrishna writes in a blog post. "The software development and build process used by SolarWinds is common throughout the software industry, so we believe that sharing this information openly will help the industry guard against similar attacks in the future and create safer environments for customers."
Many cybersecurity experts and government officials, including the U.S. intelligence establishment, have attributed the campaign to a Russian advanced persistent threat group. Some more specifically attribute the hack to Russia's foreign international service, the SVR.
On Monday, security firm Kaspersky noted that code in the Sunburst backdoor overlapped with code previously seen in the "Kazuar" backdoor, which security researchers have tied to Russian attackers. Code overlap or reuse, however, is not an attribution smoking gun, because, for example, the code could have been used as a false flag.
But experts have said that the SolarWinds supply chain attack has all of the hallmarks of an espionage operation.
Confirmed victims of second-stage attacks include FireEye, Microsoft and up to 10 U.S. government agencies, including the Department of Justice and branches of the Pentagon, as well as the Commerce, Homeland Security, State, Energy and Treasury departments.
Credit for discovering the attack campaign goes to cybersecurity firm FireEye. After the company suffered a breach and the theft of penetration testing tools, its investigators unearthed the wider attack campaign, alerted SolarWinds on Dec. 12, 2020, and issued a public alert the next day.
One ongoing question has been: How did attackers sneak their backdoor into SolarWinds' code?
To investigate the root cause of the intrusion after the breach came to light, SolarWinds brought in KPMG and CrowdStrike, which say they have now determined the malicious code got added via tampering with SolarWinds' software development builds.
SolarWinds says digital forensic investigators have determined that the attackers appear to have first penetrated its network by September 2019. It appears that, the following month, they altered Orion code builds to test their ability to inject malicious code into the software builds, the company says.
Beginning on Feb. 20, 2020, attackers inserted "Sunspot" - an implant - into the SolarWinds software build process. It was designed to install Sunburst into Orion software builds.
"The perpetrators remained undetected and removed the Sunburst malicious code from our environment in June 2020," SolarWinds says.
Why attackers might have removed Sunburst from fresh builds of Orion is not clear. SolarWinds says that while it regularly fixed vulnerabilities and code flaws in its software, the malicious code injection remained undetected until FireEye's alert led the company to review its entire code base and software development processes.
CrowdStrike, which has not attributed the attack to any group - instead referring to the attackers by the codename "StellarParticle" - says the operation was extremely sophisticated. It has released updated tools, techniques and procedures - or TTPs - that organizations can use to spot signs that they were hit by Sunburst, both currently as well as by reviewing logs to study historical activity.
"The design of Sunspot suggests StellarParticle developers invested a lot of effort to ensure the code was properly inserted and remained undetected and prioritized operational security to avoid revealing their presence in the build environment to SolarWinds developers," CrowdStrike's intelligence team reports in a Monday blog post.
Sunspot Target: Development Tools
CrowdStrike says Sunspot monitored SolarWinds' Microsoft Visual Studio development tools - in particular, the MsBuild.exe component - and only added the Sunburst backdoor code when the tools were being used to compile Orion code.
"When Sunspot finds an MsBuild.exe process, it will spawn a new thread to determine if the Orion software is being built and, if so, hijack the build operation to inject Sunburst," CrowdStrike says. "The monitoring loop executes every second, allowing Sunspot to modify the target source code before it has been read by the compiler."
Attackers also appear to have edited software build warnings so that their code wouldn't trigger any errors and reveal itself to developers, and to have verified the code to be injected, using MD5 hashes, to ensure that only working backdoor code would be injected, thus avoiding inadvertent software-build errors, CrowdStrike says.
The attackers' sophistication continued beyond sneaking the backdoor into the code; they were able to communicate with infected systems without being detected.
"Analysis suggests that by managing the intrusion through multiple servers based in the United States and mimicking legitimate network traffic, the attackers were able to circumvent threat detection techniques employed by ... SolarWinds, other private companies and the federal government," SolarWinds' Ramakrishna writes in his blog post.
SolarWinds Pledges Ongoing Transparency
SolarWinds' investigation into the breach is continuing. The company says it has been freely sharing all affected, proprietary code libraries with security researchers and continues to work with law enforcement and intelligence agencies investigating the intrusion.
Ramakrishna has pledged to release further details from SolarWinds' investigation as they become available.
"Our concern is that, right now, similar processes may exist in software development environments at other companies throughout the world," he says. "The severity and complexity of this attack has taught us that more effectively combatting similar attacks in the future will require an industrywide approach as well as public-private partnerships that leverage the skills, insight, knowledge and resources of all constituents."
SolarWinds has been hit by shareholder lawsuits over the breach, alleging that the company lacked sufficiently strong security defenses.
Shares of SolarWinds, which trade on the New York Stock Exchange, were valued at $23.55 per share on Dec. 11, just before the supply chain attack against the company was discovered and publicly disclosed, leading to a dive in its stock price. By the end of trading on Monday, SolarWinds stock was trading at $15.
Story updated on Jan. 13 to clarify that CrowdStrike is not a customer of SolarWinds and never used its Orion software.