IBM Faces Heat Over Aussie Census StumblePrime Minister Malcolm Turnbull 'Bitterly Disappointed'
Australia's embarrassing technical problems with its online census form this week have put pressure on IBM, the main contractor for a project that was aimed at saving tens of millions of dollars by moving away from paper forms.
The census stumble has raised questions about whether Big Blue will face repercussions over the shutdown of the census web service on the evening of Aug. 9, when the Australian Bureau of Statistics hoped to gain a once-in-five years snapshot of the nation's people.
The ABS voluntarily took the form offline after the site suffered four distributed denial-of-service attacks - believed to have originated overseas - over fears that personal data might have been targeted. But the Australian Signals Directorate has since confirmed that personal data was not compromised during the disruptions.
The online census form was restored on the afternoon of Aug. 11. But the incident only compounded troubles for the ABS, which had faced weeks of criticism over the extended period of time that it's chosen to retain personal data, as well as complaints over mandatory names and address submission (see Australia in Privacy Furor Over Census).
Prime Minster Malcolm Turnbull has offered the most clear information to date about the site's disruption, saying that measures were not put in place to prevent DDoS attacks, and that the problems were compounded by hardware failures and poor resiliency planning.
"There are clearly very big issues, very big issues for IBM, the system provider for the census, and for the Australian Bureau of Statistics itself," Turnbull says in a radio interview on Aug. 11. "I am bitterly disappointed."
"We genuinely regret the inconvenience that has occurred," IBM says in a statement. But the company has so far declined to answer specific questions about the failures.
"We are committed to our role in the delivery of this project," IBM says. "Our cybersecurity experts are partnering with national intelligence agencies to ensure the ongoing integrity of the site."
IBM has been a longtime partner of the ABS. In 2014, it was awarded a $9.6 million contract for the design and development of the online census project that launched this year.
The cause of the census site's failure continues to be investigated. Many have speculated that the ABS may have not fully anticipated the capacity it needed to handle the extremely high traffic levels that would hit the site, or the risk posed to the site by potential DDoS attacks.
DDoS attacks are aimed at directing data traffic toward a web service with the aim of overloading it. Large companies and organizations spend much money trying to mitigate DDoS, which is one of the most common types of cyberattacks.
No group has claimed responsibility for any such attack, although ABS head David Kalisch has described the attacks as malicious and originating overseas. That's a bit odd, as DDoS attackers often cry for attention afterwards.
But observers have raised the possibility that the ABS might be conflating a DDoS attack with an influx of real users having generated large amounts of traffic. Indeed, it's sometimes difficult to differentiate between a DDoS strike and high site usage.
The ABS has made a strong push this year for census-takers to go online - a move that was expected to save the agency up to AU$100 million (US$76 million). But by encouraging millions of Australian to access the online form on a single evening, the ABS may have created the denial-of-service situation itself.
Potential Culprit: Geoblocking
Another potential culprit is geoblocking. The ABS didn't want people from outside Australia using the census form. The agency blocked access to the census web servers and its Domain Name System servers from computers that weren't using Australian ISPs. The DNS is the internet addressing system that translates domain names into IP addresses that can be called into a browser.
If a user's DNS resolver - which answers the question of where an IP address for a domain name can be found - was located outside Australia, the traffic was dropped by the ABS's firewall. The problem is that many people in Australia use overseas DNS resolvers, such as Google's Public DNS service, OpenDNS, or one that is part of their employer's international internet connections.
If a census respondent using this arrangement tried to access the online form, the foreign DNS resolver would keep trying to get an answer from the ABS's servers, a type of repeated request that could be mistaken for an attack. And the person wouldn't be able to reach the census form.
Other errors could have also contributed to the outage. If a web application isn't coded correctly, even just a few malicious packets of data can cause it to stop responding, says Mike Holm, operations manager at AusCERT, a not-for-profit computer emergency response team based at the University of Queensland.
An application's failure "is almost a foregone conclusion if you've already got a high load on something then the right kind of denial-of-service came on," Holm says.
Sydney-based Revolution IT held several contacts with the ABS for subjecting the census site to load testing, which is a process by which a site or web service gets stress-tested to ensure it can handle high traffic volumes.
The ABS and IBM anticipated the site receiving up to 250 forms per second, but asked Revolution IT to test the service at 350 forms per second and a peak of 400 forms. Hamish Leighton, director of Revolution IT, says the site performed well in testing. When the site was voluntarily taken offline, it was only receiving around 150 forms per second.
Leighton notes that Revolution IT only load-tested the site, and didn't have a remit to do any security testing,
Steve Ingram, PwC's cyber lead for Asia-Pacific and the Americas, says it doesn't appear the ABS's contracts with suppliers covered areas such as failover plans, crisis management, emergency response and redundancy. It's possible the ABS tried to shoehorn what was a very large project into a slim budget.
"I don't see any evidence of an attack here but an inadequate system that wasn't prepared for it," Ingram says.
Legal Action Could Follow
If the government or the ABS decides to take legal action over the census site, IBM could find itself facing another large public sector headache.
In 2013, the state of Queensland sued the company for allegedly failing to deliver a payroll system on budget and on time for the state's health department. In April, the case was dismissed, with Queensland ordered to pay the company's costs, according to the ABC. By then however, the case had dragged on for three years.
Ingram says it's hard to say if IBM will end up in court again. For example, the vendor may have simply delivered what the ABS requested, and its specification may have proved to be inadequate.
IT vendors often have service-level agreements with clients, but those documents are very carefully worded to cover aspects of a service that a vendor can feasibly control, says Kay Lam-MacLeod, a technology lawyer with Idealaw, which is located in Brisbane.
Although the ABS contract with IBM has not been made public, Lam-MacLeod says she would expect that an agreement for the census site would have contained some guarantee of a defense against a DDoS attack, up to a certain level.
"It's a lot of smoke and mirrors going on out there," Lam-MacLeod says. "It's difficult to say if [the] site was vastly under-resourced or not hardened enough."