If you’ve ever obtained credit in the United States, you’re 44.3% likely to be one of the 143 million hacked Equifax accounts that the company disclosed on September 7.
A data breach in an “exploited web app” gave hackers access to social security numbers, birth dates, drivers license numbers, credit card numbers, and — in a subset of 182,000 cases — credit dispute documents.
It hasn’t come to light yet exactly how the data breach occurred, and it’s only speculation at this point that better aws application monitoring could've prevented the breach. The company says it has hired a third-party forensic investigation team to determine the root cause and help prevent future issues.
According to WIRED,
So Equifax doesn’t even know how the breach occurred (or at least that’s the current public narrative).
Sorry Open Source Software: It’s Not Me, It’s You
An investing report issued by RW Baird & Co on the same day as the data breach announcement hinted that a vulnerability in Apache’s open source Struts Java framework was to blame for the data loss:
“My understanding is the breach was perpetuated via the Apache Struts flaw,” said Jeffrey Meuler, an analyst at RW Baird & Co, in the New York Post.
class ContentTypeHandler extends Interface
ContentTypeHandler() {
this.hasQualifiedName("org.apache.strutsZ.rest.handler", "ContentTypeHandler")
}
}
class ToObjectDeserializer extends Method {
ToObjectDeserializer() {
this.getDec1aringType().getASupertype*() instanceof ContentTypeHandler and this.getSignature = "toObject(java.io.Reader,java.1ang.0bject)"
}
}Here is the code used to determine a flaw in Struts that is reportedly the root cause of the Equifax data breach.
Apache fired back two days later and denied plausible responsibility, saying that “it is not clear which Struts vulnerability would have been utilized, if any.”
Data Breach Fallout
It’s been a PR nightmare for the credit reporting bureau, to say the least. When three Equifax managers sold $1.8 million worth of stock just three days after the data breach was discovered, the journalistic fallout that it sparked didn’t help, either. (The managers in question claim they didn’t know about the hack.)
Whatever the cause, it will eventually come to light since U.S. regulatory bodies are likely to investigate the issue due to the prolific nature of the breach. Equifax already sent letters to each U.S. state attorney general and also to state and federal regulators because they know that with the 820 million consumers accounts and 91 million business accounts they house worldwide, the company is likely to take much flak for letting a huge proportion of its data slip into the hands of hackers.
Better DevOps Processes = Better Outcomes
Clearly, DevOps comes into play in preventing a data breach from occurring. (We previously wrote about the importance of integrating QA & security into the DevOps cycle — read it here.)
We know from indexed search results on Google a little about Equifax’s DevOps tech stack. They’ve been prominently featured on Chef’s blog and training site as an exemplary customer and implementation of the configuration management suite. (Chef has since removed all of its blog posts referencing Equifax, for obvious reasons.) We know that Equifax has been using Chef since April of 2014.
Equifax also uses Cloudera to analyze and data mine its vast hordes of customer data and “perform the low-latency data exploration work [they] want to do.” For load balancing, they run on AWS using Amazon’s Elastic Load Balancer.
But automated configuration management from Chef, data insights from Cloudera, and load balancing from Amazon don’t prevent data breaches. (In fact, with improper provisioning, these tools can, in fact, open up privileged access to customer data.) This is especially true when their infrastructure is so dated. According to one Twitter user, “it’s like stepping back in time a decade.”
It’s log management and server monitoring that help security professionals prevent the kind of data breaches that Equifax has experienced in the past. Let’s look at a few examples.
2016 Equifax Cross-site Scripting Vulnerability: Access Times
In 2016, a security researcher reported that Equifax’s main website was vulnerable to cross-site scripting (also called XSS), an exploit that is oftentimes easily patched.
According to Forbes, “Such XSS bugs allow attackers to send specially-crafted links to Equifax customers and, if the target clicks through and is logged into the site, their username and password can be revealed to the hacker.”
Identifying this type of fraudulent activity via logging and monitoring is actually relatively straightforward if you’ve got the right tooling. Monitoring login times and establishing a baseline for each user is one way to find anomalous logins. Granted, it doesn’t catch all cases, such as when a hacker accesses your account at the same time you do.
Still, there are entire groups of people that are consistent in their access profile, and this type of anomaly detection would serve them well. Along the same vein, a log alert that notifies on anomalous login times (especially when a server experiences an increasing number of logins during a scripted hacking attempt) can notify administrators to lock down accounts before breaches become more widespread.
2016–2017 Equifax TALX Hack: IP Address Anomalies
Equifax also experienced a data breach late last year and earlier this year with its TALX employment verification software, affecting an “unknown” number of employees of Erickson Living, a retirement community management company. (“Unknown” because Equifax could not “confirm forensically exactly which accounts were, in fact, accessed without authorization”).
Hackers used a simple, decades-old approach: exploit matched social security numbers and birthdates likely purchased on the black market with a login system utilizing those two numbers as the default login. You’re not alone if you’re seeing a pattern here in Equifax’s breaches.
The best way to prevent this type of exploit is to do away with birthdate- and social security-based logins altogether. But if one must use such a legacy login credential system, properly configured log management can help detect anomalies in login activity: namely, IP address of access endpoints.
Even with default logins, where users haven’t logged in before, log alerts can be set up to trigger if logins were detected from countries outside of the United States or in a location far enough away from employee’s addresses on file (after all, Equifax knows where you live). TALX was smartly rebranded as Equifax Workforce Solutions after the data breach, but the lesson remains the same: Log users’ activity and monitor for anomalies.
Log Management and Server Monitoring Can Save the Day
In all, Equifax should have immediately discovered the attack on its data when the breach started occurring. Per their own report, Equifax indicated hackers had access to customer data from mid-May through July 29, when the attack was discovered. That’s a period of approximately 75 days. Then there was a gap of 40 days from discovery to announcement.
No logging or monitoring software — if that’s how Equifax engineers discovered the attack — should take 75 days to report a problem. As Baird Equity wrote in the aforementioned investing report, “one of the most concerning things to us is that the unauthorized access occurred for about 2.5 months before being identified, raising concerns more broadly regarding Equifax’s data security/practices.”
Equifax can set up 10 firewalls — or 100 — and it wouldn’t protect sensitive customer data as long as hackers have access to login credentials for accounts privileged to access the data. Log and server monitoring can help detect data breaches and stop hackers in their tracks.
We hope that Equifax will learn from this incident and improve its DevOps processes to better protect against security breaches going forward. CEO Richard Smith said in the company’s announcement of the 2017 breach:
In order to get there, the company needs to adjust its logging and monitoring tooling. It’s clear that their existing monitoring can’t even detect problems staring it in the face for 2.5 months, how will that same monitoring help them prevent a breach?
What companies like Equifax want is security, uptime, and automated remediation. Instead, monitoring tools give them data, query tools, and reporting. This is not the same! The proliferation of data on the internet has been accompanied by a similar proliferation of data in monitoring tools. The trick for monitoring is to find a tool that predicts and prevents instead of one that reacts and regrets.
Think of it this way: If a bank storing important customer funds were to be robbed, what would better help prevent a loss of assets — a video camera or a security guard? A video camera is reactive, whereas a security guard can proactively help protect assets on the spot. Centralized log management coupled with automatically configured alerts are like that security guard, proactively notifying security personnel of attempted intrusions into customer databanks.
Equifax by the Numbers
Equifax Founded In: 1899 (118 years ago)
Revenue (in 2016): $3.144 billion
# of Equifax Employees: 9,500 (Worldwide)
Equifax Analyzes Data On: 820 million consumers 91 million businesses (Worldwide)
Equifax Houses Data For: 7,100 employers
# of Publicly Announced Equifax Hacks: 6 (in past 5 years)
2017 Security Breach Occurred: Mid-May through July 29
2017 Security Breach Announced: September 7 (Delay of 40 days)
# of Accounts Hacked in 2017 Breach: 143 million
Population of United States: 323.1 million
% of U.S. Population Affected: 44.3 %
1 in 3 hackers: say accessing privileged accounts is the easiest and fastest way to obtain sensitive data*
*According to a 2017 poll of 250 hackers conducted by Thycotic
