Companies scramble to defend against newly discovered 'Log4j' digital flaw

Late last week, the staff of the popular world-building video game Minecraft published an unusual blog post announcing that a version of the game had a digital flaw that hackers could exploit to take over players' computers. The gaming company released a patch and encouraged players who run their own servers to do the same.

But the cybersecurity community quickly realized that the vulnerability, embedded in an incredibly popular and common software tool, could potentially impact billions of devices.

Over the weekend, the Department of Homeland Security's Cybersecurity and Infrastructure Security Agency (CISA) released a statement on what has become known as the "Log4j" vulnerability, or "Log4shell." The agency discussed efforts to help private-sector partners fix the problem and urged all companies to upgrade their software.

"To be clear, this vulnerability poses a severe risk," CISA Director Jen Easterly said in the statement. "We will only minimize potential impacts through collaborative efforts between government and the private sector. We urge all organizations to join us in this essential effort and take action."

The flaw was found in a commonly used bit of software

A researcher working for Chinese tech firm Alibaba discovered the bug and privately informed the Apache Software Foundation, an all-volunteer corporation that develops and maintains open-source software. It spilled into public view when Minecraft made its disclosure and the researcher posted about it online.

When programmers write code, they often rely on some extremely common and freely available bits of software — like using building blocks — to do common tasks. In this case, the vulnerable piece of software was something called Log4j, which is used in the programming language Java and essentially creates a log of activity on a device, copying down everything that happens as programs run.

"You want to think about it like a modular component that's used in many, many different kinds of software. And its job is ... just basically recording things that happened and writing them to another computer somewhere else," said Andrew Morris, founder and CEO of cyber-intelligence firm GreyNoise.

But the researcher discovered that a hacker could send a message to this logger from anywhere in the world through the internet, giving it commands. That would give the bad actor full access to take over the device.

Hackers can easily seize control

The vulnerability is particularly dangerous, cybersecurity experts say, because it impacts such a wide range of programs — nearly everything written in Java or that relies on software written in Java, ranging from products made by Amazon to Apple. Security researchers have been keeping running lists of potentially vulnerable companies and programs, including which have released patches.

The flaw also is relatively easy to exploit. "It's really not that complicated," Morris said. And when cybersecurity researchers release a proof of concept, confirming it's possible to exploit the vulnerability and explaining how to do it, bad actors can use it like a blueprint. "It's kind of like you build the machine one time, and then everybody else can use the same machine to exploit the device as you want," Morris noted.

As a result, cybersecurity experts spent the past weekend working around the clock, and that's likely to continue for days if not weeks.

"The internet's on fire," said David "Moose" Wolpoff, chief technology officer at cybersecurity firm Randori, referring to the acute stress within the cybersecurity community. "The reality is that everybody that I know professionally just worked a very long weekend and is going to continue working through the coming weeks in what is essentially a race with the hackers."

Criminals are already launching attacks using Log4j

Cybersecurity researchers are scanning the internet the same way cybercriminals are — determining which devices might be vulnerable in hopes of defending them before hackers can infect entire networks or launch more-destructive attacks.

Companies are already seeing hackers exploit the flaw, including crypto-miners hijacking computing power to mine digital currency, cybercriminals auctioning off access to networks they've penetrated and armies of zombie digital devices called botnets targeting vulnerable machines to join their ranks.

Even if hackers do break through the "open door" left by this vulnerability, companies can limit the damage by deploying multiple layers of security to prevent criminals from burrowing into networks beyond individual compromised devices, according to Katie Nickels, director of threat intelligence at cybersecurity firm Red Canary.

"Once an adversary gets on to some machine, they want to do other things. ... They want to mine for cryptocurrency, or they want to steal your information, or they want to move to other networks if they're in a big enterprise, so they can ransom sensitive files," Nickels said. "And that's why I think a lot of people lose sight of the importance of not just trying to detect adversaries as they get in or stop them from getting in, but having what we call in security 'defense in depth.' Maybe I have locks, but then I also have a security system."

Experts say the current chaos should spark conversation about how to better prepare to defend against similar attacks in the future — beyond scrambling to patch a hole.

If companies don't even know they're reliant on the vulnerable Java library, for example, they won't be able to fix the problem.

That's why the White House is now requiring companies that sell software to the government to include what's called a software bill of materials, like a "recipe" of code, Nickels said. Even so, she noted that some companies also might not know all the layers of software that are baked into the off-the-shelf software they use: "We rely on so many cloud services, so many different software components. Who should we even be asking?"

Figuring out the full number of companies that use software like Log4j, let alone many other common software tools, will be a massive undertaking, Nickels said.

But cybersecurity experts also emphasized the importance of open-source software such as Log4j, which was created, was developed and is maintained by a volunteer who isn't getting paid for that work.

"I cannot stress enough to you how dire and severe the situation is as it relates to the amount of technical dependencies that fall onto software products that are open-source, that are run by a handful of people," said Morris of GreyNoise. "Sometimes one person in their spare time as they're juggling other stuff, working other jobs.

"It's really important that we think about how we support the people that write the software that keeps our world moving forward."