Power

The ransomware dilemma: To pay or not to pay?

Ransomware victims face a "devil's bargain" over paying demands.

A screen showing a ransomware message

Everyone says you shouldn't pay hackers. But it's not always that simple.

Photo: Alexander Ryumin via Getty Images

Ransomware victims often find themselves in what feels like an impossible position: There are endless decisions to be made, but it's a race against the clock to save computer systems and data.

That's what New Orleans city officials learned firsthand in the early morning hours on a Friday in December. At 6 a.m., there was a blip of suspicious activity on the city's cybersecurity alert system — not a definite sign of attack, but an early warning of what would come, the city's chief information officer Kim Walker LaGrue recounted in an interview with Protocol. At about 9 a.m., when many city employees started arriving to work, there was an uptick in reports of unusual behavior on their computers.

Get what matters in tech, in your inbox every morning. Sign up for Source Code.

By 11 a.m., the diagnosis was clear: Ransomware was spreading through the network. The malicious code was encrypting devices and information, rendering them inaccessible unless city officials decided to contact the attackers. Presumably, they would then get instructions about paying a ransom.

City officials had a game plan in place to guide them through some of the toughest decisions ransomware victims face — the biggest being whether or not to pay the demand. The Louisiana state government and three of its school districts were hit by two separate ransomware attacks in 2019, so New Orleans officials had prepared and rehearsed an incident response plan.

The first step LaGrue took — before she even consulted with the mayor — was to disconnect all city systems from the internet. More than 3,000 computers and 400 servers were shut down to prevent the malicious code from spreading, she said.

"We sounded every type of alarm we could," LaGrue said. The city declared a state of emergency within about 30 minutes of identifying the attack, she added. Within an hour, the FBI and other state and federal agencies were on the ground to help respond to the attack. Normal lines of communication — including the city's email system — were inaccessible, and a war room was set up to help relay information.

City officials knew early on that they would not engage with the attackers and that no ransom would be paid. The U.S. Conference of Mayors, of which New Orleans Mayor LaToya Cantrell is a member, last year adopted a resolution opposing such payments after cities including Baltimore and Atlanta were hit by ransomware attacks. Additionally, the FBI discourages organizations from paying ransomware demands, arguing that it incentivizes crime, emboldens attackers and is sometimes futile — attackers on occasion don't decrypt the data after a payment is made.

Because it didn't pay the attackers, New Orleans is still recovering; LaGrue estimates that it will take another four months to make a full recovery. So far, her staff has focused on restoring key priorities. The first step was inspecting and cleaning thousands of government computers, and removing ones that were too old to run up-to-date software, she said. Over the last month, the city has focused on restoring data from backups, as well as services including email and printing. Over the last few weeks, the city's IT staff has been particularly focused on assisting the city's infrastructure and public safety agencies in their preparations for Mardi Gras on Tuesday.

The financial impact of the attack is still uncertain. A city spokesperson said that the current estimated cost of the attack is $7.2 million, adding that he "wouldn't be shocked if it got higher." The city has a $3 million cyber insurance policy that will help cover some of the costs, which include replacing between 500 and 800 computers as a result of the incident, LaGrue said. The city is planning to add three cybersecurity specialists to its IT staff, which currently includes 65 workers, she said.

Ransomware has been a rapidly growing problem for all types of organizations, including Fortune 500 firms and small nonprofits. The antimalware firm Malwarebytes said in August that it observed a roughly 365% spike in ransomware detections between 2018 and 2019. Municipalities, educational institutions and health care organizations are prime targets for ransomware attacks due to their lack of spending on security and outdated hardware and software, according to the report.

Because ransomware affects a range of organizations, the decision to pay can't be one-size-fits-all, according to Chris Hallenbeck, CISO for the Americas at cybersecurity firm Tanium.

"In an ideal world, no one pays the ransom," he said. "It's feeding the beast and creating an ecosystem that encourages criminals. But the reality of many organizations is that every day they're down there's massive impacts to the services they provide."

Overall, about a third of companies pay the ransom, according to security firm Emisisoft.

Which brings us to Maastricht University in the Netherlands, a recent example of an organization that decided to pay a demand after falling victim to ransomware. On Dec. 30, the university decided to pay 30 bitcoin, or about $305,000, to regain access to its systems and data.

Several factors made the attack particularly disruptive, and increased the university's incentives to pay the demand, according to a forensic report from FOX-IT, the security firm that assisted with the incident. A key factor was that attackers were able to encrypt backups for some of the university's critical systems. That meant that, without obtaining a decryption key from the attackers, the university would have to rebuild its infected systems from scratch — a process that would stall the work of students and researchers for many months. Additionally, unlike New Orleans, Maastricht University did not carry cyber insurance to help cover the costs of lengthy disruptions.

The attacker had access to the university's infrastructure for months and studied it to figure out how to maximize damage. They first gained access through a phishing email on Oct. 15 that instructed someone in the university community to sign an attached document and scan it back to the sender. Upon clicking the attachment, malware was installed that gave the attacker administrative access to the device.

On Nov. 21, while scanning the university's network, the attacker found a server with missing security updates and used it to obtain full access to the organization's infrastructure. On Dec. 23, the attacker deployed the Clop ransomware variant on 267 servers. The timing was particularly challenging for the university: Most faculty, students and support staff were away for the holiday break. Employees from IT, facilities, communications and a range of other departments had to skip part of their holiday to respond to the emergency, according to a spokesperson. One week after the ransomware was installed, the university decided to pay the ransom.

Nick Bos, vice president of the university's executive board, said in the report that the decision to pay the demand was a "devil's bargain": The university would have to live with the ethical issues that come from paying criminals in order to resume operations, research and studies.

At a symposium this month attended by students and faculty, Bos said the risks of losing the data were so great that the university did not attempt to negotiate with the attackers; administrators wanted the process to be as calm as possible, he said.

Bos said he was confident that paying the demand was the right choice. About five weeks after paying the demand, students and faculty were able to access almost all of the university's technologies, as well as attend classes and take exams.

"We felt, in consultation with our management and our supervisory bodies, that we could not make any other responsible choice when considering the interests of our students and staff," Bos said in the report.

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins