A 'nightmare scenario': Data-tampering attacks are hard to detect, with devastating consequences

Attacks involving manipulation of data could pose an even more severe threat than data theft or ransomware in some cases, but are not top of mind for most businesses, experts told Protocol.

A 'nightmare scenario': Data-tampering attacks are hard to detect, with devastating consequences

"If you’re not looking for the threat, you pretty much fall for it every time," one cyber security expert said of data manipulation.

Illustration: Christopher T. Fong/Protocol

Imagine a cybersecurity catastrophe like this one: A pharmaceuticals maker suffers a data breach, but no data is stolen and no ransomware is deployed. Instead the attacker simply makes a change to some of the data in a clinical trial — ultimately leading the company to release the wrong drug.

It's a hypothetical scenario, for now. Ransomware and the theft of sensitive data remain massive top-of-mind security concerns, of course, but at least there are tools and procedures available to mitigate those issues.

Data-tampering represents a different type of threat, and one that could be potentially even more serious for certain organizations, depending on the situation. And yet it's not on the radar for many businesses, experts told Protocol, due to the fact that few such attacks have occurred and come to light.

But this type of attack is not totally unprecedented. In early 2021, for instance, a hacker who broke into a Florida water treatment plant was able to elevate the sodium hydroxide, or lye, in the water to an unsafe level. (The modification was quickly caught by an operator.)

Will Ackerly, a former NSA security architect who invented a data-protection standard used by U.S. defense and intelligence agencies, is among those who believe that data manipulation is poised to become a burgeoning threat in coming years.

Compared with other threats to data security, the manipulation of data is probably the "most nefarious and hardest to detect," said Ackerly, who is now co-founder and CTO of data security startup Virtru. And on the attacker side of the equation, the fact remains that today, "there are a lot of adversaries looking to trick someone into thinking something that's not true," he said.

Another example is the growing use of deepfake audio and video in cyberattacks. A recent VMware study found that two-thirds of cyber incident responders investigated attacks that involved fabricated audio or video over the past year, up 13% from a year ago.

But as jarring as it is, the deepfake phenomenon is just one part of the larger threat that businesses are facing from manipulated data, experts told Protocol.

‘Nightmare scenario’

Lou Steinberg, who was the CTO of TD Ameritrade from 2011 to 2017, said he's spoken with numerous CISOs in industries from financial services to pharmaceuticals who are increasingly worried by the threat of data manipulation attacks, sometimes referred to as attacks on "data integrity."

In another example of this type of attack, a threat actor might corrupt a portion of a public company's data and then publicize this fact, leaving it unable to close its books at the end of the quarter, said Steinberg, who is now the founder of cybersecurity research lab CTM Insights.

"What happens when you can't trust your own data?" he said. "This is a nightmare scenario."

Such attacks have been warned about for years . And the fact that few have made headlines suggests they could be harder to pull off than it might seem.

But the fact remains that both the technology and the awareness needed to combat data manipulation threats are not where they need to be, experts said.

Technologies for protecting against data integrity attacks include file integrity monitoring services for detecting file changes, which can be used in combination with logging and backups to secure against such threats from external attackers or malicious insiders, the National Institute of Standards and Technology noted in a 2020 report .

But such an approach won't necessarily detect data changes by someone who appears to be an authorized user, because they're using stolen credentials, Steinberg said, or because they're a malicious insider.

"What happens when you can't trust your own data? This is a nightmare scenario."

The second issue is whether the speed at which modern data is collected and overwritten would actually make it practical to recover the untainted version of the data in question, he said. For files that change constantly, "a rollback can create more damage than the attack," Steinberg said.

Most businesses are also preoccupied with other data security issues, such as protecting the confidentiality of their data, said Heidi Shey, a principal analyst at Forrester.

"I think something like data integrity protection is so much further down the list for many people," Shey said. "There's a lot of other priorities that just are louder, and demand more of their attention."

Still, "I'd say it's a topic that is worth companies taking a closer look at," she said. While data manipulation may only constitute a "simmering" threat at this point, "we know that the potential consequences could be pretty major for this type of attack," Shey said.

Believable fakes

The threat isn't limited to changes in data values either: Thanks to the same AI-powered technology that's used to create deepfake videos, researchers say the threat of manipulated images, such as medical scans, is growing as well.

Image fakery is of course nothing new, and in recent years, a number of military disinformation efforts have embraced the tactic. But the strategic insertion of an altered image in place of the original could be much harder to spot.

A study published in 2019 by researchers at Ben-Gurion University found that CT scans, which they manipulated with the help of AI, were consistently able to trick radiologists into misdiagnosing lung conditions.

"If you’re not looking for the threat, you pretty much fall for it every time."

"If you’re not looking for the threat, you pretty much fall for it every time," said Yisroel Mirsky, who led the study and is head of the university's Offensive AI Research Lab . The experiments also found that even after the radiologists were told that some images had been faked, they were still fooled 60% of the time.

The research was intended to illustrate a larger threat — that "an attacker may perform this act in order to stop a political candidate, sabotage research, commit insurance fraud, perform an act of terrorism, or even commit murder," the researchers wrote in their paper on the study.

Notably, deepfake image generation technology has advanced significantly since the study was conducted, Mirsky told Protocol. "Every few months it's getting better — higher resolution, higher fidelity," he said.

Attacks on machine learning

One type of data manipulation attack that has received comparatively more attention is what's known as "adversarial machine learning," in which an attacker attempts to dupe an ML model with false data during its training phase.

While the motives for doing this can vary, the result is that the ML model won't perform properly. The case of Microsoft's short-lived Twitter chatbot, Tay , is one infamous example of adversarial ML — but there are many documented cases of successful data-poisoning attacks on ML models, both by threat actors and researchers.

Those types of attacks usually don’t result in an actual data breach, however. The attackers have instead managed to influence the ML models from the outside. But that doesn't mean that the data stores that inform key ML models don't represent a ripe target for a motivated hacker, said Lisa O'Connor, managing director for Accenture Security and head of security R&D at Accenture Labs.

And given the world's growing reliance on algorithms, adversarial ML threats are a serious concern, O'Connor said. "The stakes are very high for protecting that ecosystem," she said, pointing to efforts such as the MITRE ATLAS initiative that aim to protect against threats to ML models.

The bottom line is that — regardless of the data source in question — it's clear in today's digital threat landscape that "seeing is not believing anymore," said Carey O'Connor Kolaja, CEO at identity verification vendor AU10TIX.

"There's been a shift in how our society is making decisions and the type of information we're making decisions on — whether it's an enterprise or the government or an individual," she said. "And that information can easily be manipulated."


Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more .

Keep Reading Show less
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison ( @dgoodison ) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich ) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.


Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories