A 'nightmare scenario': Data-tampering attacks are hard to detect, with devastating consequences

Attacks involving manipulation of data could pose an even more severe threat than data theft or ransomware in some cases, but are not top of mind for most businesses, experts told Protocol.

A 'nightmare scenario': Data-tampering attacks are hard to detect, with devastating consequences

"If you’re not looking for the threat, you pretty much fall for it every time," one cyber security expert said of data manipulation.

Illustration: Christopher T. Fong/Protocol

Imagine a cybersecurity catastrophe like this one: A pharmaceuticals maker suffers a data breach, but no data is stolen and no ransomware is deployed. Instead the attacker simply makes a change to some of the data in a clinical trial — ultimately leading the company to release the wrong drug.

It's a hypothetical scenario, for now. Ransomware and the theft of sensitive data remain massive top-of-mind security concerns, of course, but at least there are tools and procedures available to mitigate those issues.

Data-tampering represents a different type of threat, and one that could be potentially even more serious for certain organizations, depending on the situation. And yet it's not on the radar for many businesses, experts told Protocol, due to the fact that few such attacks have occurred and come to light.

But this type of attack is not totally unprecedented. In early 2021, for instance, a hacker who broke into a Florida water treatment plant was able to elevate the sodium hydroxide, or lye, in the water to an unsafe level. (The modification was quickly caught by an operator.)

Will Ackerly, a former NSA security architect who invented a data-protection standard used by U.S. defense and intelligence agencies, is among those who believe that data manipulation is poised to become a burgeoning threat in coming years.

Compared with other threats to data security, the manipulation of data is probably the "most nefarious and hardest to detect," said Ackerly, who is now co-founder and CTO of data security startup Virtru. And on the attacker side of the equation, the fact remains that today, "there are a lot of adversaries looking to trick someone into thinking something that's not true," he said.

Another example is the growing use of deepfake audio and video in cyberattacks. A recent VMware study found that two-thirds of cyber incident responders investigated attacks that involved fabricated audio or video over the past year, up 13% from a year ago.

But as jarring as it is, the deepfake phenomenon is just one part of the larger threat that businesses are facing from manipulated data, experts told Protocol.

‘Nightmare scenario’

Lou Steinberg, who was the CTO of TD Ameritrade from 2011 to 2017, said he's spoken with numerous CISOs in industries from financial services to pharmaceuticals who are increasingly worried by the threat of data manipulation attacks, sometimes referred to as attacks on "data integrity."

In another example of this type of attack, a threat actor might corrupt a portion of a public company's data and then publicize this fact, leaving it unable to close its books at the end of the quarter, said Steinberg, who is now the founder of cybersecurity research lab CTM Insights.

"What happens when you can't trust your own data?" he said. "This is a nightmare scenario."

Such attacks have been warned about for years. And the fact that few have made headlines suggests they could be harder to pull off than it might seem.

But the fact remains that both the technology and the awareness needed to combat data manipulation threats are not where they need to be, experts said.

Technologies for protecting against data integrity attacks include file integrity monitoring services for detecting file changes, which can be used in combination with logging and backups to secure against such threats from external attackers or malicious insiders, the National Institute of Standards and Technology noted in a 2020 report.

But such an approach won't necessarily detect data changes by someone who appears to be an authorized user, because they're using stolen credentials, Steinberg said, or because they're a malicious insider.

"What happens when you can't trust your own data? This is a nightmare scenario."

The second issue is whether the speed at which modern data is collected and overwritten would actually make it practical to recover the untainted version of the data in question, he said. For files that change constantly, "a rollback can create more damage than the attack," Steinberg said.

Most businesses are also preoccupied with other data security issues, such as protecting the confidentiality of their data, said Heidi Shey, a principal analyst at Forrester.

"I think something like data integrity protection is so much further down the list for many people," Shey said. "There's a lot of other priorities that just are louder, and demand more of their attention."

Still, "I'd say it's a topic that is worth companies taking a closer look at," she said. While data manipulation may only constitute a "simmering" threat at this point, "we know that the potential consequences could be pretty major for this type of attack," Shey said.

Believable fakes

The threat isn't limited to changes in data values either: Thanks to the same AI-powered technology that's used to create deepfake videos, researchers say the threat of manipulated images, such as medical scans, is growing as well.

Image fakery is of course nothing new, and in recent years, a number of military disinformation efforts have embraced the tactic. But the strategic insertion of an altered image in place of the original could be much harder to spot.

A study published in 2019 by researchers at Ben-Gurion University found that CT scans, which they manipulated with the help of AI, were consistently able to trick radiologists into misdiagnosing lung conditions.

"If you’re not looking for the threat, you pretty much fall for it every time."

"If you’re not looking for the threat, you pretty much fall for it every time," said Yisroel Mirsky, who led the study and is head of the university's Offensive AI Research Lab. The experiments also found that even after the radiologists were told that some images had been faked, they were still fooled 60% of the time.

The research was intended to illustrate a larger threat — that "an attacker may perform this act in order to stop a political candidate, sabotage research, commit insurance fraud, perform an act of terrorism, or even commit murder," the researchers wrote in their paper on the study.

Notably, deepfake image generation technology has advanced significantly since the study was conducted, Mirsky told Protocol. "Every few months it's getting better — higher resolution, higher fidelity," he said.

Attacks on machine learning

One type of data manipulation attack that has received comparatively more attention is what's known as "adversarial machine learning," in which an attacker attempts to dupe an ML model with false data during its training phase.

While the motives for doing this can vary, the result is that the ML model won't perform properly. The case of Microsoft's short-lived Twitter chatbot, Tay, is one infamous example of adversarial ML — but there are many documented cases of successful data-poisoning attacks on ML models, both by threat actors and researchers.

Those types of attacks usually don’t result in an actual data breach, however. The attackers have instead managed to influence the ML models from the outside. But that doesn't mean that the data stores that inform key ML models don't represent a ripe target for a motivated hacker, said Lisa O'Connor, managing director for Accenture Security and head of security R&D at Accenture Labs.

And given the world's growing reliance on algorithms, adversarial ML threats are a serious concern, O'Connor said. "The stakes are very high for protecting that ecosystem," she said, pointing to efforts such as the MITRE ATLAS initiative that aim to protect against threats to ML models.

The bottom line is that — regardless of the data source in question — it's clear in today's digital threat landscape that "seeing is not believing anymore," said Carey O'Connor Kolaja, CEO at identity verification vendor AU10TIX.

"There's been a shift in how our society is making decisions and the type of information we're making decisions on — whether it's an enterprise or the government or an individual," she said. "And that information can easily be manipulated."


Steel decided World War II. Chips will decide whatever is next.

“Chip War: The Fight for the World’s Most Critical Technology” foreshadows the coming battle between nations over semiconductors.

“Chip War” outlines the nature of the coming battle over semiconductors, showing how the power to produce leading-edge chips fell into the hands of just five companies.

Image: Scribner; Protocol

“World War II was decided by steel and aluminum, and followed shortly thereafter by the Cold War, which was defined by atomic weapons,” Chris Miller, a professor at Tufts University’s Fletcher School of Law and Diplomacy, writes in the introduction to his latest book. So what’s next? According to Miller, the next era, including the rivalry between the U.S. and China, is all about computing power.

That tech rivalry and the story of how the chip industry got from four to 11.8 billion transistors are all part of Miller’s book, “Chip War: The Fight for the World’s Most Critical Technology,” which comes out Oct. 4. “Chip War” outlines the nature of the coming battle over semiconductors, showing how the power to produce leading-edge chips fell into the hands of just five companies: three from the U.S., one from Japan, and one from the Netherlands.

Keep Reading Show less
Hirsh Chitkara

Hirsh Chitkara ( @HirshChitkara) is a reporter at Protocol focused on the intersection of politics, technology and society. Before joining Protocol, he helped write a daily newsletter at Insider that covered all things Big Tech. He's based in New York and can be reached at hchitkara@protocol.com.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.

Musk’s texts reveal what tech’s most powerful people really want

From Jack Dorsey to Joe Rogan, Musk’s texts are chock-full of überpowerful people, bending a knee to Twitter’s once and (still maybe?) future king.

“Maybe Oprah would be interested in joining the Twitter board if my bid succeeds,” one text reads.

Photo illustration: Patrick Pleul/picture alliance via Getty Images; Protocol

Elon Musk’s text inbox is a rarefied space. It’s a place where tech’s wealthiest casually commit to spending billions of dollars with little more than a thumbs-up emoji and trade tips on how to rewrite the rules for how hundreds of millions of people around the world communicate.

Now, Musk’s ongoing legal battle with Twitter is giving the rest of us a fleeting glimpse into that world. The collection of Musk’s private texts that was made public this week is chock-full of tech power brokers. While the messages are meant to reveal something about Musk’s motivations — and they do — they also say a lot about how things get done and deals get made among some of the most powerful people in the world.

Keep Reading Show less
Issie Lapowsky

Issie Lapowsky ( @issielapowsky) is Protocol's chief correspondent, covering the intersection of technology, politics, and national affairs. She also oversees Protocol's fellowship program. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University's Center for Publishing on how tech giants have affected publishing.


Circle’s CEO: This is not the time to ‘go crazy’

Jeremy Allaire is leading the stablecoin powerhouse in a time of heightened regulation.

“It’s a complex environment. So every CEO and every board has to be a little bit cautious, because there’s a lot of uncertainty,” Circle CEO Jeremy Allaire told Protocol at Converge22.

Photo: Circle

Sitting solo on a San Francisco stage, Circle CEO Jeremy Allaire asked tennis superstar Serena Williams what it’s like to face “unrelenting skepticism.”

“What do you do when someone says you can’t do this?” Allaire asked the athlete turned VC, who was beaming into Circle’s Converge22 convention by video.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.


Is Salesforce still a growth company? Investors are skeptical

Salesforce is betting that customer data platform Genie and new Slack features can push the company to $50 billion in revenue by 2026. But investors are skeptical about the company’s ability to deliver.

Photo: Marlena Sloss/Bloomberg via Getty Images

Salesforce has long been enterprise tech’s golden child. The company said everything customers wanted to hear and did everything investors wanted to see: It produced robust, consistent growth from groundbreaking products combined with an aggressive M&A strategy and a cherished culture, all operating under the helm of a bombastic, but respected, CEO and team of well-coiffed executives.

Dreamforce is the embodiment of that success. Every year, alongside frustrating San Francisco residents, the over-the-top celebration serves as a battle cry to the enterprise software industry, reminding everyone that Marc Benioff’s mighty fiefdom is poised to expand even deeper into your corporate IT stack.

Keep Reading Show less
Joe Williams

Joe Williams is a writer-at-large at Protocol. He previously covered enterprise software for Protocol, Bloomberg and Business Insider. Joe can be reached at JoeWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

Latest Stories