Power

DARPA is pouring millions into a new AI defense program. Here are the companies leading the charge

Intel and Georgia Tech are spearheading Pentagon-backed efforts to defend against attacks that could trick tomorrow's self-driving cars, facial recognition and weapons detection software.

A machine learning algorithm being spoofed by an adversarial attack.

Is that a person or a stop sign? Intel and Georgia Tech are spearheading efforts against adversarial attacks that fool machine learning systems into making such mistakes.

Photo: Georgia Institute of Technology

The Pentagon is teaming up with some of the biggest names in tech to combat hacks designed to mess with the automated systems we'll rely on in the near future.

In February, DARPA issued a call for proposals for a new program. Like most DARPA projects, it had a fantastic acronym: Guaranteeing Artificial Intelligence (AI) Robustness against Deception (GARD). It's a multimillion-dollar, four-year initiative that's aiming to create defenses for sensor-based artificial intelligence — think facial recognition programs, voice recognition tools, self-driving cars, weapon-detection software and more.

Today, Protocol can report that DARPA has selected 17 organizations to work on the GARD project, including Johns Hopkins University, Intel, Georgia Tech, MIT, Carnegie Mellon University, SRI International and IBM's Almaden Research Center. Intel will be leading one part of the project with Georgia Tech, focusing on defending against physical adversarial attacks.

Sensors that use AI computer vision algorithms can be fooled by what researchers refer to as adversarial attacks. These are basically any hack to the physical world that tricks a system into seeing something other than what's there.

In the nascent self-driving car industry, there's already concern about how these attacks could manifest. There could be stickers, effectively invisible to the human eye, that render a stop sign unrecognizable. The stickers could trick other cars into clearing the road for your own vehicle, changing a roadside sign to make a vehicle turn into a certain business's parking lot, or, in the future, even more nefarious acts.

"We certainly know from the current world that there are people who want to create mischief, and there are one-off bad actors," Bruce Draper, GARD program manager at DARPA, told Protocol. "We also know that there are larger threats out there, and if you imagine a city that in the future has many, many self-driving cars in it, then if you suddenly cause all those self-driving cars to misbehave, you could really tie up a city. You could have a major impact. An even more frightening scenario would be if the first responder vehicles were self-driving and you could disable the first responders."

The project is split among three groups. One set of organizations will be looking at the theoretical basis for adversarial attacks on AI, why they happen and how a system can be vulnerable. Another group will be building the defenses against these attacks, and the last set of teams will serve as evaluators. Every six months, they'll test the defenses others built by throwing a new attack scenario their way and looking at criteria like effectiveness and practicality.

Over the next four years, DARPA will check in with each organization to evaluate their progress. But this is a research project — at least for now, there's no hard list of deliverables those involved need to provide to the agency.



Protocol Cloud, your weekly guide to the future of enterprise computing. Sign up now.



Intel was chosen to lead the physical adversarial attacks aspect of the project, as DARPA saw promise in the company's experience in simulating external environments for self-driving cars. Intel acquired Mobileye, a vehicle computer-vision sensor company, for $15 billion in 2017. Some consider Intel to be a dark horse in the race to build autonomous vehicles. It's made other bets in AI recently, and it also shed its smartphone modem business — which was playing catch-up to others like Qualcomm as the company missed the world's computing needs shift to mobile. Whether it has enough to win future battles in AI remains to be seen, but it's sitting on a strong base now. The company beat expectations in its last earnings report, posting over $20 billion in revenue for the quarter — nearly $1 billion more than expected. (That was, of course, before the world tumbled into a pandemic.)

The compounding computing problem

Arun Chandrasekaran, vice president and analyst at Gartner, said his research firm has seen an uptick in generative adversarial network (or GAN)-generated malicious attacks. Although the sophistication of both attack and defense technology varies greatly, he said, overall, "the detection and response to it… is not as advanced as the attacks themselves."

We're still "right on the cusp" of this problem, Draper said. The use of algorithms in military and civilian settings has skyrocketed in recent years, from providing movie recommendations to managing the stock market to making decisions on the battlefield. "What you see now is that dam breaking," Draper said. "I have a hard time projecting forward five or 10 years and thinking of any major aspect of either military or civilian life that doesn't have AI embedded into it."

And the problem is compounding: As the machine-learning framework and training data available to vendors become more widely available to hackers, Chandrasekaran said he believes malicious adversarial attacks will increase. And as the number of attacks increase, so too will their effectiveness.

"Fundamentally, this is really about trying to evade the existing detection systems," Chandrasekaran said. "You have existing systems that rely on a combination of images — in some cases, this could be voice and voice recognition… The fact that you could recreate something that's eerily similar to your voice or your image means that in many cases, you can essentially bypass the accuracy of your existing prevention system." These existing mechanisms must "significantly evolve," he added, to ensure confidence in their security.

Intel's currently focusing on the future — plugging in vulnerability holes and getting ahead of the threats downstream. "An important thing to know about this particular topic is this isn't a today threat," Jason Martin, a senior staff research scientist at Intel Labs, said. But it's a rarity in research to be able to spend time worrying about tomorrow's problems. "It's a nice place to be; it's not a 'panic now' sort of scenario," he said. "It's a 'calmly do the research and come up with the mitigations.'"

Intel and Georgia Tech have partnered on adversarial attack research for years. One of their focuses has been the ease with which bad actors can trick an algorithm into thinking a bird is a bicycle, for example, or mislabeling a stop sign — just by changing a few pixels.

The research so far, led by Duen Horng "Polo" Chau, associate professor of computing at Georgia Tech, has landed on an especially relevant takeaway: If you can't make something invulnerable from an attack, then make it computationally infeasible. For example, in some cryptography systems, there's some probability of an attacker figuring out the code key by using up considerable computing resources, but it's so improbable that it approaches impossible. Martin wants to approach the defense of physical adversarial attacks in a similar way: "The hope is that the combination of techniques in the defensive realm will make the cost of constructing an adversarial example too expensive," he said.

Intel and Georgia Tech's plan of attack

Intel and Georgia Tech plan to use some of the largest open-source image datasets — ImageNet and Microsoft's Common Objects in Context, for example. Open-source data is a DARPA requirement, and they're prioritizing research reproducibility. But those datasets are public and widely used. It begs the question: How does this effort differ from past ones?

Chau argues their innovation lies in how they plan to use the data. The plan boils down to teaching AI "coherence" — helping it see things more holistically, or, in a way, exercise common sense. It's something humans typically are born with or learn early on — and it's extremely difficult, if near impossible, to recreate with technology. Intel and Georgia Tech aim to provide three concrete solutions for a nebulous issue: temporal, semantic and spatial coherence.

Temporal coherence here relates to understanding of physics — things don't typically suddenly appear or disappear out of nowhere. For example, if a self-driving car registers a human, a stop sign or another object flickering into its view and then vanishing, then a hacker could be tampering with its system.

Semantic coherence relates to meaning. Humans identify things as a sum of their parts — a bird comprises eyes, wings and a beak, for example. The research team's plan is to incorporate a second line of defense into a sensing system — if it registers a bicycle, then it should next check for the wheel, handlebar and pedals. If it doesn't find those components, then something is likely wrong.



Get in touch with us: Share information securely with Protocol via encrypted Signal or WhatsApp message, at 415-214-4715 or through our anonymous SecureDrop.



Then there's spatial coherence, or knowledge of the relative positioning of things. If an object detector senses people floating in midair, for example, then that should be a red flag. And for all three of these strategies, the team hopes to not only teach object detectors to flag an attack but also correct it.

"It's definitely a good place to start," Chandrasekaran said of Intel and Georgia Tech's plan. He compared it to Cornell University research that used a deep forgery discriminator (a system that discriminates between real and fake items) to use human-like judgment to gauge whether or not an image of a face was fake.

Implications for the future

Chau's long-term priority is scalability, he said — making the team's research and techniques transparent by delivering open-source software. He acknowledged the risk involved in publicizing information that bad actors could use to their advantage, but argued it's even more important for people in charge of the technology to be informed. And AI systems are often a black box anyway.

"The reality is that sometimes these AI and machine learning techniques do not work — and more dangerously, sometimes it works this second, and the next second a completely different [outcome is] produced," Chau said. "It's important to know, when it works well, why does it work so well and, more importantly, when someone is really deliberately attacking it, intentionally, what is it really exploiting?"

In order for the team to counter threats, it's vital for them to proactively discover vulnerabilities that bad actors aren't yet aware of. If they don't, bad actors could end up with the tools to disassemble any new techniques they use.

"Because we're not convinced that we'll necessarily find the perfect defense, we're trying to advance the theory [and] figure out, 'What are the limits?'" Draper said. "We're going to try to defend them as best we can, make them as invulnerable as possible, but we also want to have enough of a theoretical background to develop the theory in such a way that we can tell people, when they're deploying an AI system, the extent to which it may be vulnerable or not."

And for the critical systems of tomorrow, that could make all the difference. "If you're doing a system whose job it is to recommend movies, maybe you're not that terrified of an attack — how much damage could an adversary do?" Draper said. "But if you're doing a self-driving car, then someone's life is on the line."

The fast-growing paychecks of Big Tech’s biggest names

Tech giants had a huge pandemic, and their execs are getting paid.

TIm Cook received $82 million in stock awards on top of his $3 million salary as Apple's CEO.

Photo: Mario Tama/Getty Images

Tech leaders are making more than ever.

As tech giants thrive amid the pandemic, companies like Meta, Alphabet and Microsoft have continued to pay their leaders accordingly: Big Tech CEO pay is higher than ever. In the coming months, we’ll begin seeing a lot of companies release their executive compensation from the past year as fiscal 2022 begins.

Keep Reading Show less
Nat Rubio-Licht
Nat Rubio-Licht is a Los Angeles-based news writer at Protocol. They graduated from Syracuse University with a degree in newspaper and online journalism in May 2020. Prior to joining the team, they worked at the Los Angeles Business Journal as a technology and aerospace reporter.

COVID-19 accelerated what many CEOs and CTOs have struggled to do for the past decade: It forced organizations to be agile and adjust quickly to change. For all the talk about digital transformation over the past decade, when push came to shove, many organizations realized they had made far less progress than they thought.

Now with the genie of rapid change out of the bottle, we will never go back to accepting slow and steady progress from our organizations. To survive and thrive in times of disruption, you need to build a resilient, adaptable business with systems and processes that will keep you nimble for years to come. An essential part of business agility is responding to change by quickly developing new applications and adapting old ones. IT faces an unprecedented demand for new applications. According to IDC, by 2023, more than 500 million digital applications and services will be developed and deployed — the same number of apps that were developed in the last 40 years.[1]

Keep Reading Show less
Denise Broady, CMO, Appian
Denise oversees the Marketing and Communications organization where she is responsible for accelerating the marketing strategy and brand recognition across the globe. Denise has over 24+ years of experience as a change agent scaling businesses from startups, turnarounds and complex software companies. Prior to Appian, Denise worked at SAP, WorkForce Software, TopTier and Clarkston Group. She is also a two-time published author of “GRC for Dummies” and “Driven to Perform.” Denise holds a double degree in marketing and production and operations from Virginia Tech.

Hybrid work has some distinct advantages when it comes to onboarding.

Photo: LogMeIn

Jo Deal is the chief human resources officer at LogMeIn. She is responsible for leading global people strategy with a focus on attracting, developing and engaging talent.

The desire for change that sprung up during the pandemic resulted in the highest attrition levels in decades and a fierce war for talent playing out in the market. The Great Resignation forced managers to suddenly make hiring their top priority, and recruitment partners became everyone’s best friend as leaders scrambled to replace key roles within their teams.

Keep Reading Show less
Jo Deal
Jo Deal serves as LogMeIn’s Chief Human Resources Officer. She is responsible for leading global people strategy with a focus on attracting, developing and engaging world class talent by expanding LogMeIn’s reputation as one of tech’s most desirable career destinations, and by providing a collaborative learning environment where employees can grow their careers.
Boost 2

Can Matt Mullenweg save the internet?

He's turning Automattic into a different kind of tech giant. But can he take on the trillion-dollar walled gardens and give the internet back to the people?

Matt Mullenweg, CEO of Automattic and founder of WordPress, poses for Protocol at his home in Houston, Texas.
Photo: Arturo Olmos for Protocol

In the early days of the pandemic, Matt Mullenweg didn't move to a compound in Hawaii, bug out to a bunker in New Zealand or head to Miami and start shilling for crypto. No, in the early days of the pandemic, Mullenweg bought an RV. He drove it all over the country, bouncing between Houston and San Francisco and Jackson Hole with plenty of stops in national parks. In between, he started doing some tinkering.

The tinkering is a part-time gig: Most of Mullenweg’s time is spent as CEO of Automattic, one of the web’s largest platforms. It’s best known as the company that runs WordPress.com, the hosted version of the blogging platform that powers about 43% of the websites on the internet. Since WordPress is open-source software, no company technically owns it, but Automattic provides tools and services and oversees most of the WordPress-powered internet. It’s also the owner of the booming ecommerce platform WooCommerce, Day One, the analytics tool Parse.ly and the podcast app Pocket Casts. Oh, and Tumblr. And Simplenote. And many others. That makes Mullenweg one of the most powerful CEOs in tech, and one of the most important voices in the debate over the future of the internet.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Entertainment

Peloton’s terrible, horrible, no good, very bad year

2022 just started, and Peloton has already halted bike production and is talking about mass layoffs. How did the pandemic darling get here?

How did Peloton go from pandemic star to sinking ship? One answer is the classic problem of supply and demand.

Image: Peloton; Protocol

It’s been a hell of a ride for Peloton. The headlines have been practically nonstop, from 2019’s cringey wife ad to 2021’s series of unfortunate “Sex and The City” events. But in 2020, Peloton could do no wrong. The at-home fitness company saw a 172% spike in sales over the course of that year, buoyed by the pandemic forcing wealthy gym-goers to stay home.

But nothing is ever easy or certain when it comes to Peloton. In the past week, Business Insider reported that Peloton is considering laying off 41% of its sales and marketing staff and closing down stores. CNBC learned that the company has hired McKinsey & Co. to help cut costs. And yesterday, CNBC reported that Peloton is temporarily halting production of its bikes. Peloton shares promptly plunged 24%.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Entertainment

Netflix looks to expand gaming with major IP deals, Fortnite-like updates

Remarks made to investors and recent job postings hint at big ambitions for Netflix’s nascent gaming efforts.

Netflix may be taking some cues from games like Fortnite and Apex: Legends for its own video game initiative.

Photo: Cameron Venti/Unsplash

Two months after launching mobile games to all of its members, Netflix is looking to double down on gaming: The company told investors Thursday that it wants to expand its portfolio of games “across both casual and core gaming genres.” Recent job offers suggest that this could include both live services games as well as an expansion to PC and console gaming, and the company's COO hinted at major licensing deals ahead.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Latest Stories
Bulletins