Source Code: Your daily look at what matters in tech.

next-upnext upauthorJanko RoettgersNoneDo you know what's coming next up in the world of tech and entertainment? Get Janko Roettgers' newsletter every Thursday.9147dfd6b1
×

Get access to Protocol

Will be used in accordance with our Privacy Policy

I’m already a subscriber
People

Adobe wants to use AI to make you a better dancer

The company's new tech demo could improve everyone's TikTok videos — and hints at the potential for AI to democratize video editing and visual effects.

Adobe wants to use AI to make you a better dancer

The technology can also be used to take clips and adjust movements to a different song, or take multiple clips from different sources and adjust all of them to the piece of music.

Image: Adobe

Can't dance? You're not alone.

"Syncing up music and dancing can be hard," said Adobe Research Scientist Jimei Yang during a recent interview. Not only can holding the beat be challenging for some people, but using consumer-grade recording equipment can also introduce additional delays that make the result look off-beat. "It isn't that trivial," Yang said.

That's why Yang's team at Adobe developed technology that can tweak an existing video and make dance moves match the rhythm of the music. The researchers are set to demonstrate their work, dubbed "Project On The Beat," at Adobe's annual Max conference Wednesday, but Yang gave Protocol an exclusive preview last week.

This is how Adobe's dance improvement algorithms work in a nutshell:

  • First, the project uses Adobe's Sensei AI tech for skeletal tracking to identify the motions of the dancer.
  • Then, it breaks those motions down to find the strongest movements — think claps and stomps — which Adobe's researchers call "motion beats."
  • The video's music is segmented based on these key poses, and the motion beats are compared to the actual beats of the song.
  • Finally, the video is adjusted segment by segment to match the motion beats to the music. "We just need to retime the video, basically warp the video," Yang said.

The technology can also be used to take clips and adjust movements to a different song, or take multiple clips from different sources and adjust all of them to the piece of music. This even works with videos that only show a person's upper body.

Like much of the technology shown during these Adobe Max research demos, Yang's work isn't part of any Adobe product yet. However, that could change. "We are actively working with the product team," Yang said. One could imagine it as an addition to a future version of Adobe Premiere Rush, the company's mobile video-editing app for influencers and other social media users.

Beyond any such specific use cases, the demo shows the potential for AI to automate and ultimately democratize video editing. Until now, video editors would have to manually segment and adjust each part of a video to synchronize music and movement with desktop video-editing software. By letting AI do the work, the same can be done faster, and on mobile devices, literally putting this kind of video editing into many more hands. "AI is really changing the video editing business, especially for amateurs," Yang said.

A version of this story will appear in our entertainment newsletter, Next Up. Sign up here to get it in your inbox every Thursday.

People

To combat disinformation, centralize moderation

There's more to content moderation than deplatforming.

In addition to interplatform collaboration, big tech companies would also benefit from greater collaborations with academic researchers, government agencies or other private entities, the authors argue.

Image: Twitter

Yonatan Lupu is an associate professor of political science and international affairs at George Washington University. Nicolás Velasquez Hernandez is a lecturer at the Elliott School of International Affairs and a postdoctoral researcher at GW's Institute for Data, Democracy and Politics.

Florida Gov. Ron DeSantis' signing of a bill that penalizes social media companies for deplatforming politicians was yet another salvo in an escalating struggle over the growth and spread of digital disinformation, malicious content and extremist ideology. While Big Tech, world leaders and policymakers — along with many of us in the research community — all recognize the importance of mitigating online and offline harm, agreement on how best to do that is few and far between.

Keep Reading Show less

As President of Alibaba Group, I am often asked, "What is Alibaba doing in the U.S.?"

In fact, most people are not aware we have a business in the U.S. because we are not a U.S. consumer-facing service that people use every day – nor do we want to be. Our consumers – nearly 900 million of them – are located in China.

Keep Reading Show less
J. Michael Evans
Michael Evans leads and executes Alibaba Group's international strategy for globalizing the company and expanding its businesses outside of China.
Protocol | Fintech

Marqeta turns to a fintech outsider

Randy Kern, a Salesforce and Microsoft veteran, is taking a plunge into the payments world.

Randy Kern is joining Marqeta after decades at Microsoft and Salesforce.

Photo: Marqeta

Marqeta has just named a new chief technology officer. And it's an eyebrow-raising choice for a critical post as the payments powerhouse faces new challenges as a public company.

Randy Kern, who joined Marqeta last month, is a tech veteran with decades of engineering and leadership experience, mainly in enterprise software. He worked on Microsoft's Azure and Bing technologies, and then went on to Salesforce where he last served as chief customer technology officer.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

Protocol | Policy

What can’t Jonathan Kanter do?

Biden's nominee to lead the DOJ's antitrust section may face calls to remove himself from issues as weighty as cracking down on Google and Apple.

DOJ antitrust nominee Jonathan Kanter's work as a corporate lawyer may require him to recuse himself from certain cases.

Photo: New America/Flickr

Jonathan Kanter, President Joe Biden's nominee to run the Justice Department's antitrust division, has been a favorite of progressives, competitors to Big Tech companies and even some Republicans due to his longtime criticism of companies like Google.

But his prior work as a corporate lawyer going after tech giants may require him to recuse himself from some of the DOJ's marquee investigations and cases, including those involving Google and Apple.

Keep Reading Show less
Ben Brody

Ben Brody (@ BenBrodyDC) is a senior reporter at Protocol focusing on how Congress, courts and agencies affect the online world we live in. He formerly covered tech policy and lobbying (including antitrust, Section 230 and privacy) at Bloomberg News, where he previously reported on the influence industry, government ethics and the 2016 presidential election. Before that, Ben covered business news at CNNMoney and AdAge, and all manner of stories in and around New York. He still loves appearing on the New York news radio he grew up with.

Protocol | Enterprise

Couchbase plots escape from middle of database pack with $200M IPO

The company has to prove it can beat larger rivals like MongoDB, as well as fast-growing competitors like Redis Labs, not to mention the big cloud companies.

Couchbase celebrates its initial public offering on the Nasdaq market.

Photo: Nasdaq

At first glance, Couchbase appears to be stuck in the middle of the cloud database market, flanked by competitors with more traction and buzz. But fresh off a $200 million IPO Thursday, CEO Matt Cain relished the opportunity ahead to prove why his company can beat out rivals the market considers more valuable.

The NoSQL database provider's public offering helped propel Couchbase to a $1.2 billion valuation. But unlike one of the last big data-related IPOs, market leader Snowflake's historic debut on the public markets last December, Couchbase has some work to do to differentiate itself.

Keep Reading Show less
Joe Williams

Joe Williams is a senior reporter at Protocol covering enterprise software, including industry giants like Salesforce, Microsoft, IBM and Oracle. He previously covered emerging technology for Business Insider. Joe can be reached at JWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

Latest Stories