Protocol | Enterprise

'It’s going to get worse before it gets better': Inside the lucrative corporate data struggle

Enterprise tech's vision of "big data" largely fell flat inside silos. But now, an army of providers think they've figured out the problems. And customers and investors are taking note.

Corn silos in grain field

Corporate data tends to settle in silos that makes it harder to understand the bigger picture. Enterprise tech vendors smell a lucrative opportunity.

Photo: Jim Witkowski/Unsplash

Data isn't the new oil; it's the new gold. And in any gold rush, the ones who make the most money in the long run are the tool makers and suppliers.

Enterprise tech vendors have long peddled a vision of corporate America centered around so-called "big data." But there was a big problem: Many of those projects failed to produce a return. An army of new providers think they've finally figured out the problem, and investors and customers are taking note.

If the first stage of this gold rush centered around collecting data, the current stage is rewarding businesses that help companies sort, manage and analyze that data across different corporate departments and applications. Siloed data is nothing new; organizations faced that challenge even when information was stored on-premises and they largely relied on only relational databases from the likes of Oracle and Microsoft.

But the landscape has changed dramatically. Enterprises are sprinting to the cloud and, after a rush in adoption of web-based applications that largely operated independently from one another, they're trying to enable those systems to work better together. Companies are also now relying on a much broader array of storage options, including data lakes, data warehouses and graph databases.

That's making the longstanding challenge of unifying that corporate information borderline impossible — at least, right now.

"Data is becoming increasingly distributed, disparate and diverse. That is placing an inherent stress on our data and analytic architectures," said Gartner Research Vice President Adam Ronthal. "If we had a silo problem when everything was on-premises, well that just got worse. And it's going to get worse before it gets better."

But organizations remain steadfast in their efforts to tackle that problem. And vendors are sprinting to own part of the estimated $216 billion big data and analytics market.

The bonanza spans everything from integration technology that helps companies share data easier between applications to tools that connect data from multiple cloud platforms. That broader exchange of information between systems, coupled with an increasingly diverse set of end users like data scientists, is mandating new tools that can create an enterprisewide governance layer.

The rise of increasingly specialized databases is adding new complexity to IT stacks, forcing organizations to find ways to defragment that storage layer. Evolving corporate needs are also spurring the adoption of newer tools like reverse ETL, a flip on the typical extract, transform and load paradigm for data transfers that helps ensure that information stored in web-based systems like Salesforce is consistent and up to date.

And that's only a taste of all the different data-focused services that are currently in the marketplace.

"There's a lot of different stress points in the market right now that are giving rise to their own set of vendors," said Ronthal. "But the end state is they are all going to be participating in a broader ecosystem."

'Some assembly required'

The world of data storage, analytics and governance is fast becoming a major area of investment. Among the startups getting flooded with cash is Snowflake-backed Alation, which is now valued at $1.2 billion after a $110 million series D round. Collibra is reportedly set to be valued at $5 billion after a $250 million funding round. VAST Data is valued at $3.7 billion after an $83 million funding round in May. And Matillion raised $150 million at a $1.5 billion valuation, its second triple-digit funding round in 2021.

But it's not just private upstarts. Publicly-traded vendors are also seeing double-digit growth in this business.

Last quarter, sales at Alteryx rose 25% to $120 million. Pure Storage's revenue jumped 23% in the three months through July to $496.8 million. Informatica, a data management vendor that was founded in 1993 and is gearing up to go public, saw subscription sales rise 25% in 2020 to $593 million, per a recent federal filing. And in a telling sign of how lucrative the market is, private equity firm Thoma Bravo spent $2.4 billion to take Talend private.

The intense focus on data and the gravity of the problem facing enterprises, including those managing an IT landscape split between on-premises and the cloud, is forcing some vendors to revamp their product suites. Pure Storage, for example, recently rolled out a new platform that makes it easier for IT departments to manage disparate data clusters by combining data sets and overlaying specified governance structures, helping developers tap the ones that work best for a specific project.

"Right now, storage is very fragmented," Pure Storage CEO Charlie Giancarlo told Protocol. "When storage is fragmented, it also means that data is fragmented. And when data is fragmented, you don't have a single source of truth and you can't get the best insight out of your information."

There's still a massive runway ahead for the sector. For example, the data analytics governance market, which includes companies like Informatica and Collibra, has been adopted by less than 1% of the potential market, Gartner estimates. The research firm doesn't expect the market to reach an "innovation plateau" for over a decade.

It's one reason why vendors are spending big to scale fast to try to get an early lead in many of the still nascent segments of the market. Fivetran, for example, spent $700 million to acquire quasi-rival HVR, a sum that dwarfs the $565 million series D funding round it announced the same day as the deal.

"It shows how serious we are about the core data movement problem," said Fivetran CEO George Fraser. "We are very determined to grind on this problem in every content, every app, every database, and just cover all the bases. It's a very hard problem and we recognize what a hard problem it is."

While industry experts say there's not yet an out-of-the-box product that will solve all of a corporation's data issues, the industry is heading in that direction. Google Cloud, for example, has Dataplex, its own unified analytics and data management tool.

As that evolution continues, it's likely the cloud vendors — namely Google Cloud, Alibaba, Oracle, AWS and Microsoft Azure — only get more powerful. That means independent software vendors (ISVs) are increasingly going to need to cozy up even closer to that small group to reap the financial benefits of being a part of whichever ecosystem an end customer primarily operates in.

"We're starting to move towards [a] 'some assembly required' approach to data management," said Ronthal. And as a result, "there will be this coalescence in the market that starts to refine itself around a half-dozen offerings."

That means a wave of consolidation is likely looming ahead. But while the crystal ball suggests a much more streamlined industry ahead, ISVs still think there's an opportunity to win without the crutch of a cloud partner.

"Do I think it will happen? I do," said Talend CEO Christal Bemont. But "to be completely beholden to one thread like that, some will do that, but the large majority will struggle with being completely single-threaded."

Protocol | Workplace

Google contractor says she was fired for "ungoogley" behavior

According to a charge filed with the National Labor Relations Board, "ungoogley" is Google's term for having a bad attitude.

A contractor at Google staffing firm Modis claims she was fired from her job after asking about pay.

Photo: Future Publishing/Getty Images

A contractor at Google staffing firm Modis claims she was fired from her job for "ungoogley" behavior after asking about holiday pay at a meeting with management, according to a charge filed with the National Labor Relations Board by a lawyer for the Alphabet Workers Union.

Tuesday Carne said in an interview with Protocol that she was fired after just nine days of working in the data contracting facility in South Carolina. Carne's termination letter (which Protocol reviewed) called her behavior at the meeting "unacceptable and 'ungoogley'" and claimed that her behavior was the reason for her firing.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

The fintech developers who made mobile banking as routine as texting or online shopping aren't done. The next frontier for innovation is open banking – fintech builders are enabling consumers to be at the center of where and how their data is used to provide the services they want and need.

Most people don't even realize they're using open banking services today. If they connected their investment and banking accounts in a personal financial management solution or app, they're using open banking. Perhaps they've seen ads about how they can improve their credit score by uploading pay stubs or utility records to that same app – this is also powered by open banking.

Keep Reading Show less
Bob Schukai
Bob Schukai is Executive Vice President of Technology Development, New Digital Infrastructure & Fintech at Mastercard, where he leads the technical design, execution and support of innovative open banking and fintech solutions, as well as next generation technologies to support global payment and data capabilities. Prior to Mastercard, Schukai’s work focused on cognitive computing, financial technology, blockchain, user experience and digital identity. He is also a member of the Institute for Electrical and Electronics Engineers.
Protocol | Policy

Biden FCC nominee Sohn is walking a tightrope with Republicans

Gigi Sohn faces plenty of GOP opposition, but the longtime net-neutrality advocate is hoping to pick up a little Republican support as she deals with Democrats’ narrow margins.

Gigi Sohn’s work for net neutrality has become an issue in her confirmation hearings for the FCC.

Photo: Alex Wong/Getty Images

Gigi Sohn wouldn’t mind getting support from a Republican or two, and it’d certainly make her path back to the Federal Communications Commission easier.

During her Senate Commerce Committee confirmation on Wednesday, Sohn, a progressive favorite and longtime net-neutrality advocate, touted her commitment to ensuring a diversity of voices on the airwaves, her past fights for small conservative networks she personally disagrees with and her habit of socializing with those she battles on policy.

Keep Reading Show less
Ben Brody

Ben Brody (@ BenBrodyDC) is a senior reporter at Protocol focusing on how Congress, courts and agencies affect the online world we live in. He formerly covered tech policy and lobbying (including antitrust, Section 230 and privacy) at Bloomberg News, where he previously reported on the influence industry, government ethics and the 2016 presidential election. Before that, Ben covered business news at CNNMoney and AdAge, and all manner of stories in and around New York. He still loves appearing on the New York news radio he grew up with.

Protocol | Workplace

Microsoft Teams is going after small businesses

Microsoft Teams Essentials offers longer, bigger meetings for a relatively small price tag.

Companies can now buy a standalone version of Teams.

Photo: Mika Baumeister/Unsplash

Microsoft announced Wednesday that companies can now buy a standalone version of Teams — one of its most important products and a major player in work messaging and video chat, alongside Slack and Zoom. The product, called Microsoft Teams Essentials, aims to give small or medium-sized businesses a communication hub that costs less than its competitors'.

Microsoft will charge small businesses $4 per user per month for Microsoft Teams Essentials, while Zoom’s cheapest paid plan is $14.99 per user per month and Slack’s is $6.67 per user each month, when billed annually. The free version of Microsoft Teams still exists, as do the various other Microsoft 365 plans that include Teams. Teams Essentials offers longer meeting times, larger group meetings and more cloud storage.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Protocol | Policy

5 things to know about NTIA nominee Alan Davidson

If confirmed, the former Googler will play a key role in shaping the unprecedented expansion of broadband across the country.

Alan Davidson has been nominated to lead the NTIA.

Photo: Win McNamee/Getty Images

On Wednesday, the Senate Commerce Committee is holding a hearing to confirm President Joe Biden’s nominee to lead the National Telecommunications and Information Administration — a traditionally somewhat-sleepy role that is taking on new prominence in the wake of the passage of the bipartisan infrastructure bill.

That law gives the NTIA authority to write the rules and oversee the distribution of $42.5 billion in broadband infrastructure grants to states, a duty that will require it to massively scale its internal resources. To lead the charge, Biden has nominated Alan Davidson, a well-known figure in Washington who has spent his career cycling through government, industry and advocacy groups. If confirmed, Davidson would have perhaps the most important role in guiding an unprecedented expansion of internet access in America.

Keep Reading Show less
Issie Lapowsky

Issie Lapowsky ( @issielapowsky) is Protocol's chief correspondent, covering the intersection of technology, politics, and national affairs. She also oversees Protocol's fellowship program. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University's Center for Publishing on how tech giants have affected publishing.

Latest Stories