Enterprise

Google Cloud just built a data lakehouse on BigQuery

BigLake, a new data lake storage engine that resembles data lakehouses built by newer data companies, will be at the center of Google Cloud’s data platform strategy.

Sudhir Hasbe, Google Cloud’s senior director of Product Management for data analytics

BigLake allows enterprises to unify their data warehouses and data lakes to analyze data without worrying about the underlying storage format or systems, according to Sudhir Hasbe, Google Cloud’s senior director of Product Management for data analytics.

Photo: Google Cloud

Google Cloud plans to launch a new data lake storage engine based on its popular BigQuery data warehouse to help remove barriers preventing customers from mining the full value of their ever-increasing data.

BigLake, now available in preview, allows enterprises to unify their data warehouses and data lakes to analyze data without worrying about the underlying storage format or systems, according to Sudhir Hasbe, Google Cloud’s senior director of Product Management for data analytics.

“The biggest advantage is then you don't have to duplicate your data across two different environments and create data silos,” Hasbe said in a press briefing prior to Wednesday’s Google Data Cloud Summit, where BigLake is being announced.

With BigLake, Google Cloud is extending the capabilities of its 11-year-old BigQuery to data lakes on Google Cloud Storage to enable a flexible, open lakehouse architecture, according to the cloud provider. A data lakehouse is an open data-management architecture that combines data-warehouse-like data management and optimization functions, including business intelligence, machine learning and governance, for data lakes that typically provide more cost-effective storage.

BigQuery is a Google Cloud-managed, serverless, multicloud data warehouse that lets customers run analytics over vast amounts of data in near real time. It processes more than 110 terabytes of customers’ data every second on average, according to Google Cloud.

“We have tens of thousands of customers on it, and we invested a lot in all the governance, security and all the core capabilities, so we're taking that innovation from BigQuery and now extending it onto all the data that sits in different formats as well as in lake environments — whether it's on Google Cloud with Google Cloud Storage, whether it's on AWS or whether it's on [Microsoft] Azure,” Hasbe said.

BigLake will be at the center of Google Cloud’s data platform strategy.Image: Google Cloud

BigLake will be at the center of Google Cloud’s data platform strategy, and the cloud provider will ensure that all its tools and capabilities integrate with it, according to Hasbe.

“We are going to seamlessly integrate our data management and governance capability with Dataplex, so any data that goes into BigLake will be managed [and] governed in a consistent fashion,” he said. “All of our machine-learning and AI capabilities … will also work on BigLake, as well as all our analytics engines, whether it's BigQuery, whether it's Spark, whether it’s Dataflow.”

Enterprise data sets are growing from terabytes to petabytes, while the types of data — from structured, semi-structured and unstructured data to IoT data collected from connected devices including sensors and wearables — also are increasing. That data typically is stored across different systems with different capabilities, whether in data warehouses for structured and semi-structured data or data lakes for other types of data, creating so-called data silos that could limit access and increase costs and risks, particularly when the data must be moved.

BigLake will support all open-source file formats and standards including Apache Parquet and ORC and new formats for table access such as Iceberg, as well as open-source processing engines such as Apache Spark.

“When you think about limitless data, it is time that we end the artificial separation between managed warehouses and data lakes,” said Gerrit Kazmaier, Google Cloud’s vice president and general manager for database, data analytics and Looker. “Google is doing this in a unique way.”

Workplace

An IPO may soon be in Notion’s future

Notion COO Akshay Kothari says there’s room to grow, aided by a new CFO who knows how to take a company public.

Notion has hired its first chief financial officer: Rama Katkar.

Photo: Courtesy of Notion

It’s been a year since Notion’s triumphant $275 million funding round and $10 billion valuation. Since then the landscape for productivity startups trying to make it on their own has completely changed, especially for those pandemic darlings that flourished in the all-remote world.

As recession looms, companies looking to cut costs are less likely to spend money on tools outside of their Microsoft or Google workplace bundles. Enterprise platforms are bulking up and it could spell trouble for the productivity startups trying to unseat them. But Notion COO Akshay Kothari says the company is still aiming to build the next Microsoft, not be the next Microsoft. And in a move signaling a new chapter of maturity, Notion has hired its first chief financial officer: Rama Katkar, Instacart’s former VP of finance.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.
Securing the Enterprise

Securing the enterprise

There’s no let-up in the surge of cyberattacks against businesses. But shutting down the hackers will require many enterprises to evolve their strategy.

In today’s enterprise, “identity and security are very merged.”

Illustration: iStock/Getty Images Plus; Protocol
the Protocol team
Protocol focuses on the people, power and politics of tech, with no agenda and just one goal: to arm decision-makers in tech, business and public policy with the unbiased, fact-based news and analysis they need to navigate a world in rapid change.
Fintech

How neobanks are helping consumers game credit scoring

The CFPB says it is closely monitoring secured credit cards offered by neobanks.

Regulators are scrutinizing neobanks' card offerings.

Photo: Oscar Wong/Moment/Getty Images

About one in six Americans has a credit score below 619, according to the CFPB. Another 23% have too thin a credit file to score or no file at all. That puts them in a credit trap: To build credit, these consumers need someone to give them a line of credit with which they can demonstrate good financial habits. But with scores that low, few lenders are prepared to offer them anything.

Neobanks say they can solve the problem through a new twist on secured credit cards. But regulators are already scrutinizing their offerings.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

Policy

Steel decided World War II. Chips will decide whatever is next.

“Chip War: The Fight for the World’s Most Critical Technology” foreshadows the coming battle between nations over semiconductors.

“Chip War” outlines the nature of the coming battle over semiconductors, showing how the power to produce leading-edge chips fell into the hands of just five companies.

Image: Scribner; Protocol

“World War II was decided by steel and aluminum, and followed shortly thereafter by the Cold War, which was defined by atomic weapons,” Chris Miller, a professor at Tufts University’s Fletcher School of Law and Diplomacy, writes in the introduction to his latest book. So what’s next? According to Miller, the next era, including the rivalry between the U.S. and China, is all about computing power.

That tech rivalry and the story of how the chip industry got from four to 11.8 billion transistors are all part of Miller’s book, “Chip War: The Fight for the World’s Most Critical Technology,” which comes out Oct. 4. “Chip War” outlines the nature of the coming battle over semiconductors, showing how the power to produce leading-edge chips fell into the hands of just five companies: three from the U.S., one from Japan, and one from the Netherlands.

Keep Reading Show less
Hirsh Chitkara

Hirsh Chitkara ( @HirshChitkara) is a reporter at Protocol focused on the intersection of politics, technology and society. Before joining Protocol, he helped write a daily newsletter at Insider that covered all things Big Tech. He's based in New York and can be reached at hchitkara@protocol.com.

Latest Stories
Bulletins