Ahead of its sixth annual user conference, Snowflake Summit 2024, Snowflake launched its Polaris Catalog, a vendor-neutral, open catalog implementation for Apache Iceberg – an open standard of choice for implementing data lakehouses, data lakes and other architectures.
The Polaris Catalog will be open-sourced for the next 90 days to provide enterprises like Goldman Sachs and the Iceberg community with more choice, flexibility and control over their data. The offering also promises full enterprise security and Apache Iceberg interoperability with AWS, Confluent, Dremio, Google Cloud, Microsoft Azure, Salesforce and more.
Christian Kleinerman, EVP of Product, Snowflake, told ERP in a media briefing session: “We are bringing together a number of industry partners to make sure that we can give our mutual customers a choice to mix and match multiple query engines and coordinate read and write activity, and most importantly, to do so in an open fashion without having lock in.”
Explore related questions
Kleinerman also emphasized that this can “further simplify how organizations access their data across diverse systems with increased flexibility and control”.
Emerging from incubation to a top-level Apache Software Foundation project in May 2020, Apache Iceberg has since surged in popularity to become a leading open-source data table format. Building up on this, with Polaris Catalog, users can gain a single, centralized place for any engine to find and access an organization’s Iceberg tables with open interoperability.
To help Polaris Catalog meet the evolving needs of the wider community, Snowflake is collaborating with the Iceberg ecosystem to drive the project forward.
For example, Chris Grusz, MD of technology partnerships at Amazon Web Services, said that AWS is committed to working with partners on open-source solutions that can accelerate choice for customers: “We’re pleased to work with Snowflake to continue to make Apache Iceberg interoperable across our engines.”
Similarly, Raveendrnathan Loganathan, EVP of software engineering at Salesforce, said that Apache Iceberg’s popularity has contributed to an open storage standard that simplifies zero-copy data access for organizations across their ecosystem.
“We’re thrilled to have Snowflake as a member of our Zero Copy Partner Network and we’re excited to see how this new open catalog standard will further zero-copy access in the enterprise,” he said.
This development follows the recent partnership expansion between Snowflake and Microsoft, with the two supporting the industry’s leading open standards for storage formats – Apache Iceberg and Apache Parquet.
With Polaris Catalog, they continue with a joint mission of enabling users to harness their enterprise data, regardless of where it is stored, to create AI-powered applications at scale.
