All Posts

Data & Analytics

Snowflake and Databricks Just Opened Up Their Data Catalogs to Rivals

Snowflake and Databricks Just Opened Up Their Data Catalogs to Rivals

Bhavika J

Techshorts Editorial Team

The fight moved from storage to the catalog

For most of the last decade, buyers picking a data platform were really choosing a storage format: Snowflake's proprietary tables, or an open lakehouse built on Parquet files. That choice mattered less by the end of 2025. Databricks and Snowflake both spent the year opening their table catalogs to outside engines, while the underlying open table formats, Apache Iceberg and Delta Lake, converged toward reading and writing each other's tables. The interesting fight now is not which file format wins. It is which catalog gets to govern access to the data once it is sitting in open storage.

What an open table format actually does

An open table format is a metadata layer that sits next to raw Parquet files in cloud object storage (S3, ADLS, GCS) and tracks schema, partitioning, and snapshot history in plain files rather than inside one vendor's proprietary engine. Apache Iceberg, Delta Lake, and Apache Hudi are the three that matter. Because the metadata is open, multiple compute engines, Spark, Trino, Flink, DuckDB, or a cloud warehouse, can read and write the same physical data without copying it into each vendor's own storage. That is the opposite of a traditional warehouse like classic Snowflake or Redshift, where data lives in a closed format only that vendor's engine reads efficiently.

Why the format war is ending and the catalog war is starting

The two leading formats spent 2025 closing their remaining gaps instead of trying to eliminate each other. Delta Lake 4.0, released September 25, 2025, added catalog-managed tables and general availability for UniForm, its feature that lets a table written in Delta be read as Iceberg from another engine (Delta Lake, 2025). Apache Iceberg's V3 specification was ratified in 2025, adding deletion vectors and native row lineage, capabilities Delta already had (Google Open Source Blog, 2025). Amazon Web Services rolled out production support for those V3 deletion vectors and row lineage fields in November 2025 (Amazon Web Services, 2025), a sign the spec moved from paper to production quickly.

With the formats converging, the question that decides lock-in shifted up one layer: which catalog controls permissions, lineage, and optimization across those tables.

Two different bets on the catalog

Databricks open-sourced Unity Catalog under the Apache 2.0 license in 2024 (Databricks, 2024). In June 2025, it extended Unity Catalog so external engines could read and write governed, performance-optimized Iceberg managed tables directly through the Iceberg REST Catalog API, rather than reading a static export (Databricks, 2025). Snowflake took a different route: it donated its Polaris Catalog to the Apache Software Foundation, which accepted the project into incubation on August 9, 2024 (Apache Software Foundation, 2024). Polaris has shipped regular releases since, including version 1.1.0 in September 2025 (Apache Polaris, 2025).

Both moves answer the same customer complaint: table metadata locked inside one vendor's control plane. They differ in who still holds the steering wheel. Unity Catalog's most advanced governance and performance features remain tied to running compute on Databricks. Polaris is developed under Apache Software Foundation process with a broader contributor base, though Snowflake remains its largest contributor, and the project has not yet graduated out of incubation.

What to check before committing to a catalog

Buyers evaluating either option should look past the "open" label and check specifics:

Read and write parity matters more than read-only access. Confirm a non-vendor engine can actually write to a managed table under governance, not just read a snapshot export.

Feature parity across engines is not guaranteed. Row-level lineage and deletion vectors from Iceberg V3 may be available on the vendor's own compute first and elsewhere later, or not at all.

Contributor diversity is checkable for Apache-hosted projects. Look at who is actually committing code to Polaris beyond Snowflake employees before assuming Apache incubation means vendor-neutral governance today.

Migration cost does not disappear because the target format is open. Moving pipelines and testing query plans across a new catalog and engine combination is still a real project with real engineering hours.

What commonly goes wrong

Teams treat "open format" as shorthand for "open platform" and skip due diligence on the catalog layer. A table stored as Iceberg files is still functionally locked in if the only catalog offering governed write access ties back to one vendor's compute. Budget the catalog and governance migration as a separate project from the storage format migration. They are not the same decision, and two different vendors are currently trying to own the second one.

The money behind the openness push

The push toward open catalogs is happening while both platforms scale fast. Databricks raised $1 billion in September 2025 at a valuation above $100 billion, reporting a $4 billion annualized revenue run rate at the time (Bloomberg, 2025; TechCrunch, 2025), then raised additional capital at a $134 billion valuation in December 2025 (CNBC, 2025). Snowflake reported $1.21 billion in third-quarter fiscal 2026 revenue on December 3, 2025, up 29% year over year, with remaining performance obligations of $7.88 billion, up 37% (Snowflake, 2025). Meanwhile Fivetran and dbt Labs, two vendors sitting above the catalog layer in ingestion and transformation, signed a definitive agreement on October 13, 2025 to merge in an all-stock deal, creating a combined company with close to $600 million in annual revenue (Reuters, 2025). Consolidation is moving up the stack even as the storage and catalog layers open up.

What to watch next

Polaris has not yet graduated from Apache incubation to a top-level project. That graduation vote is the milestone that will show whether its governance has genuinely moved beyond Snowflake's control. Buyers should also watch for the first independently run, reproducible cross-engine benchmark comparing write performance and cost across Unity Catalog and Polaris, using a third-party engine as the client rather than a vendor testing against its own competitor.