datartist

Six Things Called Catalog, One Credential Underneath

David Leconte — Mon, 31 Aug 2026 09:00:00 GMT

Lakehouse architecture · Part one of two

Six Things Called Catalog, One Credential Underneath

Table truth in the lakehouse has converged on one protocol. Everything above it has not, and the reason is a credential. A layer model for the parts that still differ, where Databricks, Snowflake and IBM actually stand in August 2026, and the question nobody puts in the diagram.

David Leconte · 31 August 2026 · 17 min read

Disclosure. I work at IBM France, as a Customer Success Engineer in the Data & AI team for Horizon Customers. This is written in a personal capacity and is not an IBM publication. I have a stake in one of the platforms discussed. Every factual claim is sourced to primary documentation with a retrieval date of 31 August 2026; each platform, IBM's included, receives a real criticism; and where the documentation is silent, contradictory, or merely announced, I say which. The postings on this site are my own and do not necessarily represent IBM’s positions, strategies or opinions.

The interesting part of the lakehouse catalog question is settled, and the part that replaced it is barely being discussed.

Table truth — what makes a set of Parquet files a table with a schema, snapshots and commits — has converged. All three major estates now serve the Apache Iceberg REST catalog specification. Unity Catalog implements it and vends credentials to external engines.² Snowflake's Horizon exposes it natively, powered by Apache Polaris, which graduated to a Top-Level Project at the Apache Software Foundation in February 2026.³¹ IBM's Metadata Service implements selected APIs from both the Iceberg REST and Unity Catalog specifications.⁴⁰

So the read path is contractually convergent, and every remaining difference has moved somewhere else. Upward, into who may write and under what authority, whether a policy survives leaving its perimeter, whether a semantic definition is portable. And downward, into a mechanism that almost every architecture diagram draws as a single arrow and that turns out to decide the whole governance question: what exactly the catalog hands an engine when it says yes.

You cannot see any of it while "catalog" remains one word.

How to read this. The first half builds the instrument — a six-layer model, a table of false friends, and where the three platforms actually stand in August 2026. The second half, from The credential question onward, is the part I had not seen written down anywhere and is where the argument earns its keep. If the layer model is already familiar to you, skip straight there.

One word, six jobs

Databricks says Unity Catalog and means a governance perimeter. Snowflake says Horizon Catalog and means a control surface, while Open Catalog means a hosted Apache Polaris, while a database is what actually namespaces tables. IBM says catalog and may mean a watsonx.data object binding storage to engines, or the metastore behind it, or a governance construct, or a Presto engine binding, or a Db2 system catalog.

These are not synonyms across vendors, and not reliably synonyms within one. The consequence is worse than imprecision: a comparison conducted across mismatched layers is unfalsifiable, because no evidence can settle a claim whose subject changes between speakers.

The fix is to stop treating "catalog" as an object and start treating it as a stack of jobs. Each layer answers exactly one question. An object belongs to the layer whose question it answers — and if it answers two, a vendor has collapsed layers, which is a design decision with a price attached.

The layers are separable because their lifecycles are. The dashed verticals are the table's own birth and death; every bar that crosses them, stops short of them, or repeats between them belongs to a clock the table does not control.

That right-hand panel is the argument, not decoration. The same physical table carries six independently-clocked lifecycles at once: a storage registration that outlives it, because the bucket persists after DROP; a table truth bounded by DDL; engine bindings created and dropped per engine without touching the table; policies versioned on their own cadence; a semantic definition that survives re-platforming, because a "gross margin" metric outlives the table it reads; and a share revoked while the table lives on.

Six independent lifecycles, six layers. Any claim that one object governs all of this is a claim about collapse.

A note on the numbering. L1 to L6 is my own convention for this analysis, not a vendor's. IBM, for instance, documents three access levels of its own and the label "L4" appears nowhere in its corpus. Use the layers as a lens, not as terminology to quote back at anyone.

The hourglass

Draw the market as it stands and it makes an hourglass. Many engines above, many storage systems below, and in the middle a single narrow protocol that everything now passes through.

Hourglasses are what happens when an industry agrees on exactly one thing. The internet has one at IP; the container ecosystem has one at the OCI image. The shape is diagnostic: above and below the waist, variety proliferates cheaply, because anything that speaks the waist protocol composes with everything else that does. That is precisely what has happened to table truth. An engine written against the Iceberg REST specification reads tables owned by any of the three platforms, and a table can sit on any object store any of them supports.

What makes this particular hourglass worth drawing is that it is only an hourglass in the middle. Look at what sits above the engines and the pattern breaks completely.

Convergence at the waist, divergence everywhere above it. The dashed band is the subject of the second half of this article.

The waist is real and it is the good news: an engine written against the Iceberg REST specification can read tables owned by any of the three. That is a genuine, hard-won interoperability win, and it happened fast.

The top band is the part that gets glossed over. There is no interchange standard at all for governance, semantics or distribution. No Iceberg-REST equivalent for a policy. Every enforcement model is proprietary to its perimeter, every semantic definition is expressed in a vendor's own object, and the only way to move policy between estates is to re-implement it. When someone tells you the lakehouse is now open, ask which band they mean.

The false friends

Six words survive translation between vendors while changing referent, which is the precise definition of a false friend. Read the column, not the word.

Word	Databricks	Snowflake	IBM estate
"catalog"	Top namespace level inside Unity Catalog (`catalog.schema.table`)	Historically nothing — the namespace is a database; now Horizon and Open Catalog contexts	A watsonx.data object binding storage and metastore to engines; or a governance catalog; or a Presto catalog; or a Db2 system catalog
"metastore"	The Unity Catalog region-level root object	An internal service, never user-named	The Metadata Service — table truth since version 2.1³⁹
"share"	An OpenSharing object, protocol-backed⁵	A grant of database objects to consumer accounts	Not a first-class object; nearest equivalent is REST-catalog exposure or a data product
"semantic layer"	Metric views, Domains and Pages, the latter in Beta⁷	Semantic Views³⁰	watsonx BI semantic model, single region⁴⁷; business glossary
"open catalog"	Unity Catalog OSS — a Linux Foundation sandbox project⁸	A product name: managed Polaris, closed to new first-use accounts¹⁶	The Metadata Service's Iceberg REST and Unity-compatible endpoints⁴⁰
"governance"	Grants, ABAC, now RBAC, lineage — one stack⁶	Horizon — one stack¹⁷	Ranger xor Knowledge Catalog, plus AI governance, plus engine-native — several stacks⁴²

The costliest is "open catalog". In two estates it is a common noun; in the third it is a registered product name for a service that no longer accepts new first-use accounts. A team saying "we're standardising on the open catalog" has said nothing verifiable until someone asks which one.

Where the three stand

Databricks — collapse by extension

Unity Catalog spans five of the six layers, from table truth to distribution, under one grant model and one metastore per region. Managed Iceberg reached general availability alongside Delta on 21 May 2026¹, and August 2026 alone added role-based access control at GA on the 19th, sharing of managed Iceberg tables on the 14th, and three separate ABAC extensions.⁶ Classification is native and agent-driven⁹, and business semantics now ship as metric views, Domains and Pages, where A Page is a governed, authoritative definition of a business concept⁷ — a business glossary in all but name.

The cost of the collapse comes in three parts. Policies, metrics, shares and audit exist only inside the Unity Catalog boundary. Federated catalogs are read-only, with internal Hive metastores the sole exception³, and credential vending is explicitly unsupported on foreign Iceberg tables.² And the open-source escape hatch is partial: Unity Catalog OSS remains a Linux Foundation sandbox project.⁸

Snowflake — open at the bottom, re-centred at the top

Snowflake genuinely opened table truth, contributed Polaris to the ASF, and made an unusually specific anti-fork commitment:

"Horizon Catalog uses Apache Polaris as its catalog engine, the same open source project anyone can download and operate themselves… We are not shipping a separate 'community edition' that behaves differently from what we run in production."²⁹

Read that alongside the fact that Open Catalog, the managed Polaris service, no longer accepts new first-use accounts and new customers are pointed at Horizon.¹⁶ The strategy legible from the artefacts is to commoditise the layer where a competitor led, and differentiate above it.

Which makes the governance claim the one to examine, and it will not survive the next section intact.

IBM — six layers, no single owner

IBM occupies every layer with a different product at each. The convergence point is table truth: the Metadata Service implements selected APIs from the Iceberg REST Catalog and Unity Catalog Open API spec⁴⁰. Selected is load-bearing, and IBM says so — multi-level namespace creation is unsupported, and Presto cannot query views created through the REST catalog API, an incompatibility that extends to Spark-created views too. Still, it makes watsonx.data the only one of the three whose technical catalog natively speaks a competitor's contract.

The fragmentation is at governance: an instance integrates with only one of the following policy engines — Apache Ranger or IBM Knowledge Catalog, not both.⁴² And the choice is not symmetric. Ranger's service types cover Presto and Spark⁴³, while IBM documents Knowledge Catalog governance for Presto (C++), and Presto (Java) engines only. IBM's own governance product covers fewer of IBM's own engines than the Apache alternative does. Above that, the semantic layer runs in a single region⁴⁷ and there is no marketplace equivalent to what the other two ship.

"Bidirectional" means three things

Every vendor here uses "bidirectional interoperability" for capabilities that are not the same capability. As of today there are three distinct directions with three different maturities, and conflating them is the most common technical error in these conversations.

The confusion is not accidental, and it is not quite dishonest either. Each direction genuinely is a form of writing across a catalog boundary, so each genuinely earns the adjective. What the shared phrase hides is that they answer different questions. Outbound asks whether this platform can act as a client of someone else's catalog. Inbound asks whether this platform will let someone else's engine mutate tables it owns. Through asks whether this platform will broker access to tables nobody here owns. A team that needs the second and is sold the first has bought a working feature that does not solve its problem.

They also carry different risk. Outbound writes put your data in a catalog you control, executed by an engine you are already paying for. Inbound writes hand mutation rights over your tables to an engine you may not operate, under a credential discussed later in this article. The maturity dates below matter, but the authority question matters more and does not appear on any release note.

The third panel is the one worth pausing on. A catalog that brokers access to tables it does not own has moved from layer two to layer three — it has stopped being a metastore and started being an engine binding. If you have one word for both, you cannot see that happen.

The credential question

Here is the mechanism that every architecture diagram draws as a single arrow.

When an external engine wants to read an Iceberg table, it asks the catalog to load it and signals that it can accept delegated storage access. The Iceberg REST specification defines this as an HTTP header, X-Iceberg-Access-Delegation, with exactly two permitted values — vended-credentials and remote-signing. The specification is careful to make it a request, not a demand: it is an optional signal to the server that the client supports delegated access, and the server may choose to supply access via any or none of the requested mechanisms.³²

What is striking is how little the specification says about what comes back. The StorageCredential object has exactly two required fields: a prefix, which indicates a storage location prefix where the credential is relevant, and an untyped string map. No credential type is mandated, no expiry field is defined, no lifetime semantics exist at all.³² Everything concrete is left to implementations, and they have filled the vacuum differently: Apache Polaris calls AWS STS AssumeRole with an inline session policy scoped to the specific table locations and operations on S3, mints a user-delegation SAS on Azure, and downscopes an OAuth token on GCS.³³ Databricks documents a one-hour default expiry and a dedicated privilege that is deliberately excluded from ALL PRIVILEGES to avoid accidental exfiltration.¹⁰

Now follow what happens to the user.

The end user is not in the room

This is the finding that reorganises everything above it.

Search the entire Iceberg REST OpenAPI specification for a mechanism to propagate end-user identity to storage and there is nothing — no impersonation, no on-behalf-of, no principal forwarding. The only identity plumbing in the document belongs to the deprecated OAuth token-exchange endpoint and authenticates a client to the catalog, never to the object store.³² Apache Polaris makes the consequence explicit and countable: Three distinct IAM identities take part in the S3 credential-vending flow, and the end user is not among them. Polaris uses the caller's identity to decide whether to vend and how narrowly to scope the prefix, then the client uses those credentials to talk to S3 and KMS directly.³⁵

So the identity is spent, not carried. It is consumed at the catalog to make one authorisation decision, and the storage layer never learns who asked. Object storage cannot distinguish two analysts holding credentials vended from the same role — which means per-user policy is not merely unenforced at that layer, it is unevaluable there. And a credential's granularity ceiling is a path prefix. A column mask and a row filter are both finer than any prefix can express.

None of this is a defect in the specification. It is a consequence of the thing that makes the specification valuable: direct engine-to-storage reads, with no proxy in the data path. You cannot have both that and enforcement in the same request.

What the project itself says

No primary source states the bypass in blunt terms. There is no sentence in the specification, the Iceberg documentation or the Polaris documentation saying "vended credentials bypass column masking and row filtering." That absence is worth reporting honestly, and it is not the end of the matter, because the project has documented the underlying problem in its own words — and more damningly than a blunt statement would.

The active fine-grained-access-control proposal for the REST specification puts every enforcement obligation on the client: the catalog returns restrictions, and a reader must enforce projections on the columns it is actually reading.³⁷ It has been open and unmerged since 2026; two earlier attempts to place fine-grained access control in the specification were closed as not planned. On the project's development list the reasoning is stated plainly:

"we are conflating trust with proof: proof is very hard and very expensive, and arguably unachievable"… "Trust contract (out of spec): admin/catalog decides which clients are non-malicious and wired up to enforce."³⁶

And on identity, from the same discussion: identity propagation across multi-tenant query engines is a real problem, but it's orthogonal to the spec.³⁶

The honest reading is that the tension is not denied. It is relocated — reclassified from a specification problem into a deployment and trust problem, and moved out of scope. Which is a defensible engineering decision, and one that transfers the entire burden onto whoever operates the estate. If your governance model assumes the catalog enforces, and the catalog's own project says enforcement is a property of clients you have chosen to trust, those are different models.

Worth knowing too: the alternative delegation mode, remote signing, keeps the catalog in the path for every object request — and the project's own contributors describe the cost as per-file round trips that can cause server overwhelm for very huge tables, with a non-configurable fifteen-minute signature lifetime on AWS and GCS.³⁶ The choice is not between safe and unsafe. It is between a proxy you must scale and a credential you must trust.

What each vendor actually does about it

All three had to resolve the same tension, and the convergence in their answers is the strongest evidence that the tension is structural rather than incidental.

Platform	Under vended credentials	When policy must be enforced	The cost of the resolution
Databricks	Refuses outright. You cannot use Iceberg REST catalog or Unity REST APIs to access tables with row filters or column masks.¹¹ Views and materialized views are excluded too.	Stops vending and filters server-side. Cross-engine ABAC: Databricks uses a specialized serverless compute layer to filter and return sanitized data to the external engine.¹²	Beta since 28 May 2026, and Only reads are supported from external engines when fine-grained access controls (FGAC) are enforced. The engine no longer touches storage.
Snowflake	Says so in a caption. The plain Iceberg-REST-plus-vended-credentials configuration example is headed, twice, This code doesn't enforce data protection policies:¹⁸	Routes the query home. The Spark connector supports querying tables that are protected by Snowflake policies by routing the query through Snowflake, which ensures efficient use of compute and consistent enforcement.²⁵	One connector, one engine, three policy types; Queries on tables that are protected with any other data policy result in an error. And no external writes to policy-protected tables at all.
IBM	Documents nothing. Vending arrived in version 2.2.1 and is framed purely as key hygiene — without the need to manage long-lived access keys³⁹ — with no mention of Ranger, masking, filtering or policy on any vending page.	Enforces inside the engine. Ranger service types govern tables used by Presto engine and Spark respectively, via plugins; external Spark must load IBM's own extension into the session.⁴³	A cooperating-client model, not an enforcement boundary. And the interaction between vending and policy is undocumented in both directions.

On that last cell, a word about my own employer. I searched IBM's documentation corpus in both directions: no vending page mentions Ranger, Knowledge Catalog, masking or filtering, and no policy page mentions vended credentials or the delegation header. IBM neither claims that policies still apply nor warns that they do not. I am reporting a documented silence, not a documented answer — the mechanism in the adjacent cell makes one outcome more plausible than the other, but IBM has not said so and I am not going to say it for them. If you are designing on watsonx.data, this is a question to put to IBM in writing rather than infer from an article.

Two of the three converged on the identical resolution: when fine-grained policy has to hold, stop vending credentials and bring the query back inside. Both accepted the same collateral damage — read-only, and a single engine or connector. Neither has demonstrated fine-grained enforcement co-existing with genuine direct-to-storage open-catalog access, because on the evidence above, nobody can.

One contradiction to be aware of before you quote anyone

Snowflake's own pages disagree about how far its enforcement reaches. The Horizon overview makes an engine-agnostic claim — Masking and row-access policies are enforced across any Iceberg REST Catalog-compatible engine¹⁷ — while every operational page narrows it to Apache Spark with the Snowflake Spark connector, at specified minimum versions.²⁵

What happens when Trino or DuckDB — both listed as supported engines against the same endpoint — reads a table carrying a masking policy? The documentation does not say. I found no statement that the request is blocked and none that unmasked data is returned. A connector-based mechanism has no obvious path to a non-Spark engine, which makes one answer more likely, but that is inference and I am not going to publish it as fact. If your design depends on the answer, test it or get it in writing; do not take it from either the overview page or from me.

And the key custody question underneath

There is a matching pattern one layer down, and it is the same on both major platforms, which is what makes it worth noticing rather than dismissing as a product gap.

Open-format tables sit outside each vendor's strongest key-custody boundary. Databricks supports customer-managed keys for Unity Catalog, and the scope statement is explicit: This feature only applies to catalogs backed by default storage. It doesn't apply to catalogs with external storage locations.¹⁴ Snowflake's Tri-Secret Secure gives real custody — If the customer-managed key (CMK) in the composite master key hierarchy is revoked, your data can no longer be decrypted by Snowflake²⁸ — but Iceberg tables that use Snowflake storage support only server-side encryption (SSE). Customer-managed keys (CMK) are not supported, even if your account has Tri-Secret Secure enabled.²⁷

IBM deserves the same scrutiny and gets it. watsonx.data supports BYOK through Key Protect and KYOK through Hyper Protect Crypto Services at provision time — but read the scope: Backend object storage repositories for internal metadata and 10 GB limited trial bucket.⁴⁴ That is internal metadata and a trial bucket, not customer-registered data buckets. The frequently-quoted assurance that KYOK provides technical assurance that IBM cannot access the customer keys is a statement in the Hyper Protect documentation about HPCS⁴⁶ — where the same table marks that assurance No for BYOK — and it does not appear anywhere in the watsonx.data corpus. Nor is HPCS available in most of the non-US regions watsonx.data lists.

The pattern is consistent across all three, and it is not a coincidence. Each vendor's strongest guarantees — fine-grained policy, key custody — are strongest exactly where it owns the whole path. Open the format, open the catalog, hand out a credential, and the guarantees thin out in proportion. That is the actual trade in "open lakehouse", and it is almost never priced.

What to do with this

The six layers are not a scoring rubric and they produce no winner. They produce the ability to be specific about disagreement, which is the only durable thing in a market that ships something material every fortnight.

When someone says a catalog is "more open", the model forces the question at which layer — and the answer is almost always the waist, where all three have converged and the differences have largely evaporated. A conversation that stays at the waist is a conversation about a solved problem. That is precisely why so many of them stay there.

Five questions worth more than any feature matrix:

Which layer is this claim about? If it cannot be placed, it cannot be checked.
Which direction of write? Outbound is GA since October 2025²², with Unity on Azure only from April 2026²³; inbound read went GA in February and inbound write in May 2026²¹¹⁹, having been preview in March²⁰; through-Horizon access is thirteen days old and in preview²⁴. Three capabilities, one phrase.
Is the engine reading storage directly? If yes, the granularity ceiling is a path prefix and no mask or filter is in play, whatever the overview page says.
Where does the end user's identity stop? On the evidence here, at the catalog. Design as though everything past that point is one service principal, because it is.
What is inside the key-custody boundary, and is your open-format data in it? On both major platforms, currently, it is not.

One practical habit, learned expensively. Status in this market lives in dated release notes, not on feature pages — several Snowflake feature pages carry no GA or preview banner at all, so the page documenting a capability cannot tell you whether it is generally available.¹⁸ Databricks publishes monthly platform notes with explicit GA language and dates⁴, and those are the citable artefact. Put an expiry on a claim rather than a date of writing: anything preview-tier deserves re-checking at four weeks, anything GA at a quarter.

Including this article.

Part two, Authoring Is Not Enforcing, looks at the layer this one leaves alone: the enterprise governance catalog — Collibra, Alation, Informatica, Microsoft Purview and IBM's own Knowledge Catalog. Where it sits on the model, what it can and cannot enforce, why none of them speaks the protocol at the waist, and what the seam between authoring and enforcement actually costs to operate. It ends by joining its argument to this one, and the join is the part I would read first if I were reading them in reverse.

Notes

Every source below was retrieved on 31 August 2026. Vendor engineering blogs and press releases are marked as such and used for wording, never for feature status. Where IBM's published documentation site refused automated retrieval, IBM's own documentation source repository was used instead and the provenance is flagged. Entries marked consulted support a statement without being quoted.

Databricks, Platform release notes — May 2026: managed Iceberg, foreign Iceberg and Iceberg v3 generally available, 21 May 2026; also the 28 May entry for cross-engine ABAC. docs.databricks.com ↩
Databricks, External access to Iceberg — the Iceberg REST implementation, the read/write matrix, and Credential vending on Foreign Iceberg tables is not supported. Last updated 3 August 2026. docs.databricks.com ↩
Databricks, Catalog federation: Internal Hive metastores allow reads and writes — the sole exception to read-only foreign catalogs. docs.databricks.com ↩
Databricks, Platform release notes — April 2026: ABAC GA 28 April, governed tags GA 2 April, Data Classification GA 20 April. docs.databricks.com ↩
Databricks, Platform release notes — June 2026: external lineage GA; Following the release of open-source OpenSharing, Delta Sharing is now OpenSharing. docs.databricks.com ↩
Databricks, Platform release notes — August 2026: RBAC GA 19 Aug; managed Iceberg sharing GA 14 Aug; three ABAC extensions 11, 17 and 21 Aug; Pages Beta 12 Aug. docs.databricks.com ↩
Databricks, Unity Catalog semantics and Pages: A Page is a governed, authoritative definition of a business concept. business-semantics · Pages ↩
Unity Catalog OSS: Unity Catalog is currently a sandbox project with LF AI and Data Foundation (part of the Linux Foundation). github.com/unitycatalog ↩
Databricks, Data classification: Databricks Data Classification uses an agent to automatically classify and tag tables in your catalog. docs.databricks.com ↩
Databricks, Credential vending: Issued credentials allow direct access to the cloud storage location, scoped to the relevant path.; The default expiration time is one hour.; and To avoid accidental exfiltration, ALL PRIVILEGES does not include the EXTERNAL USE SCHEMA privilege. Whether the expiry is configurable is not documented on this page. docs.databricks.com ↩
Databricks, Filters and masks: You cannot use Iceberg REST catalog or Unity REST APIs to access tables with row filters or column masks. and Path-based access to files in tables with policies is not supported. docs.databricks.com ↩
Databricks, Cross-engine ABAC (Beta): Databricks uses a specialized serverless compute layer to filter and return sanitized data to the external engine. and Only reads are supported from external engines when fine-grained access controls (FGAC) are enforced. Note the tension with note 11, which still states the flat prohibition. docs.databricks.com ↩
Databricks, Storage credentials: A storage credential represents an authentication and authorization mechanism for accessing data stored on your cloud tenant. — the service identity behind vending. docs.databricks.com — consulted
Databricks, Customer-managed keys for Unity Catalog: This feature only applies to catalogs backed by default storage. It doesn't apply to catalogs with external storage locations. and, on revocation, After you disable the key, Databricks can no longer decrypt data in catalogs using this CMK configuration. docs.databricks.com · CMK overview ↩
Databricks, SCIM provisioning — account-level SCIM from the identity provider; workspace-level SCIM is legacy. docs.databricks.com — consulted
Snowflake, Open Catalog overview: Snowflake Open Catalog is a managed service for Apache Polaris™; Customers who haven't previously created a Snowflake Open Catalog account can't sign up for their first Open Catalog account. docs.snowflake.com ↩
Snowflake, Horizon Catalog — the engine-agnostic enforcement claim: Masking and row-access policies are enforced across any Iceberg REST Catalog-compatible engine. docs.snowflake.com ↩
Snowflake, Access Snowflake-managed Iceberg tables using an external query engine through Horizon Catalog — the vended-credentials configuration, captioned twice This code doesn't enforce data protection policies:; the restriction list including Writing to tables that have fine-grained access control policies or tags isn't supported.; and the per-user-and-role token model, You need to obtain an access token for each user — service or human — and role. This page carries no GA or preview banner. docs.snowflake.com ↩
Snowflake, release note 26 May 2026 — inbound external write GA. docs.snowflake.com ↩
Snowflake, release note 16 March 2026 — inbound write, Preview. docs.snowflake.com ↩
Snowflake, release note 6 February 2026 — inbound read, GA. docs.snowflake.com ↩
Snowflake, release note 17 October 2025 — externally managed Iceberg writes and catalog-linked databases, GA. docs.snowflake.com ↩
Snowflake, release note 6 April 2026 — Iceberg write support for Azure Unity Catalog, GA. docs.snowflake.com ↩
Snowflake, release note 18 August 2026, Preview: You can now use the Horizon Iceberg REST Catalog (IRC) API to access externally managed Apache Iceberg™ tables in a catalog-linked database from external engines such as Apache Spark™, Trino, DuckDB, or PyIceberg. docs.snowflake.com ↩
Snowflake, Enforce access policies when querying with an external engine through Horizon Catalog: The Spark connector supports querying tables that are protected by Snowflake policies by routing the query through Snowflake, which ensures efficient use of compute and consistent enforcement. and Queries on tables that are protected with any other data policy result in an error. Scoped throughout to Apache Spark at specified minimum versions. docs.snowflake.com ↩
Snowflake, release note 27 January 2026 — enforce data protection policies when querying Iceberg tables from Apache Spark. docs.snowflake.com — consulted
Snowflake, Iceberg tables on Snowflake storage: Iceberg tables that use Snowflake storage support only server-side encryption (SSE). Customer-managed keys (CMK) are not supported, even if your account has Tri-Secret Secure enabled. docs.snowflake.com ↩
Snowflake, Tri-Secret Secure: If the customer-managed key (CMK) in the composite master key hierarchy is revoked, your data can no longer be decrypted by Snowflake. Neither this page nor the encryption-management page mentions Iceberg or external volumes. docs.snowflake.com ↩
Snowflake Engineering, Apache Polaris: The End of Data Vendor Lock-In, 9 April 2026. snowflake.com — vendor engineering blog ↩
Snowflake, release note 2 March 2026 — querying semantic views in standard SQL, GA. docs.snowflake.com ↩
Apache Polaris, Apache Polaris Graduates to Top Level Project!, 19 February 2026. polaris.apache.org · foundation announcement 5 March 2026: news.apache.org ↩
Apache Iceberg, REST Catalog OpenAPI specification, main branch — the X-Iceberg-Access-Delegation parameter (Optional signal to the server that the client supports delegated access…, The server may choose to supply access via any or none of the requested mechanisms.), the StorageCredential schema (Indicates a storage location prefix where the credential is relevant.), the /sign endpoint, and the deprecation of the built-in token endpoint. The specification defines no credential type, no expiry field, and no end-user identity propagation. github.com/apache/iceberg ↩
Apache Polaris 1.7.0, Vended credentials: Polaris calls AWS STS AssumeRole with an inline session policy scoped to the specific table locations and operations (read, list, write) the caller is authorized to perform. polaris.apache.org ↩
Apache Polaris 1.7.0, Access control — the privilege model, in which TABLE_READ_DATA Enables reading data from the table by receiving short-lived read-only storage credentials from the catalog: reading and receiving credentials are the same grant, and the vocabulary bottoms out at the table. polaris.apache.org — consulted
Apache Polaris 1.7.0, Configuring AWS S3: Three distinct IAM identities take part in the S3 credential-vending flow. and the client uses those credentials to talk to S3 and KMS directly. polaris.apache.org ↩
Apache Iceberg developer list, fine-grained access control and read restrictions, 2026: we are conflating trust with proof: proof is very hard and very expensive, and arguably unachievable; Trust contract (out of spec): admin/catalog decides which clients are non-malicious and wired up to enforce.; identity propagation across multi-tenant query engines is a real problem, but it's orthogonal to the spec. On remote signing: one of the concerns of the current remote signing was that it was requested was per file and it can cause server overwhelm for very huge tables. lists.apache.org · remote-signing thread — official project mailing list; discussion, not ratified specification ↩
Apache Iceberg, pull request 13879 — read restrictions for the REST specification, open and unmerged: A reader must enforce projections on the columns it is actually reading. Quoted from the raw diff. Two earlier attempts, issues 10909 and 14187, were closed as not planned. patch-diff.githubusercontent.com ↩
Apache Polaris 1.7.0, External identity providers: Apache Polaris supports authentication via external identity providers (IdPs) using OpenID Connect (OIDC) — note this is an implementation choice; the specification defines no SSO or OIDC integration. polaris.apache.org — consulted
IBM, Release notes for watsonx.data: MDS replaces the Hive Metastore from version 2.1; and at 2.2.1, The Metadata Service (MDS) in watsonx.data now supports issuing vended credentials through the Iceberg and Unity REST APIs., framed as without the need to manage long-lived access keys. IBM documentation source repository — ibm.com/docs refuses automated retrieval; this is IBM's own docs source, repository HEAD dated 26 May 2026 ↩
IBM, Unity Catalog REST API and Iceberg REST Catalog API: Metadata Service implements selected APIs from the Iceberg REST Catalog and Unity Catalog Open API spec.; X-Iceberg-Access-Delegation is supported for vended-credentials.; unsupported features include Multi-level namespace creation; and Presto cannot query views created using the Iceberg REST catalog API due to encoding incompatibility. IBM documentation source repository ↩
IBM, MDS Iceberg REST Catalog and Unity Catalog API specification — the temporary-table-credentials and temporary-path-credentials endpoints, whose response carries aws_temp_credentials, azure_user_delegation_sas, gcp_oauth_token and expiration_time. No credential lifetime is documented as an input. cloud.ibm.com/apidocs — consulted
IBM, Integrating Apache Ranger: You can only integrate with one of the following policy engines starting with watsonx.data version 2.1. — Apache Ranger or IBM Knowledge Catalog. The Knowledge Catalog page states its own scope: You can define IKC governance policies for Presto (C++), and Presto (Java) engines. Ranger page · Knowledge Catalog page — IBM documentation source repository ↩
IBM, Ranger service types: Create resource policies in this Ranger service type to enforce security on catalogs(Iceberg, Hive and Hudi), buckets, schemas and tables used by Presto engine in watsonx.data., with the equivalent Hadoop SQL type for Spark; and for external Spark, add authz.IBMSparkACExtension to spark.sql.extensions — a cooperating-client model. Ranger service types · external Spark extension ↩
IBM, Data security: Backend object storage repositories for internal metadata and 10 GB limited trial bucket are encrypted by using AES-256. Customer Bring-your-own-key (BYOK) via Key Protect and Keep-your-own-key(KYOK)via Hyper Protect Crypto(HPCS) are supported at provision time — quoted verbatim, spacing included. The regional tables list HPCS as unavailable in Frankfurt, London, Tokyo and Sydney. IBM documentation source repository ↩
IBM, Registering Amazon S3 storage: The Role ARN is used to generate temporary credentials with permissions defined by the role's access policies. — the vended credential carries the role's permissions, not the caller's. IBM documentation source repository — consulted
IBM, Hyper Protect Crypto Services FAQ: In addition to the BYOK capabilities, KYOK provides technical assurance that IBM cannot access the customer keys. The same page's comparison table marks technical assurance No for BYOK and Yes for KYOK. This statement is about HPCS and does not appear in the watsonx.data documentation. IBM documentation source repository ↩
IBM Cloud Global Catalog API, live query, 31 August 2026 — watsonx BI carries geo_tags ['us-south']. globalcatalog.cloud.ibm.com ↩

Stated limits. Three things in this article rest on absence rather than statement, and are marked as such in the text: that no primary Iceberg source declares the vending bypass in plain terms; that IBM documents no interaction between credential vending and its policy engines; and that no source settles what a non-Spark engine receives when it reads a policy-protected Snowflake table. IBM findings were retrieved from IBM's own documentation source repository because ibm.com/docs refuses automated retrieval; that repository's HEAD is dated 26 May 2026, so anything IBM published since would not appear here. Corrections are welcome and will be applied with the date attached.

David Leconte is a Customer Success Engineer in the Data & AI team at IBM France, covering Horizon Customers. He writes about lakehouse architecture, retrieval systems and the parts of data engineering that resist being tidied up. Corrections and correspondence: linkedin.com/in/davidleconte.

Part two: Authoring Is Not Enforcing.

Disclaimer. The postings on this site are my own and do not necessarily represent IBM’s positions, strategies or opinions. This article reflects the author’s own analysis and is not an IBM publication.

Trademarks. IBM, watsonx and watsonx.data are trademarks or registered trademarks of International Business Machines Corporation. Databricks and Unity Catalog are trademarks or registered trademarks of Databricks, Inc. Snowflake and Snowflake Horizon are trademarks or registered trademarks of Snowflake Inc. Collibra is a trademark or registered trademark of Collibra NV. Apache, Apache Iceberg, Apache Polaris, Apache Ranger and Apache Spark are trademarks or registered trademarks of the Apache Software Foundation. All other marks are the property of their respective owners. Their use here is nominative and descriptive; no affiliation, sponsorship or endorsement is implied.

© 2026 David Leconte. All rights reserved. Quotations from third-party documentation remain the property of their publishers and are reproduced, in short form and with attribution, for the purposes of analysis, criticism and commentary.

All articles · RSS · Mentions légales

Authoring Is Not Enforcing

David Leconte — Mon, 31 Aug 2026 09:00:00 GMT

Lakehouse architecture · Part two of two

Authoring Is Not Enforcing

The lakehouse catalogs have absorbed the glossary, the classification, the lineage and the marketplace. So what is an enterprise governance catalog for now — and why does none of them speak the one protocol the market converged on? Collibra and IBM's Knowledge Catalog, held to the same test.

David Leconte · 31 August 2026 · 15 min read

Disclosure, and it matters more here than in part one. I work at IBM France, as a Customer Success Engineer in the Data & AI team for Horizon Customers. IBM sells a product in the category this article assesses — watsonx.data intelligence, formerly IBM Knowledge Catalog — and it competes directly with Collibra, which is the other product assessed. Writing a critical piece about a competitor while employed by a rival would be worthless, so I have done the only thing that makes it worth reading: every test applied to Collibra is applied to IBM's product in the same section, and on the criterion that matters most, IBM comes out narrower. Sources are primary and dated 31 August 2026. This is not an IBM publication. The postings on this site are my own and do not necessarily represent IBM’s positions, strategies or opinions.

Part one of this pair, Six Things Called Catalog, One Credential Underneath, ended at a waist: table truth in the lakehouse has converged on the Iceberg REST catalog, and everything above it — governance, semantics, distribution — has not converged on anything at all.

This article is about the products that live in that unconverged band and have done for a decade. Collibra, Alation, Atlan, Informatica, Microsoft Purview, IBM's Knowledge Catalog. The category has a positioning problem that nobody in it enjoys discussing, and it comes down to one sentence.

An enterprise governance catalog is an L4/L5 product with no presence at L2 and no enforcement engine of its own. Everything else follows from those two facts, including the things it is genuinely good at.

What "enforcement" means when you do not own the engine

Start with the mechanism, because the marketing word covers two different things.

Collibra can enforce policy on exactly four platforms, through two separate products, and in every case by pushing policy into the platform's native engine. Collibra Protect supports the following data sources: AWS Lake Formation, BigQuery, Databricks, Snowflake¹, and its own documentation describes the mechanism as Edge capabilities translating the representation to actions toward the data source provider using their technology. On Databricks the output is a native object: Data access standards created in Protect result in column-based policies on Databricks.²

The newer Collibra Data Access — which appears to be the productised form of the Raito acquisition, though no Collibra document says so¹³ — covers BigQuery, Databricks and Snowflake plus Entra ID and Okta, and unlike Protect it reads existing native grants as well as writing them.⁴ Collibra warns that the two must not overlap: To prevent policy drift and inconsistent security, do not manage the same data sources with Data Access and Protect at the same time. Note the seam that leaves: Data Access does not cover Lake Formation, so a Lake Formation customer cannot move to the newer product.

Three documented limitations shape the operating model rather than the feature list, and they are the ones to raise in a design review:

It cannot reconcile with reality. Protect can't import or synchronize policies created directly in the data source. Synchronization is one-way: you must define your policies (standards and rules) in Collibra.³ Anything an engineer sets natively is invisible to it.
It does not revoke access, it removes masks. Although Protect removes any masking or row filtering, users can still access the data until they manually revoke the access in Databricks or Snowflake.³
Effect lags intent. Synchronisation runs hourly by default. Policy change and policy effect are separated by a window, and nothing in the authoring product tells you when the window closed.

The same test, applied to IBM

Now the part that makes the rest worth reading.

IBM's watsonx.data intelligence — the product formerly and still widely called IBM Knowledge Catalog — occupies the same slot in the model. Held to the same criterion, its enforcement reach is narrower than Collibra's, and it is not close.

Take the two documented scopes and put them side by side, because that is the only comparison the evidence actually supports. Collibra states its own: Protect supports the following data sources: AWS Lake Formation, BigQuery, Databricks, Snowflake.¹ IBM states its own: You can define IKC governance policies for Presto (C++), and Presto (Java) engines.¹⁵ Four platforms on one side; two engines inside one IBM product on the other. Both sentences are the vendor's own, unhedged, and that is the whole comparison — four platforms against two engines.

What I could not establish, stated before I draw any conclusion from it. I did not find an IBM statement that Knowledge Catalog pushes policy into Snowflake, Databricks, BigQuery or Lake Formation native engines — but I also did not find an IBM statement denying it, and the page where such a thing would live, IBM's data-protection-rules enforcement topic, refused automated retrieval on every attempt, including on the day of publication. So the honest form of the claim is narrower than the one I would like to make: IBM documents a narrower enforcement scope than Collibra documents. Whether IBM enforces more than it documents, I do not know, and neither does anyone reading only the documentation.

The connector asymmetry sharpens it. IBM's metadata harvest reaches the major non-IBM platforms — Snowflake, Databricks, BigQuery, Redshift, Teradata, SAP. But the connector list that carries governance into watsonx.data is eleven sources long and contains none of those four: IBM Knowledge Catalog - watsonx.data integration supports the following connectors for governance: Hive, Iceberg, Hudi, Delta Lake, Oracle, PostgreSQL, MySQL, SQL Server, Db2, Teradata, MongoDB.¹⁵ Breadth of cataloguing is not breadth of control, and the gap between the two lists is where that distinction lives.

And IBM's own governance product covers fewer of IBM's own engines than the Apache alternative does. A watsonx.data instance integrates with only one of the following policy engines — Apache Ranger or IBM Knowledge Catalog.¹⁶ Ranger's service types cover Presto and Spark, including external Spark via an extension. Knowledge Catalog is documented for the two Presto engines only. Choosing IBM's governance catalog over the open-source policy engine costs you Spark enforcement. That is a real trade and IBM does not, as far as I can find, publish it as a comparison table.

Two vendors, the same shape. The gap between the wide band and the narrow one is where "we govern your data estate" quietly becomes "we catalogue your data estate".

Two more findings that belong in the same paragraph as the Collibra limitations, for symmetry. IBM documents its own enforcement defects, and they are not cosmetic: Attempt to download protected data assets is allowed, and Masked data might be profiled when the data source is IBM watsonx.data.¹⁷ And IBM's own API documentation uses the verb the whole category should be using: the Knowledge Catalog APIs let you define data protection rules¹⁸. Define. Not enforce.

Nobody in this category speaks the protocol at the waist

Here is the structural point, and it applies to both products equally.

L2 converged on the Iceberg REST catalog. Collibra has no Iceberg REST catalog integration and no Apache Polaris integration. Across its current product documentation and both 2026 release-note sets, the only appearance of Iceberg is as a value of a Snowflake attribute — the table type shown as 'Dynamic Table,' 'Hybrid Table,' 'Iceberg Table,' or 'Base Table.'⁵ Collibra can record that a Snowflake table happens to be Iceberg. It cannot talk to an Iceberg catalog.

Its integration surface is overwhelmingly JDBC metadata harvest⁶ — even AWS Lake Formation is reached through Amazon Athena's JDBC driver rather than a Lake Formation API⁷ — with outbound push on exactly two platforms: Unity Catalog, which supports both inbound and outbound (in preview) metadata flows⁸ and is gated behind a support request, carrying tags only; and Google Knowledge Catalog, where outbound went GA.⁹

IBM is in the identical structural position, and sounds as though it is not. IBM markets deep Iceberg, Unity and Polaris adjacency — but that capability belongs to the watsonx.data lakehouse query engine, which is a different product from the governance catalog. The Metadata Service implements selected APIs from the Iceberg REST Catalog and Unity Catalog Open API spec¹⁹, and the federation to Snowflake Open Catalog and Unity Catalog is engine-level, done by Presto and Spark for query purposes.²⁰ The governance layer reaches that estate through a connection configured with a hostname, a port and an engine ID. It sees tables through a connector surface, exactly like Collibra does.

What this costs in practice. In an Iceberg-first estate, the governance catalog sees your tables through a platform, never through the catalog. Register the same Iceberg tables with two engines and you get two unrelated asset sets, because the harvest has no notion of the shared catalog identity underneath. The lakehouse's central promise — one table, many engines — is the thing this ingestion model structurally cannot represent.

The Apache Ranger problem

For anyone running on-premises, this is the sharpest gap in the category, and it cuts both ways.

Ranger is the enforcement engine of most Hadoop-descended and Trino-based estates, and of watsonx.data whenever Ranger rather than Knowledge Catalog is elected. There is no Collibra-built Apache Ranger integration. No Ranger page exists in Collibra's product documentation. The route that does exist is a third-party partner listing whose own terms state that Your Master Agreement with Collibra for the Collibra Service DOES NOT apply to your use of the Partner Offerings.¹⁰ The same partner supplies the Trino and Starburst equivalent. Neither listing shows a version or a supported Collibra release.

There is a matching lineage asymmetry on both sides, and it is almost comic. Collibra ingests metadata from Trino, Presto and Starburst — all three are on its certified connector list¹² — but none of the three appears among the seventeen SQL dialects it parses for technical lineage.¹¹ And IBM's lineage configurations, which run to thirty-two sources including Informatica, SSIS, Talend, SAS and Db2 for z/OS, do not include Trino or Presto either²¹ — Presto being the engine IBM's own lakehouse runs on. That second finding is weaker evidence than the first: it comes from the documentation site's navigation, corroborated in two separate collections, because the authoritative table refused retrieval. Collibra's gap is quoted from a dated page; IBM's is inferred from an index. Two governance catalogs, neither of which parses lineage for the query engine at the centre of the architecture they are governing.

One counterweight worth recording, because it is the only real bridge in either direction: IBM's Common Policy Gateway is an extensibility point, and a release note names Collibra as a candidate engine — a downloadable plugin, enabling seamless integration with any policy engine (for example, IBM Knowledge Catalog, Apache Ranger, Collibra)²². But the gateway's own topic page narrows the list to two, and the plugin is something the customer writes. It is a socket, not a shipped integration.

The seam

Everything above collapses into one operating principle, and it is the sentence I would put on the wall of any team running an enterprise catalog next to a lakehouse.

Policy is authored once in the governance catalog and compiled into the platform's own enforcement engine. It does not travel by itself. Everything that matters operationally — drift, latency between intent and effect, what happens when someone edits natively, what happens when the sync fails — lives in that compilation step, and it appears in neither product's feature list.

Read the four boxes again and notice what they have in common: none of them is a feature gap. They are properties of the arrangement. You would get all four from a perfectly implemented product on both ends, because the arrangement puts a translation step between the place where intent is expressed and the place where it binds.

Which is why the useful questions in a vendor evaluation are not about the catalog's feature matrix at all. They are: what is the compilation interval, is it one-way or reconciling, what does the target refuse to express, and what monitors the compiler.

The overlap, and why the positioning changed

Five years ago the division of labour was clean. Metastores were dumb; the enterprise catalog held the glossary, the classifications, the lineage, the quality scores and the stewardship workflow. That division no longer holds on any of the three platforms.

Capability	Unity Catalog	Snowflake Horizon	IBM estate	Still differentiated above?
Business glossary	Pages, Beta — governed definitions of business concepts²³	Semantic Views and Horizon context	Knowledge Catalog glossary	Only across estates. Within one platform, no.
Classification	Native, agent-driven, GA²⁴	Automatically discover and classify columns²⁵	Native classes and profiling	No.
Lineage	Computed natively; Collibra reads it from system tables rather than parsing¹¹	Column-level, plus OpenLineage feeds²⁵	Dedicated lineage engine — but not for Presto²¹	Only for the seams — dbt, BI tools, cross-platform hops.
Data quality	Expectations and monitors	Native	Native rules and scoring	Contested.
Marketplace	Databricks Marketplace	Internal Marketplace: Discover and share governed data products across teams without copying data.²⁵	Data Product Hub	Only across estates.
Enforcement	Native, in the query path	Native, in the query path	Ranger or Knowledge Catalog, in the query path	Never — the category delegates by construction.

Collibra's own positioning concedes the shift, and the verb is the tell. Announcing its Databricks partner award in June 2026, the company wrote that Unity Catalog gives Databricks customers a powerful governance foundation for data and AI assets across the Lakehouse and that Collibra complements that foundation, enriching Unity Catalog with the broader enterprise context layer.¹⁴ Complements. Enriches. Not governs.

IBM does the same thing in its own documentation, and more revealingly, because it is technical prose rather than a press release. IBM describes Snowflake Open Catalog is a unified governance solution for Apache Iceberg tables in Snowflake and Unity Catalog as a unified governance solution for data and AI assets in Databricks²⁰ — in the course of documenting how to federate to them. IBM is calling its competitors' catalogs governance solutions while positioning its own governance catalog above the same estate.

This is not a decline story. It is a change of job. But it is worth naming, because a great many architecture diagrams still show the enterprise catalog sitting on top as though it were the control plane, and it is not one.

What the layer is genuinely for

Four things survive the encroachment, and they are the four to buy for:

Federation across estates nobody owns. A bank with Snowflake in one division, Databricks in another, Teradata and a mainframe underneath, and an on-premises lakehouse for the regulated core has no native catalog that spans them. Collibra's certified connector list runs to several dozen sources including Dremio, Denodo, Starburst, SAP HANA and Netezza¹²; IBM's lineage configurations reach Informatica PowerCenter, SSIS, Talend, SAS and Db2 for z/OS.²¹ No platform catalog offers that breadth, and it is not close to being replicated.
Stewardship as a workflow, not a field. Ownership, approval chains, certification, deprecation — the human process that keeps a definition true. Native catalogs give you an owner attribute; they do not give you the process behind it.
Regulatory assessment and AI inventory. Collibra's AI Command Center, which replaced its previous AI Governance product in May 2026, is a registry with risk assessments, sign-offs and a trust score²⁶. Governance-as-documentation rather than runtime control — which is exactly what an EU AI Act file needs and exactly what a query engine cannot produce.
A metadata interface for agents. The Collibra MCP server is documented and its OAuth support reached GA in June 2026: an assistant can discover data assets, explore lineage, search the Business Glossary²⁷. Being the cross-platform context an agent queries is a defensible position, whatever one thinks of the category.

One deployment caveat that decides sovereign designs

Collibra is not SaaS-only: Collibra Platform Self-Hosted is actively developed, and air-gapped installation is documented, with FIPS mode mandatory in that mode.²⁸ But the self-hosted product is materially reduced, and the published Edge capability list for it covers ingestion, profiling, classification, sampling, synchronisation and the data-quality connector — with neither Protect nor Data Access on it.

I want to be careful. I could not find a page stating in words that policy enforcement is unavailable on self-hosted Collibra; the capability list omits it, which is strong but indirect. Treat "air-gapped Collibra cannot push policy" as a well-supported inference to verify with the vendor, not as a documented fact. If it holds, it decides any sovereign design: you get the authoring plane and you do not get the compilation step.

For data-residency reviews, one more: technical lineage on the SaaS path is processed by Collibra-operated regional cloud services — Collibra Data Lineage service instances are cloud-hosted endpoints that process and store technical lineage metadata.²⁹ Metadata leaves the network, not data. A self-hosted air-gapped lineage package exists, so this is a SaaS-path constraint rather than an absolute one.

The standard that does not exist

Which brings the two articles together.

At the waist, the market has a contract. An engine written against the Iceberg REST specification reads tables owned by any of the three platforms, and that convergence happened in about two years. Above the waist there is nothing of the kind. There is no interchange standard for policy. OpenLineage covers lineage. The Open Data Contract Standard covers data products. Apache Ossie — the Open Semantic Interchange, renamed on entering the Apache Incubator in July 2026³⁰ — is attempting semantics. Nothing covers a masking rule.

That absence is the whole explanation for this article. It is why enforcement is a per-platform push-down rather than a protocol. It is why the compilation step is bespoke, unmonitored and invisible to both ends. It is why Collibra reaches four platforms and IBM reaches its own Presto engines, rather than both reaching everything that speaks a common contract. And it is why the category's honest self-description has shifted from governs to complements.

The Iceberg project has looked at putting fine-grained access control into the REST specification twice, and closed both attempts as not planned. The current proposal puts the enforcement obligation on the client — a reader must enforce projections on the columns it is actually reading³¹ — and the reasoning on the project's own list is that proof is unaffordable, so trust will have to do. That is a coherent position. It also means the layer above the waist is not going to converge by protocol any time soon.

And here the two halves of this pair meet, which is the thing I did not see until both were written. Part one ended on a credential: to let an engine read the bytes, the catalog hands out a storage token scoped to a path prefix and carrying nobody's identity, at which point a column mask and a row filter become not merely unenforced but unevaluable. This article ends on an absence: above the waist there is no interchange contract for policy at all. They are the same fact seen from two ends. Below the waist, openness was bought by handing out a credential that fine-grained policy cannot survive. Above it, no standard emerged to carry that policy across perimeters — because the layer that would have had to transport it is precisely the layer the open protocol declined to model. The lakehouse got one contract, and it got it at the exact altitude where governance stops being expressible.

So the practical answer to "do we still need an enterprise governance catalog?" is: yes, and for narrower reasons than the category advertises. Not because it governs the lakehouse — it does not, and the platform does. Because it is the only thing that spans estates nobody owns, holds the human process around a definition, and produces the documentation a regulator asks for. Buy it for federation, stewardship and evidence. Do not buy it as a control plane, and do not draw it as one.

And whichever product you pick, the question that decides whether the design works is the same one, and neither vendor's feature matrix answers it: what compiles authored intent into enforced policy, how often, and how do you find out when it stops?

Notes

Every source below was retrieved on 31 August 2026. Press releases and product marketing pages are marked as such and cited only for positioning, never for feature status. Where IBM's published documentation site refused automated retrieval, IBM's own documentation source repository was used and the provenance is flagged; where a finding rests on a page's navigation rather than its body text, that is flagged too.

Collibra, About Collibra Protect: Protect supports the following data sources: AWS Lake Formation, BigQuery, Databricks, Snowflake.; and on mechanism, Edge capabilities translating the representation to actions toward the data source provider using their technology. Dated 30 June 2026. productresources.collibra.com ↩
Collibra, Databricks policies in Protect: Data access standards created in Protect result in column-based policies on Databricks. Column-based policies are applied directly to the columns on Databricks. Dated 4 May 2026. productresources.collibra.com ↩
Collibra, Protect FAQ: Protect can't import or synchronize policies created directly in the data source. Synchronization is one-way; Although Protect removes any masking or row filtering, users can still access the data until they manually revoke the access in Databricks or Snowflake.; hourly synchronisation by default. Dated 26 June 2026. productresources.collibra.com ↩
Collibra, Data Access: These access controls are automatically enforced in your underlying data sources; supported sources BigQuery, Databricks, Snowflake plus Entra ID and Okta; Both inbound and outbound; and To prevent policy drift and inconsistent security, do not manage the same data sources with Data Access and Protect at the same time. Dated 11 August 2026. productresources.collibra.com ↩
Collibra, Release notes 2026.05 and 2026.06 — the Snowflake table-type attribute ('Dynamic Table,' 'Hybrid Table,' 'Iceberg Table,' or 'Base Table.'), Semantic Views ingestion reaching GA, and the Unity Catalog outbound gate: This feature is in public preview and not enabled by default. Contact Collibra Support to activate it for your instance. 2026.06 · 2026.05 ↩
Collibra, Register and integrate data sources — the canonical integration list. Dated 16 July 2026. productresources.collibra.com ↩
Collibra, Install Protect: Ingest data from AWS Lake Formation: Download the JDBC driver for Amazon Athena. productresources.collibra.com ↩
Collibra, Databricks Unity Catalog integration methods: The integration supports both inbound and outbound (in preview) metadata flows to keep your catalog in sync with Databricks Unity Catalog. Dated 21 August 2026. productresources.collibra.com ↩
Collibra, Google Knowledge Catalog ingestion: It supports both inbound and outbound metadata flows; and the scope limit, The Knowledge Catalog integration ingests metadata only; it does not create technical lineage. Dated 12 August 2026. productresources.collibra.com ↩
Collibra Marketplace, Lorang — Data access control for Apache Ranger, a third-party Partner Offering: Partners create, own and are responsible for their Partner Offerings… Your Master Agreement with Collibra for the Collibra Service DOES NOT apply to your use of the Partner Offerings. The same partner lists a Trino/Starburst equivalent. Neither shows a version or supported Collibra release. marketplace.collibra.com ↩
Collibra, Technical lineage — supported data sources. Seventeen JDBC SQL dialects listed; Trino, Presto and Starburst appear on none of the lineage tables. Databricks lineage is read from the lineage system tables rather than parsed. Also records that The CLI lineage harvester reached its end of life on July 31, 2026. Dated 29 July 2026. productresources.collibra.com ↩
Collibra, Supported data sources for Edge — the certified connector list, including Presto, Starburst, Trino, Dremio, Denodo, SAP HANA and Netezza alongside the cloud warehouses. productresources.collibra.com ↩
Collibra, press release, Acquisition of Raito, 5 June 2025. Attributing Collibra Data Access to this acquisition is an inference from timing and scope match; no Collibra documentation names Raito as its origin. collibra.com — vendor press release ↩
Collibra, press release, Collibra named Databricks Governance Partner of the Year, 16 June 2026: Collibra complements that foundation, enriching Unity Catalog with the broader enterprise context layer. collibra.com — vendor press release, cited for positioning ↩
IBM, Integrating IBM Knowledge Catalog with watsonx.data: You can define IKC governance policies for Presto (C++), and Presto (Java) engines. and IBM Knowledge Catalog - watsonx.data integration supports the following connectors for governance: followed by Hive, Iceberg, Hudi, Delta Lake, Oracle, PostgreSQL, MySQL, SQL Server, Db2, Teradata, MongoDB. IBM documentation source repository — ibm.com/docs refuses automated retrieval; repository HEAD dated 26 May 2026 ↩
IBM, Integrating Apache Ranger: You can only integrate with one of the following policy engines starting with watsonx.data version 2.1.; Ranger service types cover tables used by Presto engine in watsonx.data and, via Hadoop SQL, the Spark engine. IBM documentation source repository ↩
IBM, Known issues and limitations for IBM Knowledge Catalog and watsonx.data intelligence: Attempt to download protected data assets is allowed; Masked data might be profiled when the data source is IBM watsonx.data. The page opens The following known issues and limitations apply to IBM Knowledge Catalog and to watsonx.data intelligence. ibm.com/docs — re-retrieved and both quotations re-confirmed verbatim on 31 August 2026 ↩
IBM, IBM Knowledge Catalog as a Service API: You can use the IBM Knowledge Catalog as a Service APIs to establish business vocabulary, import and enrich data assets, analyze data quality, define data protection rules, and more. cloud.ibm.com/apidocs ↩
IBM, Unity Catalog REST API and Iceberg REST Catalog API: Metadata Service implements selected APIs from the Iceberg REST Catalog and Unity Catalog Open API spec. IBM documentation source repository ↩
IBM, federation pages: Snowflake Open Catalog is a unified governance solution for Apache Iceberg tables in Snowflake. and Databricks Unity Catalog is a unified governance solution for data and AI assets in Databricks. — with federation performed by the Presto and Spark engines through the Iceberg REST Catalog API. Snowflake page · Databricks page — IBM documentation source repository ↩
IBM, watsonx.data intelligence lineage configurations — thirty-two source-specific entries including Snowflake, Teradata, Informatica PowerCenter, SSIS, Talend, SAS, four BI tools, OpenLineage and Db2 for z/OS; Databricks appears only as "Microsoft Azure Databricks"; Trino and Presto appear on none of them. ibm.com/docs — this finding comes from the documentation site's navigation, corroborated independently in a second collection; the authoritative table page refused retrieval, so it is weaker evidence than the quoted material elsewhere in these notes ↩
IBM, Release notes for watsonx.data: A new lightweight CPG is now available as a downloadable plugin, enabling seamless integration with any policy engine (for example, IBM Knowledge Catalog, Apache Ranger, Collibra). The Common Policy Gateway topic page narrows the named list to two engines and the plugin is customer-written. IBM documentation source repository · gateway plugin page ↩
Databricks, Pages: A Page is a governed, authoritative definition of a business concept — Beta. docs.databricks.com ↩
Databricks, Data classification: Databricks Data Classification uses an agent to automatically classify and tag tables in your catalog. docs.databricks.com ↩
Snowflake, Horizon Catalog: Automatically discover and classify columns.; Column-level lineage across Snowflake, external databases, BI tools, and OpenLineage feeds.; Internal Marketplace: Discover and share governed data products across teams without copying data. docs.snowflake.com ↩
Collibra, AI Command Center: AI Command Center helps you to monitor and manage your AI landscape from a centralized dashboard.; it replaces the fragmented product pages of AI Governance. Dated 3 August 2026. productresources.collibra.com ↩
Collibra, Model Context Protocol: The Collibra MCP server connects your AI assistant to governed metadata and business context in Collibra. Through it, an assistant can discover data assets, explore lineage, search the Business Glossary. Dated 26 June 2026; OAuth support GA per release notes 2026.06. productresources.collibra.com ↩
Collibra Platform Self-Hosted, Extend Edge capabilities: The Zarf package method is only supported if you have an Air-gapped Collibra Platform and environment.; For Air-gapped environments, FIPS mode is mandatory. This page carries the Edge capability list on which neither Protect nor Data Access appears — the basis for the inference stated in the text. productresources.collibra.com · self-hosted overview ↩
Collibra, Data Lineage service instances: Collibra Data Lineage service instances are cloud-hosted endpoints that process and store technical lineage metadata. Eleven regional endpoints. Dated 1 April 2026. productresources.collibra.com ↩
Snowflake, 8 July 2026: The project has been accepted into the Apache Incubator under a new name: Apache Ossie (Incubating) — the Open Semantic Interchange rename. snowflake.com ↩
Apache Iceberg, pull request 13879 — read restrictions for the REST specification, open and unmerged: A reader must enforce projections on the columns it is actually reading. Two earlier attempts, issues 10909 and 14187, were closed as not planned. patch-diff.githubusercontent.com ↩

Stated limits. Three claims rest on absence rather than statement and are marked as inferences in the text: that policy enforcement is unavailable on self-hosted Collibra; that neither Collibra nor IBM's governance catalog has an Iceberg REST or Apache Polaris integration; and that IBM does not push policy into non-IBM native engines. Each rests on a full read of the reachable product documentation, but absence cannot prove a page does not exist. Separately, a Snowflake–Collibra bi-directional integration was announced in June 2026 with no corresponding documentation or release-note entry, so it should be read as announced, not shipped. IBM's published documentation site refuses automated retrieval, so IBM findings come from its own documentation source repository, whose HEAD is dated 26 May 2026. Corrections are welcome and will be applied with the date attached.

Part one: Six Things Called Catalog, One Credential Underneath.

All articles · RSS · Mentions légales