Federated Metadata: Beyond the Catalog
A Three-Essay Series on Coordination, Authority, and Cost
Data catalogs serve fundamentally different purposes. Builders need speed and operational metadata. Compliance needs coverage and audit trails. Consumers need curated, ready-to-use products.
Modular catalogs recognize this: the Platform Catalog for builders, the Governance Registry for compliance, the Data Marketplace for consumers. Purpose-built, fit-for-context, deliberately bounded. When everything becomes findable in one catalog, nothing gets found. Separate catalogs solve that.
But when metadata about the same object lives in multiple places, how does it stay coherent? This is the coordination challenge — and it’s architectural, but also about authority:
Who maintains coherence across distributed systems, and who holds the power to enforce the standards that make coordination possible?
And as agentic AI moves into operational workflows, the stakes shift from inconvenience to operational failure. Humans can mentally stitch together inconsistent metadata. Agents cannot. When an agent follows an identifier that has drifted between systems, the failure is immediate and operational — not a report to fix later, but a system that stops working now.
The Essays
When Everything Becomes Findable, Nothing Gets Found makes the case for modular catalogs and shows why comprehensive approaches collapse. The Platform Catalog for builders operates at speed. The Governance Registry for compliance prioritizes coverage and auditability. The Data Marketplace for consumers curates ready-to-use products. Different rhythms, different quality bars, deliberate boundaries.
The Metadata Coordination Problem examines how metadata stays coherent across distributed systems without recreating centralization. Federated metadata treats coordination as fundamentally distinct from cataloging — using shared identifiers and synchronization standards at the boundaries where domains genuinely overlap. But coordination requires authority, not just good intentions. The question is where that authority lives.
What Federation Actually Costs identifies three inherent costs of distributed ownership that become blockers when unrecognized: expertise in metadata coordination, mechanisms for version alignment across domains, and authority to enforce coherence standards. These aren't temporary friction or implementation bugs — they're ongoing investments that successful federation requires by design.
Why This Matters Now
Federation isn’t a future possibility — it’s the architecture that emerges when intelligence operates at the edges. When agents live in workflows at the point where data originates, you cannot centralize what lives distributed.
The question isn’t whether to federate. It’s whether organizations fund what federated metadata actually demands — or treat coordination as something that should just happen, then watch initiatives stall on metadata problems nobody budgeted for.
A Learning Journey
These essays represent my own exploration of what federation actually demands. The first essay was straightforward — cataloging’s failure modes are well-traveled ground. The coordination problem in the second essay pushed me into less settled territory, working through how federated metadata stays coherent without recreating centralization.
The third essay is where I’m still learning. The coordination tax feels real and observable, but I’m not yet confident I have the mental models right. The relationship between distributed ownership and central discipline, the way agents change the calculus, the question of what “minimum viable coordination” actually means in practice — these need further iteration and more mature thinking.
I’m publishing this as exploration, not conclusion. If you’re navigating similar challenges, I’d genuinely value hearing what rings true, what doesn’t, and where the thinking needs refinement. Which question hits your current challenge?
Additional Reading
For more on the authority question that runs through these essays:
The Authority Crisis in Data Governance — why governance without enforcement authority becomes expensive advocacy
The CIO Should Eat the CDO — why system integrity has replaced data governance as the organizing principle
You seem to write about the things I am thinking about in my daily work these days. Looking forward to our next chat!
My approach to federation is: ‘there is probably always someone who is outside your data community who you want to interact with regardless’. This is what I learned when working with data producers/owners/providers across Europe. Federating data and catalogues as well as the applications themselves through shared architectures and modules was a feasible way of approaching this.