frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Vendor lock-in vs. open metadata architecture? What works?

https://medium.com/datastrato/if-youre-not-all-in-on-databricks-why-metadata-freedom-matters-35cc5b15b24e
2•wey-gu•2mo ago

Comments

wey-gu•2mo ago
I was reading an article earlier today, and it brought me back to a question I’ve heard over and over again in real data/infra teams: Do we just accept vendor lock-in because it’s convenient, or do we take the pain and build an open, multi-engine metadata stack? For context (not my product, just what triggered the thought): https://medium.com/p/35cc5b15b24e I’m not trying to argue Gravitino vs. UC here — I’m more interested in the architectural mindset behind these two approaches. On the vendor-integrated side, the upsides are obvious: smoother UX one place for lineage/policies fewer moving parts But so are the downsides: cost keeps creeping up you end up tied to one engine/format migrations basically don’t happen in real life And on the open/composable side: Spark/Trino/Flink/Ray all first-class Iceberg/Hudi/Delta can actually coexist Metadata isn’t tied to compute But again: inconsistent metadata models everywhere no unified governance layer someone eventually owns a pile of glue code forever So I’m curious: what actually works in practice? If your company had to make this choice: Did you go all-in on a vendor, or build something open? Did the decision age well after a year or two? Has anyone actually avoided metadata sprawl without getting locked in? Where do lineage, ACLs, policies, and the “source of truth” actually live in your setup? Really interested in what folks think, especially if you're juggling multiple engines, table formats, and clouds.
iFire•2mo ago
My take from working on free and opensource Godot Engine and 3d formats metadata is that the main difference is if the people have the knowledge / knowledge transferred of how the process works.

If you lost the knowledge and are substituting a library (vendor) for that knowledge, you have to rewrite that library to understand its gaps and how to update it.

iFire•2mo ago
For example let's say you have a free and opensource 3d formats pipeline.

For a digital content creation tool studio (dcc) which has one tool (maya) and they use a intermediate format called (fbx) and you want to interchange a 3d avatar in Godot Engine.

If you want to amend the process to swap out maya with blender, you would need to understand how the fbx format works and also how maya, blender and Godot Engine works.

Sure you can outsource the library to an external project like assimp, but the moment a particular fbx is broken you basically start rewriting assimp. If the errors are close to 80% of the imported cases, you'd need to rewrite assimp.

Also fbx in Godot Engine is a reimplementation of FBX as there's no specifcation of FBX file format. This is similar to your vendor locked in description.

This isn't a typical enterprise data exchange process but maybe my change of the theme of the process can help.

iFire•2mo ago
> This isn't a typical enterprise data exchange process but maybe my change of the theme of the process can help.

A typical enterprise data exchange would be like parts and suppliers for inventory.

gusye•2mo ago
I’ve seen teams struggle on both sides of this.

The vendor route feels great at the beginning with clean UX and fewer moving parts, until costs creep up or you suddenly need an engine or table format the platform doesn’t really support.

The open route gives you freedom, but then you’re managing multiple catalogs, inconsistent metadata models, and a bunch of glue code nobody planned for but still ends up living forever. Gravitino seems to be tackling the “one catalog vs many catalogs” issue.

Where do lineage and ACLs actually live in your setup? I’m genuinely curious how people are handling this today.

(Bsky thread) "This turns the maintainer into an unwitting vibe coder"

https://bsky.app/profile/fullmoon.id/post/3meadfaulhk2s
1•todsacerdoti•39s ago•0 comments

Software development is undergoing a Renaissance in front of our eyes

https://twitter.com/gdb/status/2019566641491963946
1•tosh•58s ago•0 comments

Can you beat ensloppification? I made a quiz for Wikipedia's Signs of AI Writing

https://tryward.app/aiquiz
1•bennydog224•2m ago•1 comments

Spec-Driven Design with Kiro: Lessons from Seddle

https://medium.com/@dustin_44710/spec-driven-design-with-kiro-lessons-from-seddle-9320ef18a61f
1•nslog•2m ago•0 comments

Agents need good developer experience too

https://modal.com/blog/agents-devex
1•birdculture•3m ago•0 comments

The Dark Factory

https://twitter.com/i/status/2020161285376082326
1•Ozzie_osman•3m ago•0 comments

Free data transfer out to internet when moving out of AWS (2024)

https://aws.amazon.com/blogs/aws/free-data-transfer-out-to-internet-when-moving-out-of-aws/
1•tosh•4m ago•0 comments

Interop 2025: A Year of Convergence

https://webkit.org/blog/17808/interop-2025-review/
1•alwillis•5m ago•0 comments

Prejudice Against Leprosy

https://text.npr.org/g-s1-108321
1•hi41•6m ago•0 comments

Slint: Cross Platform UI Library

https://slint.dev/
1•Palmik•10m ago•0 comments

AI and Education: Generative AI and the Future of Critical Thinking

https://www.youtube.com/watch?v=k7PvscqGD24
1•nyc111•10m ago•0 comments

Maple Mono: Smooth your coding flow

https://font.subf.dev/en/
1•signa11•11m ago•0 comments

Moltbook isn't real but it can still hurt you

https://12gramsofcarbon.com/p/tech-things-moltbook-isnt-real-but
1•theahura•15m ago•0 comments

Take Back the Em Dash–and Your Voice

https://spin.atomicobject.com/take-back-em-dash/
1•ingve•16m ago•0 comments

Show HN: 289x speedup over MLP using Spectral Graphs

https://zenodo.org/login/?next=%2Fme%2Fuploads%3Fq%3D%26f%3Dshared_with_me%25253Afalse%26l%3Dlist...
1•andrespi•17m ago•0 comments

Teaching Mathematics

https://www.karlin.mff.cuni.cz/~spurny/doc/articles/arnold.htm
2•samuel246•19m ago•0 comments

3D Printed Microfluidic Multiplexing [video]

https://www.youtube.com/watch?v=VZ2ZcOzLnGg
2•downboots•19m ago•0 comments

Abstractions Are in the Eye of the Beholder

https://software.rajivprab.com/2019/08/29/abstractions-are-in-the-eye-of-the-beholder/
2•whack•20m ago•0 comments

Show HN: Routed Attention – 75-99% savings by routing between O(N) and O(N²)

https://zenodo.org/records/18518956
1•MikeBee•20m ago•0 comments

We didn't ask for this internet – Ezra Klein show [video]

https://www.youtube.com/shorts/ve02F0gyfjY
1•softwaredoug•21m ago•0 comments

The Real AI Talent War Is for Plumbers and Electricians

https://www.wired.com/story/why-there-arent-enough-electricians-and-plumbers-to-build-ai-data-cen...
2•geox•23m ago•0 comments

Show HN: MimiClaw, OpenClaw(Clawdbot)on $5 Chips

https://github.com/memovai/mimiclaw
1•ssslvky1•24m ago•0 comments

I Maintain My Blog in the Age of Agents

https://www.jerpint.io/blog/2026-02-07-how-i-maintain-my-blog-in-the-age-of-agents/
3•jerpint•24m ago•0 comments

The Fall of the Nerds

https://www.noahpinion.blog/p/the-fall-of-the-nerds
1•otoolep•26m ago•0 comments

Show HN: I'm 15 and built a free tool for reading ancient texts.

https://the-lexicon-project.netlify.app/
2•breadwithjam•29m ago•1 comments

How close is AI to taking my job?

https://epoch.ai/gradient-updates/how-close-is-ai-to-taking-my-job
1•cjbarber•29m ago•0 comments

You are the reason I am not reviewing this PR

https://github.com/NixOS/nixpkgs/pull/479442
2•midzer•31m ago•1 comments

Show HN: FamilyMemories.video – Turn static old photos into 5s AI videos

https://familymemories.video
1•tareq_•32m ago•0 comments

How Meta Made Linux a Planet-Scale Load Balancer

https://softwarefrontier.substack.com/p/how-meta-turned-the-linux-kernel
1•CortexFlow•32m ago•0 comments

A Turing Test for AI Coding

https://t-cadet.github.io/programming-wisdom/#2026-02-06-a-turing-test-for-ai-coding
2•phi-system•32m ago•0 comments