Ask HN: Any good tools to pgdump multi tenant database?

3•polishdude20•1y ago

Just joined a new company and they run a multi tenant database in postgres where each tenant is distinguished by their account ID. I'd like to be able to dump the data of a single account to my local instance to assist in debugging.

Problem is, many of the entities do not include an account ID. For example, a user can have an account ID but a user post will not but it will be tied to a user ID.

Also foreign keys aren't really implemented fully.

Are there any tools or techniques that could help make this process easier to do?

Comments

tudorg•1y ago

You can try with data subsetting tools, a few that I know are: greenmask, jailer, tonic condenser, and I think there are more. They are not exactly for your use case, but might help you a bit. The problem is most of these tools can walk foreign keys, but only in one directions.

Also, since you said FK are not fully implemented, then there just won't be enough information for them to work with. So you can either work on fixing the foreign keys or have more complex scripts.

polishdude20•1y ago

Thanks for the suggestions!

Yeah due to the unavailability of FK's, I'd have to somehow map certain ID columns to the tables they're for, accounting for slight naming convention differences.

woyten•1y ago

In case you don’t have FKs you could specify virtual foreign keys in Greenmask.

Check out docs for details

https://docs.greenmask.io/latest/database_subset/#virtual-re...

abhisek•1y ago

This is poor design. Multi-tenant data model design must have a tenant or segment ID for every tenant segmented tables. While it may not be a problem initially but as the business grows, you may need to move tenant data to separate DBs or even different regions based on compliance requirements. IMHO it is a good idea to run DB migrations and have a tenant ID populated in all tenant segmented tables

polishdude20•1y ago

Oh yes I agree with you. It may become a big push to convince my org to do that

Mbmn

AWS Continuum for code vulnerabilities is now generally available

LibreSSL 4.4 Released with Faster SHA3, Bug Fixes

Volkswagen's best-seller has gone electric, will replace the ID.4 and ID.5

Elon Musk Invests $1B in Skydance (New Paramount/Warner Bros Discovery)

'Smart Wall' of AI Cameras Is Watching the Border and More Than a Million People

Pangram CEO: People believe AI writing is indistinguishable from ours. It's not

OBS Studio 33

The current state of new media

AOM Sees Rapid AV2 Adoption Across the Ecosystem Just 4 Months After Release

Welcome to 2030. I own nothing, no privacy, and life's never been better (2016)

Elon Musk Acquires $1B Stake in CNN's Parent Company

Who will become the new mathematicians of Math 2.0?

Who Wrote This Byte?

No Pussyfooting (1973)

Yab, yet Another Browser

Pi, the coding agent, compiled ahead of time to one native executable

Julia Returns to Tiobe's Top 20

Oura sues rival Ultrahuman again, threatening its U.S. smart ring comeback

Tehran has detailed plans to hit Gulf undersea cables if war resumes

Uringscope: eBPF Tracer and Analyzer for io_uring

Regular Expression Matching Can Be Simple and Fast (2007)

Hitler's Eagle's Nest retreat 'completely ablaze'

Security Infrastructure for Developers and Agents

Filekin – PDF tools with no signup, no limits and no watermark

Indian government sites are serving revoked eMudhra certificates

The Age of Reasoning [pdf]

Misure di sicurezza rafforzate NIS: stato e preparazione

Mamdani Makes a Splash with His Bidets

Sam Altman: 'I No Longer Think We're Gonna Be Able to Just Patch All the Bugs'