Ask HN: Any good tools to pgdump multi tenant database?

3•polishdude20•1y ago

Just joined a new company and they run a multi tenant database in postgres where each tenant is distinguished by their account ID. I'd like to be able to dump the data of a single account to my local instance to assist in debugging.

Problem is, many of the entities do not include an account ID. For example, a user can have an account ID but a user post will not but it will be tied to a user ID.

Also foreign keys aren't really implemented fully.

Are there any tools or techniques that could help make this process easier to do?

Comments

tudorg•1y ago

You can try with data subsetting tools, a few that I know are: greenmask, jailer, tonic condenser, and I think there are more. They are not exactly for your use case, but might help you a bit. The problem is most of these tools can walk foreign keys, but only in one directions.

Also, since you said FK are not fully implemented, then there just won't be enough information for them to work with. So you can either work on fixing the foreign keys or have more complex scripts.

polishdude20•1y ago

Thanks for the suggestions!

Yeah due to the unavailability of FK's, I'd have to somehow map certain ID columns to the tables they're for, accounting for slight naming convention differences.

woyten•1y ago

In case you don’t have FKs you could specify virtual foreign keys in Greenmask.

Check out docs for details

https://docs.greenmask.io/latest/database_subset/#virtual-re...

abhisek•1y ago

This is poor design. Multi-tenant data model design must have a tenant or segment ID for every tenant segmented tables. While it may not be a problem initially but as the business grows, you may need to move tenant data to separate DBs or even different regions based on compliance requirements. IMHO it is a good idea to run DB migrations and have a tenant ID populated in all tenant segmented tables

polishdude20•1y ago

Oh yes I agree with you. It may become a big push to convince my org to do that

Neuralwatt Doubling the Price

The great AI data centre cover-up

AI Investors Buying Accounting Companies and Force Them to Use OpenAI [video]

Zeno's Paradox and Infinite Sums – Lectures on Infinity (Lecture 1) [video]

Italy privacy watchdog fines Character.ai owner over age-check failures

Hypeplan: Turn plans into terminal keynote speeches

Semantic Fingerprinting Made Easy

Show HN: Zero2Robot – free, open source interactive textbook for robot learning

Ask HN: Recommend a Local LLM Setup (M2 Max, 9GB)

Show HN: Short SciFi/Fantasy Index

Ask HN: How do I get immoral AI?

Best Job Oriented Courses in India: Guide, Rules and Benefits

Show HN: World Clock with Twilight/Night Overlayed on Map

Ukraine's Drones Are Now Reaching Siberia and Imperiling Russian Energy Assets

Tech volatility hits highest since dot-com bust next to S

Mercor acquires Deeptune to build AI training environments

UN digital tech agency launches initiative to improve trust in AI agents

China's public QR codes are being hijacked to redirect users to porn and scams

Jukebox: Themes inspired by music cover art

Counting ArXiv Delays

China recovers Long March 10B rocket

OpenAI and Google sell AI models to blacklisted China groups

China recovers reusable rocket used in the maiden launch of Long March 10B

Think you can win on prediction markets? Here's why you're more likely to lose

I wrote about how we used Conway's Law to fix legacy architecture

"Worthless Idiot, Donkey Head": Parodies of Pedantry on the Renaissance Stage

Formally Verifying AI-Generated GPU Kernels

Why is music so much easier for AI than code?

Florida's OpenAI lawsuit moves to federal court, gets assigned to Aileen Cannon

Controlling the Human Body via EMS