Ask HN: Any good tools to pgdump multi tenant database?

3•polishdude20•1y ago

Just joined a new company and they run a multi tenant database in postgres where each tenant is distinguished by their account ID. I'd like to be able to dump the data of a single account to my local instance to assist in debugging.

Problem is, many of the entities do not include an account ID. For example, a user can have an account ID but a user post will not but it will be tied to a user ID.

Also foreign keys aren't really implemented fully.

Are there any tools or techniques that could help make this process easier to do?

Comments

tudorg•1y ago

You can try with data subsetting tools, a few that I know are: greenmask, jailer, tonic condenser, and I think there are more. They are not exactly for your use case, but might help you a bit. The problem is most of these tools can walk foreign keys, but only in one directions.

Also, since you said FK are not fully implemented, then there just won't be enough information for them to work with. So you can either work on fixing the foreign keys or have more complex scripts.

polishdude20•1y ago

Thanks for the suggestions!

Yeah due to the unavailability of FK's, I'd have to somehow map certain ID columns to the tables they're for, accounting for slight naming convention differences.

woyten•12mo ago

In case you don’t have FKs you could specify virtual foreign keys in Greenmask.

Check out docs for details

https://docs.greenmask.io/latest/database_subset/#virtual-re...

abhisek•1y ago

This is poor design. Multi-tenant data model design must have a tenant or segment ID for every tenant segmented tables. While it may not be a problem initially but as the business grows, you may need to move tenant data to separate DBs or even different regions based on compliance requirements. IMHO it is a good idea to run DB migrations and have a tenant ID populated in all tenant segmented tables

polishdude20•1y ago

Oh yes I agree with you. It may become a big push to convince my org to do that

KV Cache and Flash Attention with interactive diagrams

The Long and Unprofitable Life of the Short Story Collection

Singapore inks AI deals with Google

An OpenAI model has disproved a central conjecture in discrete geometry

China banned Nvidia 5090D V2 while CEO Jensen Huang was in town, report claims

The reason people stay on Chrome despite its terrible privacy track record

Everyone Against Us (2023)

Why Not Tail Recursion?

How We Built Security into Computer

The Refining Crisis – Part 1: Why Fuels, Not Crude, Are the Real Story

Cloudflare issue: Workers subrequests returning 1006

Russia's plan to advertise on rockets and spacecraft takes off

Show HN: expo-callkit-telecom – easily integrate CallKit/Core-Telecom

Plan Meals. Save Cash. Stress Less

Tracking Starbucks' 'recyclable' cups: none ended up at a recycling facility

Noemi Reveals Multi-Mission Strategy for Electric Amphibious Aircraft

Startup Fundraising, just like Dating, are both deeply inefficient markets

End of the beginning – the state of global energy markets after ten weeks of war

Node.js 26.0.0 (Now with Temporal)

Researchers discover a new gigantic dinosaur species in Thailand

Ask HN: US is planning another Invasion (Cuba).Can Congress/public stop this?

Understand Russian without Studying [video]

GitHub Source code leaked. Be prepared

Show HN: Llama CPU Benchmarks

What You Will Lose When You Retire – By Dan Haylett

If an LLM is too expensive it won't be next year

Citor: A header-only C++20 thread pool tuned for sub-us dispatch

Flipper One Tech Specs

Marine scientists discover record number of new species

Notched sticks to calculators: the history of counting machines