> We funded the open-source SQLite VFS shim that helped isolate the race condition almost immediately, and will help track down similar bugs in the future.

Interesting example of a company funding open source - in this case paying for the development of a new and very specific debugging tool.

Which SQLite driver for Go does Tailscale use?

Great read. So glad they took the time to tell this story. (And glad they, as a for profit corporation, took out a support contract with SQLite. I hope they continue to do so even though this problem is resolved.)

Well written post, really enjoyed reading it.

> A single Go process exclusively accesses that database, and serves the control plane for those tailnets. This single-writer design is exactly how SQLite is meant to be used.

This line led me to believe that the writer and checkpointing logic lived on the same database connection, so I was curious to find out how the data race occurred. However, the bug details on the SQLite page[0] outline that it can only ever occur if there are multiple connections open, so the writer and the checkpointer being on different threads lead to the data race.

[0] https://sqlite.org/wal.html#the_wal_reset_bug