Hey HN! I built Table Canon to solve a problem my playgroup kept running into: 3-4 hour tabletop gaming sessions leave behind massive audio recordings, but standard meeting note-takers treat every session as an isolated island, butcher fantasy terms, and don't know who is speaking.

I wanted an engine that tracks long-term state across months of games, so I built a pipeline to extract entity updates, open quest hooks, and character promises across sessions.

The Tech Stack:

* Transcription: whisper-large-v3-turbo * Diarization: pyannote for speaker embeddings & voice profile matching Extraction & Memory: OpenAI API with Structured Outputs (JSON Schema enforcement for state updates) * TTS & Audio Recaps: Kokoro / Chatterbox Turbo Music Generation: ACE-Step-v1.5-XL-Turbo for rendering session summaries into lyrics/ballads

A Few Engineering Lessons & Challenges:

* State Delta Extraction vs. Context Explosions: Feeding 20 prior session transcripts into context windows quickly becomes cost-prohibitive and noisy. Instead of re-reading raw history, each session outputs an atomic state delta (updates to NPC dossiers, new locations, resolved promises) to a database. Keeping context bounded as campaigns stretch past session 30+ has been one of the trickiest architectural hurdles. * Custom Pre-Lexicons: General STT models struggle with homebrew proper nouns (turning fantasy names into standard dictionary words). Injecting a pre-pass fantasy term dictionary into prompt context significantly improved first-pass spelling. * VAD & Audio Chunking: Passing a 4-hour raw audio file directly to Pyannote/Whisper leads to memory leaks and process timeouts. Pre-processing with Voice Activity Detection (VAD) and deterministic chunking was necessary before touching the models.

Current Limitations & Active Hard Problems:

* Entity Alias Resolution: Matching entities across sessions when players use varying aliases or informal shorthand (e.g., matching "The Red Bishop" to "Arthur" or "that cult leader guy") without accidentally merging distinct NPCs. I address this, partially, but allowing the user to Edit aliases, merge or split entities after-the-fact. * Quest & Hook Resolution Logic: Fine-tuning the LLM to reliably determine whether a promise, open mystery, or quest has actually been resolved versus remaining open or implicitly abandoned.

I'd love feedback on how others are handling these sorts of issues - or any notes for folks who try it out! No initial login required with 6 hours of upload available to try.