OK, now for the longer, I-rambled-too-much version:
For the past few weeks I've been working on a macOS journaling app called VoxThermic.
The idea for it came from my own journaling habits, where my mood tends to shift a lot depending on whatever I'm dealing with. I've tried some journaling apps, but I've never found one that can analyze mood or see how my mood changes depending on what I'm writing about.
So I just figured I'd build it myself. I've been experimenting a lot with Apple's Foundation Models and Natural Language frameworks, and those all work on-device and offline (which is good because there's no way I'm sending journal entries to some AI company's data centers!)
But there's one problem with using Foundation Models: they're tiny. About 3 billion parameters, with a context window of 4096 tokens. That's fine for analyzing ONE journal entry. But how would I get it to analyze multiple entries?
It took a lot of experiments and the Foundation Model just outright refusing to work for several of them. Turns out it just refuses to work if you send in too much data... who would have guessed? Eventually I came up with one solution: summarizing multiple entries and then passing those into the model. Sometimes the summaries get summarized too, depending on the amount of data to process.
The approach works, but summarization loses detail and nuance. The analysis for a single journal entry can have a lot of detail and depth, because there's no summarizing involved and thus no data loss. The summary for a year's worth of entries... that's not so detailed. I still haven't found a better approach that stays entirely on-device.
Oh yeah, on an unrelated note: I also like to ramble a lot (you could have guessed, huh?), but I like to ramble out loud, rather than type. So I made voice transcription a big feature of the app, though this was the easy part.
Anyway, everything here works on-device, and it can run offline. And I'm aware this is probably a niche app, but it's something I wanted for myself, and I hope it comes in handy for someone else too. The app is free to download and use, with optional features like AI summaries and PDF exports.
Lastly, I also wonder if anyone's figured a better way of doing long-term analysis with Foundation Models? My "summarize the data and pass in a collection of summaries" technique works, but I don't think it's elegant, and yet at the same time I haven't been able to figure out anything better.