My session files hit 3GB (2000+ files) and existing tools took 40+ seconds. Rewrote it in Rust with simd-json + rayon.
- First run: ~1s (vs 40s+) - Daily use: ~0.04s (1000x faster, cached) - TUI dashboard with daily/weekly/monthly breakdown - Supports Claude Code, Codex CLI, Gemini CLI - Persistent cache — history survives even after CLI deletes files
GitHub: https://github.com/mag123c/toktrack
Feedback welcome!
nadis•1h ago
mag123c•57m ago
The actual speed comes from
1.simd-json: This is the big one. It uses CPU SIMD instructions to parse multiple JSON bytes in parallel at the hardware level. We're talking ~3 GiB/s vs ~300 MB/s with standard parsers.
2.rayon: Dead simple parallel processing. Instead of parsing 2,000 files one by one, it spreads them across all CPU cores.
3.Rust itself: No GC means no random pauses when you're crunching through gigabytes of data. The original Node.js version would just... freeze sometimes.
The 40s → 0.04s improvement basically "what if we actually used the hardware properly?" SIMD for parsing, all cores for parallelism, no GC getting in the way. (I should probably fix that README line - thanks for pointing it out!)