The project is fully open source under the Apache 2.0 license. Our goal is to make it easier for developers to build useful local-first AI apps without having to stitch together a lot of different engines, runtimes, and platform-specific integrations. Under the hood, the SDK is built on top of QVAC Fabric [1], our cross-platform inference and fine-tuning engine.
QVAC SDK uses Bare [2], a lightweight cross-platform JavaScript runtime that is part of the Pear ecosystem [3]. It can be used as a worker pretty much anywhere, with built-in tooling for Node, Bun and React Native (Hermes).
A few things it supports today:
- Local inference across desktop, mobile and servers
- Support for LLMs, OCR, translation, transcription,
text-to-speech, and vision models
- Peer-to-peer model distribution over the Holepunch stack [4],
in a way that is similar to BitTorrent, where anyone can become a seeder
- Plugin-based architecture, so new engines and model types can be added easily
- Fully peer-to-peer delegated inference
A few things we know still need work:
- Bundle sizes are larger than we want right now because the current packaging of Bare add-ons is not as efficient as it should be yet
- Plugin workflow can be simpler
- Tree-shaking is already possible, but at the moment it still requires a CLI step, and we'd like to make that more automatic and better integrated into the build process
References:
[0] SDK: http://qvac.tether.io/dev/sdk
[1] QVAC Fabric: https://github.com/tetherto/qvac-fabric-llm.cpp
[2] Bare: https://bare.pears.com
[3] Pear Runtime: https://pears.com
[4] Holepunch: https://holepunch.to
[5] Docs: https://docs.qvac.tether.io
[6] Website: https://qvac.tether.io