I’ve been experimenting with a local approach to some of the classification tasks people are using Jev for. This approach uses text embeddings + logistic regression

On Banking77, which contains 77 categories of banking support questions, I get 94.25% using bge-large-en-v1.5 for embeddings, and 93.28% with all-MiniLM-L6-v2 (only the classifier gets trained, the embeddings model stays unchanged). For comparison:

  Model                    Accuracy Size/training time
  - IntenDD (SOTA):          94.86%  (~350M params, hours on GPU)
  - This script:             94.25%  (642 KB classifier, 3s on CPU)
  - ModernBERT fine-tuned:   93.99%  (149M params, minutes on GPU)
  - BERT fine-tuned:         92.76%  (110M params, minutes on GPU)
  - Jev (zero-shot):         87.0%   (closed API)
  - Laya (zero-shot):        42.5%   (421M params)

MiniLM’s encoder is roughly 91 MB in FP32, BGE is about 1.2 GB, the classifier and scaler are only 642 KB

The linked gist evaluates all 3,080 official test examples. Running python banking77_gist.py uses MiniLM by default, and you can choose the larger BGE encoder. Everything runs locally (although it needs to download the embedding model on the first run)

This is just a proof of concept, but it shows you can get very good results without needing GPUs, large amounts of data or hours of training, if you mostly just need classification

The direction I'm now exploring, is a system that handles familiar decisions with classifiers first, but asks an LLM for help when needed, then uses the LLM responses to automatically improve its local classifiers

What are you using Jev/Laya for?