So we developed Flide, a high-performance and exact inference engine for Jev-like decision models.
Flide achieves up to a 9.5x speedup over vLLM and SGLang.
It now supports 30+ popular local Jev-like decision models.
Start to make your decision immediately:
pip install flide
unikcc•2h ago
We built Flide from scratch to make Jev-like decision model inference up to 9.5x faster than vLLM and SGLang, while keeping exactness.
Here is the source code: https://github.com/OmniJev/Flide
Look forward to hearing your feedback and ideas!