15:I[981,["658","static/chunks/658-a698693f56d0116a.js","183","static/chunks/183-6beeeb801c274aaa.js","474","static/chunks/app/%5Btype%5D/layout-e8e83705b5c8d3b7.js"],"Posts"] 16:T5fd,Hi HN,

I built llm.sql, an LLM inference framework that reimagines the LLM execution pipeline as a series of structured SQL queries atop SQLite.

The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth).

When available memory is less than the model size and KV cache, the OS incurs page faults and swaps pages using LRU-like strategies, resulting in throughput degradation that's hard to notice and even harder to debug. In fact, the memory access pattern during LLM inference is deterministic - we know exactly which weights are needed and when. This means even Bélády's optimal page replacement algorithm is applicable here.

So instead of letting the OS manage memory, llm.sql takes over:

- Model parameters are stored in SQLite BLOB tables

- Computational logic is implemented as SQLite C extensions

- Memory management is handled explicitly, not by the OS

- Zero heavy dependencies. No PyTorch, no Transformers. Just Python, C, or C++

This gives us explicit, deterministic control over what's in memory at each step of inference.

Results:

Running Qwen2.5-0.5B-INT8 (~640MB model) with a peak RSS ~210MB and 7.40 tokens/s throughput.

Alpha version is available on GitHub: https://github.com/xuxianghong12/llm.sql

I'm the developer, happy to answer any technical questions about the design and implementation.17:T534,Sharing interactive scientific visualizations is hard. I don’t want my colleagues to have to install specialized software. I just want to email them a link which they can click on and see the full interactive visualization and explore my (potentially large) dataset and results. I want figures to be long-term archivable

Show HN: I built an AI alternative to NoBroker/MagicBricks that never calls you

Free Book "Rust Projects – Write a Redis Clone" Version 2.0.0

WellaBack Posture Corrector

Show HN: Llm.sql – Run a 640MB LLM on SQLite, with 210MB peak RSS and 7.4 tok/s

8087 Emulation on 8086 Systems

2026 Annual C++ Developer Survey "Lite"

Do you need a SoC 2 report?

Meta is revamping its cross-app management system

The FBI faked a field of forensic science. (2015)

Static Types for Dynamic Targets

Health-care AI is here. We don't know if it helps patients

Open grid data has a public benefit

UK Biobank leak: Health details of 500 000 people are offered for sale

Show HN: PhotoSharpener – sharpen and upscale photos in the browser with AI

Face (Sociological Concept)

Microsoft plans voluntary employee buyout for up to 7% of U.S. workforce

Rat Park – ProofOfConcept

Hear your agent suffer through your code

A Hundred Robots Are Running a Bio Lab

Show HN: leaf – a terminal Markdown previewer with a GUI-like experience

Stvor is Signal Protocol E2EE SDK for any app (zero dependencies)

Show HN: Free MVP cost estimator – see what agencies charge vs. a 72-hour sprint

I Was Lonely and Let an App Pick My New Friends. Here’s How It Went

How to be anti-social – a guide to incoherent and isolating social experiences

Building an agentic escrow for software projects

nowhere: an entire website encoded in a URL

Ask HN: Are there any difference using open source tool instead commercial subs?

Show HN: Figpack science viz – Python script → Shareable HTML Bundle in cloud

AI agent designs a complete RISC-V CPU from a 219-word spec sheet in 12 hours

How drones could be used to track plastic litter on Irish beaches

Show HN: Raw Binary Program Analysis Tool