*TL;DR: You can train your own 27B model on a 3090 with 24GB VRAM, though you would have to go through hundreds of billions of tokens for it to be good. I am working on conversion of existing models. Check repos for code and the paper if you are interested in more. I have a 40.4GB Qwen Flash* N*ext GGUF*

Hi, I do AI research. I have been working on an optimizer that doesn't need the \~16 bytes of VRAM per weight to train a model that AdamW does. I focused on training ternary networks bcs BitNet (Ma et al.) shows they match full precision quality in large scales. The problem is BitNet also keeps the full precision weights during training which means you still need to use the same amount of memory. I got this down from 16 bytes to 1 byte or even less per weight.

For you, this means, you can train your own 10-20B parameter models on your consumer GPUs. I fitted 27B model training on a single 24GB 3090 and a 23B model on a 16GB GPU. The trained model costs \~1.6 bits per weight (\~6GB at 27B), 10x smaller than 16bit, and 5x smaller than Q8.

I have also been working on converting existing models and training them. This is harder and I am still working on it but I have some promising results. I quantized and trained a Qwen3.8 27B though the file I published below has adapters which are about 2GBs, pushing the file size to 8GB. I am also making a no-adapter version right now. It scores GSM8K 0.895, the same as full precision, HumanEval 0.84 vs 0.93 and runs on a 12 GB GPU at \~2*3* tok/s on my crippled 12GB laptop 4080, though this number can heavily be improved with custom kernels, engineering, etc. I will leave it to you guys to help me with that too.

I also quantized and trained Qwen Flash Next down to 40.4 GBs including its n-gram table (at 8GB). It largely retains the skills of the network (scores equivalent on GSM8K benchmark, a little behind on HumanEval, I unfortunately don't have compute for the larger benchmarks such as TerminalBench2.1 but perhaps some of you can help me out) but forgets facts. It mostly works fine if you give it web search as far as my experience goes but this is not strictly measured.

I am still doing work on this and I will keep working on it, perhaps even for many years. The code is open for non-commercial use. Check out:

The core repo for training: https://github.com/oguzeldereli/ttf-ternary

The repo for conversion: https://github.com/oguzeldereli/ttf-conversion

Repo for running models trained by ttf: https://github.com/oguzeldereli/ttf-home

GGUF for a converted Qwen 3.8 27B with adapters (around 8 GB): https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF

GGUF for a converted Qwen 3.8 Flash Next (around 40.4GB): https://huggingface.co/oguzeldereli/ttf-Qwen3.8-Flash-Next-G...

GGUF for a converted Qwen 3.8 27B without adapters (around 6 GB): Currently in conversion and will drop soon.

I will convert a Qwen 4 27B instance when it drops.

Full paper: https://github.com/oguzeldereli/ttf-ternary/blob/main/paper/...

Also, I would appreciate github stars if you like the project.

Show HN: A linear model is an embedding table: using VW as transformer memory

Show HN: VoxaDub – Open-source AI video dubbing with speaker-aware voices

Sure mate, buy me a bloody coffee

Show HN: Cosh – An open-source AI agent orchestrator

PressPril – Unlock Everyday Vitality and Wellness

Show HN: AI Hustle Team – an AI on its own computer that does your daily work

Nanocode – a hackable coding agent in one Python file

Show HN: MbSCADA – Open-Source Modbus Poll and Scada in Rust

Show HN: Eigin. Local-first and no-shell personal AI for iOS

Small-Grid Wraparound Cellular Automata

Show HN: Chatty – Open-source AI support engine with WooCommerce sync

Show HN: Commune, one feed for newsletters on Beehiiv, Kit, Ghost, and more

Show HN: A 15 MB self-hosted Netflix and Spotify for your media server

ZX Spectrum in SQL and Daley Thompson's Decathlon

Building a Bluesky account typeahead service

Isar Aerospace Could Launch Third Spectrum Rocket as Early as 26 November

Legocam: My Lego webcam from 2000, working on a modern Mac

Selling SaaS in India

Show HN: GameRAGKit, lore-grounded NPC dialogue in C#

Show HN: I made an old-school IRC-style chat for talking about movies and series

Show HN: Tokspur a 3x faster open source inference engine for Macs

Nuance Labs (Full Duplex AI Avatar)

Show HN: Grai – A deterministic cadence gate for AI drafts

Ask HN: What future you see for software engineering?

Anthropic launches roadshow for proposed initial public offering

Show HN: You can now train your own LLMs with less memory, on your consumer GPUs

Protect your Symfony login against brute force with login_throttling

Anthropic's nearly $70B IPO plan briefly circulated to banks

Free GEO Content optimizer tool by Shoden Global

Minecraft: Java Edition 26.3 rewritten from scratch in Rust

Show HN: Commune, one feed for newsletters on Beehiiv, Kit, Ghost, and more

Comments

Show HN: A linear model is an embedding table: using VW as transformer memory

Show HN: VoxaDub – Open-source AI video dubbing with speaker-aware voices

Sure mate, buy me a bloody coffee

Show HN: Cosh – An open-source AI agent orchestrator

PressPril – Unlock Everyday Vitality and Wellness

Show HN: AI Hustle Team – an AI on its own computer that does your daily work

Nanocode – a hackable coding agent in one Python file

Show HN: MbSCADA – Open-Source Modbus Poll and Scada in Rust

Show HN: Eigin. Local-first and no-shell personal AI for iOS

Small-Grid Wraparound Cellular Automata

Show HN: Chatty – Open-source AI support engine with WooCommerce sync

Show HN: Commune, one feed for newsletters on Beehiiv, Kit, Ghost, and more

Show HN: A 15 MB self-hosted Netflix and Spotify for your media server

ZX Spectrum in SQL and Daley Thompson's Decathlon

Building a Bluesky account typeahead service

Isar Aerospace Could Launch Third Spectrum Rocket as Early as 26 November

Legocam: My Lego webcam from 2000, working on a modern Mac

Selling SaaS in India

Show HN: GameRAGKit, lore-grounded NPC dialogue in C#

Show HN: I made an old-school IRC-style chat for talking about movies and series

Show HN: Tokspur a 3x faster open source inference engine for Macs

Nuance Labs (Full Duplex AI Avatar)

Show HN: Grai – A deterministic cadence gate for AI drafts

Ask HN: What future you see for software engineering?

Anthropic launches roadshow for proposed initial public offering

Show HN: You can now train your own LLMs with less memory, on your consumer GPUs

Protect your Symfony login against brute force with login_throttling

Anthropic's nearly $70B IPO plan briefly circulated to banks

Free GEO Content optimizer tool by Shoden Global

Minecraft: Java Edition 26.3 rewritten from scratch in Rust