Ask HN: Local LLM agents on Jetson/RPi without a heavy runtime

3•takuya_h•6mo ago

I'd like to design agent flows on my Mac and ship a single tiny binary to Jetson/RPi. LLMs run on-device (no cloud). If you've tried this, what constraint hit first? Pointers to tools or write-ups you liked are welcome.

Comments

takuya_h•6mo ago

Seen Flowise/Dify/Node-RED; specifically curious about compiled-to-binary patterns for on-device LLMs.

giantg2•6mo ago

I don't have much experience but I played around with a USB Coral TPU. I did not try it with an LLM, but with used some image recognition stuff. Seemed OK, but wasn't as intense as an LLM would be. You'd probably have better luck with the Jetson you mentioned, or with one of Coral's more powerful options.

takuya_h•6mo ago

Thanks!

Ask HN: Non AI-obsessed tech forums

Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

Ask HN: Ideas for small ways to make the world a better place

Ask HN: 10 months since the Llama-4 release: what happened to Meta AI?

AI Regex Scientist: A self-improving regex solver

Ask HN: Who wants to be hired? (February 2026)

Ask HN: Any International Job Boards for International Workers?

Ask HN: Who is hiring? (February 2026)

Ask HN: Why LLM providers sell access instead of consulting services?

Tell HN: Another round of Zendesk email spam

Ask HN: What is the most complicated Algorithm you came up with yourself?

Ask HN: Is Connecting via SSH Risky?

Ask HN: Has your whole engineering team gone big into AI coding? How's it going?

Ask HN: How does ChatGPT decide which websites to recommend?

Ask HN: Is it just me or are most businesses insane?

Ask HN: Mem0 stores memories, but doesn't learn user patterns

Ask HN: Anyone Seeing YT ads related to chats on ChatGPT?

Ask HN: Does global decoupling from the USA signal comeback of the desktop app?

Ask HN: Is there anyone here who still uses slide rules?

Kernighan on Programming

We built a serverless GPU inference platform with predictable latency

Ask HN: How Did You Validate?

Ask HN: Cheap laptop for Linux without GUI (for writing)

Test management tools for automation heavy teams

Ask HN: Have you been fired because of AI?

Ask HN: Does a good "read it later" app exist?

Ask HN: OpenClaw users, what is your token spend?

Ask HN: Anyone have a "sovereign" solution for phone calls?

Ask HN: Has anybody moved their local community off of Facebook groups?

How do you deal with SEO nowadays?