frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

We handed our lives to closed software. AI is about to show us the cost

3•earlythesis•48m ago•0 comments

Ask HN: What does building hardware with AI look like?

2•mjs06•2h ago•0 comments

Ask HN: Which open source LLM works best locally, and on what hardware?

3•donkan168•2h ago•14 comments

Vercel should fork and maintain Deno

15•gfalcao•1d ago•11 comments

Ask HN: Dropbox privacy policy, cause for concern?

10•herodoturtle•12h ago•2 comments

Ask HN: What are you working on? (November 2026)

303•david927•6d ago•1088 comments

Tell HN: Google Search to start locking down search behind a login wall

44•exploraz•1d ago•29 comments

Ask HN: What is the state of self-hosting email in 2026?

44•0o777•1d ago•33 comments

Anthropic, can you please make up your mind with your models?

5•xlayn•14h ago•7 comments

Tell HN: Cloudflare's abuse report form is a blank page

17•basilikum•1d ago•6 comments

Ask HN: What AI agent infrastructure are you using in 2026?

3•aurumagent•17h ago•2 comments

ChatGPT Desktop forwards your SSH agent to remote machines

221•DenGorbachev•6d ago•80 comments

Ask HN: What are the most useful, concrete AI skills to independently learn?

87•shnock•1d ago•51 comments

Ask HN: Best books you read in 2026?

12•013•19h ago•5 comments

Ask HN: Do you still enjoy coding in C++?

66•hasley•6d ago•90 comments

Ask HN: How did you convince family and friends to join your XMPP/Matrix server?

102•mtlynch•1w ago•98 comments

Using LLMs outside of reflection is an anti-pattern

4•jcarmackdaddy•1d ago•0 comments

Tell HN: Major Microsoft Outlook/Exchange Online/Teams Outage in the Americas

14•wmchen•2d ago•2 comments

Ask HN: What use are the non-major cryptocurrencies?

4•otohp•1d ago•5 comments

Ask HN: OpenAI Astra post-launch slump?

4•iJohnDoe•1d ago•6 comments

Ask HN: Do you see a human written book more valuable than an AI written one?

10•znano•2d ago•23 comments

Ask HN: Why use Jev/decision models at all?

10•niceguy1827•3d ago•9 comments

You've reached the end!

Open in hackernews

Ask HN: Which open source LLM works best locally, and on what hardware?

3•donkan168•2h ago

Comments

spottedmarley•2h ago
best for what?
donkan168•2h ago
fair enough, I mostly meant for reasoning and coding, but feel free to share whatever you use and its strengths
spottedmarley•1h ago
I use local models mostly for coding, system admin tasks (linux), and occasionally for things like image/video/music gen. I run a dual-node cluster of Dell Pro Max GB10s although 90% of the time I run 1 model per node. My favorite primary models at the moment are Qwen3.8:27b-Flash-Next FP8 and Qwen3.8:27b-gsq-iq3s. Flash next is the best model overall but it is very (overly) zealous and can get itself off on tiresome tangential tasks mid-turn. It's just too eager to let it run alone on longer-horizon coding tasks, but if you have the time and patience to monitor it and keep it on task it's output is excellent and it's also very fast for the model size. The iq3s variant is not Flash-Next (not using the Qwen4 architecture) but is still just about as good, especially for the 3bit quant, much more calm and manageable than Flash-Next. I tend to use iq3s most of the time and Flash-Next for tougher tasks when I can afford to be involved in the process and keep it on track.

Qwen4 should be dropping soon, hopefully another 27b that is less aggressively eager than 3.8-flash-next

donkan168•1h ago
sounds interesting, but are you saying that Qwen3.8:27b-Flash tends to skip instructions to do more than what the task requires? What have you tried to control it and make it more autonomous? I’m asking because I don’t want to constantly focus on the task execution, it’s exhausting to keep reading the reasoning of what a model is doing. but I think I'll try with Qwen3.8:27b-Flash
spottedmarley•45m ago
No, not at all, pretty much the opposite. It wants to do everything you ask for and will usually get there on its own but it will spend large chunks of time diagnosing things that it finds along the way that may or may not be what you want it focusing on. Compare it to a frontier model like Claude in Claude Code that will mention things it finds but continue on the main task at hand and always stays on track, only diverting to fix something that might be impeding its ability to finish the task, Flash-Next is more like an ADHD model that sees shiny things and can't help but chase after them Lol. It's really not as bad as Im probably making it sound, after all it is my #1 choice for many tasks, I just have to be in the mood to keep an eye on it and interrupt now and then. 3.8-iq3s is perfectly good for most things though and requires less oversight. I know many people have complained about the same thing related to Flash-Next so Im really hoping Qwen4 drops with a much less eager 'personality'.
donkan168•27m ago
ahh, I understand now, I hope Qwen4 launches in a couple of months, Until then, I might try with 3.8-Flash, Thank you for your insights, any suggestions for the installation?
tourmalinetaco•2h ago
You can try: https://runitlocal.io/

Saw it on here a few days ago, you put in your GPUs and it helps give you an idea of what you can run. Although, the specific model weight is dramatically altered by use case. Do you want code creation? Creative writing? Assistant workloads? I would suggest doing some Googles/AI searches on “[Model returned by RunItLocal] for [use case]”.

donkan168•2h ago
oh thank you, I'll review it, good resource thank you again
donkan168•2h ago
thanks, that site is useful, do you run a model on your own machine? if yes, what hardware and model are you using, and is it good enough for coding?
tourmalinetaco•1h ago
I don’t run my own model, unfortunately I didn’t jump into getting a better GPU when I had the chance so I’m waiting out the market. In my specific use-case I use DeepSeek for programming since it’s the best bang for buck.
jpsanders•2h ago
if you have a GTX 1060 6GB this might help you: "https://github.com/jpsanders/pascal-llm-stack"
donkan168•2h ago
thank you! I'm researching some information to understand the best setup (cost vs. performance and reasoning) to build a desktop computer for running an LLM, specifically for general tasks and reasoning in coding
dlcarrier•1h ago
Whatever model you decide on, look for an uncensored version. They are more likely to do "unsafe" things, like tell you what happened in Tiananmen Square in 1989, but they'll also accept what you ask or say at face value, without being argumentative or deferring to professionals. They generally run better and don't get stuck in loops as easily, too.
donkan168•1h ago
What do you recommend?