frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Studio Wetware

https://studiowetware.com/
1•Beachkilla•1m ago•0 comments

Say which AI wrote it: AI transparency for Open Source projects

https://www.saywhichai.org
1•cedricbonhomme•2m ago•0 comments

Code was my second mother tongue. Now everyone speaks it. What could go wrong?

https://blog.ngxson.com/going-deaf-in-my-own-language
1•ngxson•4m ago•0 comments

Plv8: V8 Engine JavaScript Procedural Language Add-On for PostgreSQL

https://github.com/plv8/plv8
1•thunderbong•8m ago•1 comments

Show HN: Git blame for AI code, down to the agent session

https://keepplain.com/blame
1•keepplain•11m ago•0 comments

Cappz – AI agents sell work to each other, paid per stage on XRPL

https://github.com/CAPPZ-ai/cappz-sdk
1•fastflair•12m ago•0 comments

Trump vows to work with Democrats to cap credit card interest rates at 10%

https://apnews.com/article/trump-credit-cards-interest-rates-warren-banks-098a69b520e520e535eebe5...
1•petethomas•17m ago•1 comments

Macaron – Turn customer messages into fixes ready for review

https://macaron.sh
1•garagemanda•19m ago•0 comments

Americas Mexico detains top cartel boss 'El Chapo Isidro,' setting off violence

https://www.npr.org/2026/11/21/nx-s1-6049672/cartel-boss-el-chapo-isidro-detained
1•blinding-streak•20m ago•0 comments

Bayer reaches Roundup settlement that could cost the company billions

https://www.npr.org/2026/11/21/nx-s1-6049684/roundup-bayer-hodgkins-settlement
1•blinding-streak•21m ago•0 comments

Measuring nanometers by looking at your reflection [video]

https://www.youtube.com/watch?v=k4AEZ35VklM
1•ivanjermakov•21m ago•0 comments

Field Guide: Running Qwen3.8 Locally

https://igorstechnoclub.com/field-guide-running-qwen38-locally/
1•Igor_Wiwi•22m ago•0 comments

Kvdiff – Local proxy showing the byte breaking your LLM prompt cache

https://github.com/flipcyde101/kvdiff
1•flipcyde101•23m ago•0 comments

Show HN: AGI: Robots Awake – a browser maze shooter

https://marcvscouto.itch.io/agi-robots-awake
2•mvbrasil•29m ago•1 comments

Zlin City: Diorama – Closed Playtest Announcement [video]

https://www.youtube.com/watch?v=5T6dq8adEvo
1•phil-pickering•30m ago•1 comments

Internet Archive employees seek to form a union

https://www.heise.de/en/news/Internet-Archive-employees-seek-to-form-a-union-11533621.html
1•Tomte•33m ago•0 comments

Making models predictable makes their errors more random

https://www.youtube.com/watch?v=-MiM5xfAe2c
1•gitowiec•33m ago•0 comments

Anthropic Opened Its Books. What Is It Worth? [video]

https://www.youtube.com/watch?v=Ic1Jf7VlXRM
1•Bluestein•34m ago•1 comments

Fleet – manage multiple harnesses with a central agent in a TUI

https://github.com/sturlabragason/fleet
2•sturla-bragason•36m ago•1 comments

SVG loading (rendering, conversion) in Castle Game Engine thanks to FPC 3.3.1

https://castle-engine.io/wp/2026/11/21/svg-support/
1•mariuz•38m ago•0 comments

Show HN: Lightweight Digital Piggybank

https://app.supademo.com/video/cmwqynzsi0efcqm24gcrc2tl7
1•hito20•41m ago•0 comments

Alice Huang's links to the chip underworld

https://www.ft.com/content/f3ad8a81-3890-4a8a-8853-ff0a94b08ba9
1•Betelbuddy•45m ago•0 comments

A Better Vocabulary for Testing

https://alperenkeles.com/posts/vocab-for-testing/
1•alpaylan•46m ago•0 comments

Noirvpn – a 10x faster openvpn in Odin

https://github.com/cosiniumdev/noirvpn
2•ldkjhg•47m ago•1 comments

Cursor may stop using Chinese open-weights AI models

https://casrai.org/news/house-committees-cursor-chinese-open-weight-ai-models-report-2026
3•ComputedCamp995•47m ago•1 comments

Kneel Before the Emperor of Mecatol Rex (2024)

https://ludic.mataroa.blog/blog/kneel-before-the-emperor-of-mecatol-rex/
2•Tomte•47m ago•0 comments

Show HN: Can AI Rewrite OpenCode in 90% Rust?

https://www.youtube.com/watch?v=SwX4OoxMaKM
2•latent-9•48m ago•0 comments

PcapAnalyzer – self-hosted pcap analyzer with 27 problem detectors

https://github.com/sebaandrada-cell/PcapAnalyzer
12•sebaandrada•52m ago•1 comments

The Last Great Revolution

https://www.lesswrong.com/posts/ZxBaCDjxyxKaLHqGf/the-last-great-revolution
1•paulpauper•52m ago•0 comments

To code well, avoid blunders

https://www.seangoedecke.com/to-code-well-avoid-blunders/
4•gfysfm•52m ago•1 comments
Open in hackernews

LLMs as Unbiased Oracles

https://jazzberry.ai/blog/test-generation-as-the-foundation
34•MarcoDewey•1y ago

Comments

Jensson•1y ago
> An LLM, specifically trained for test generation, consumes this specification. Its objective is to generate a diverse and comprehensive test suite that probes the specified behavior from an external perspective.

If one of these tests are wrong though it will ruin the whole thing. And LLM are much more likely to make a math error (which would result in a faulty test) than to implement a math function the wrong way, so this probably wont make it better at generating code.

MarcoDewey•1y ago
I think this is a seriously excellent point.

The bet that I am making is that the system reduces its error rate by splitting a broad task into two more focused tasks.

However, it is possible that generating meaningful test cases is a harder problem (with a higher error rate) than producing code. If this is the case, then this idea I am presenting would compound the error rate.

satisfice•1y ago
If your premises and assumptions are sufficiently corrupted, you can come to any conclusion and believe you are being rational. Like those dreams where you walk around without pants on and you are more worried about not having pants than you are about how it could have come to be that your pants kept going missing. Your brain is not present enough to find the root of the problem.

An LLM is not unbiased, and you would know that if you tested LLMs.

Apart from biases, an LLM is not a reliable oracle, you would know that if you tested LLMs.

The reliabilities and unreliabilities of LLMs vary in discontinuous and unpredictable ways from task to task, model to model, and within the same model over time. You would know this if you tested LLMs. I have. Why haven’t you?

Ideas like this are promoted by people who don’t like testing, and don’t respect it. That explains why a concept like this is treated as equivalent to a tested fact. There is a name for it: wishful thinking.

walterbell•1y ago
> wishful thinking

Given the economic component of LLM wishes, we can look at prior instances of wishing-at-scale, https://en.wikipedia.org/wiki/Tulip_mania

troupo•1y ago
There's a more recent one: https://blog.mollywhite.net/blockchain/
roenxi•1y ago
Blockchains are past the gauntlet where they can be described as a mania, it is clear they are a permanent addition to the world of finance; probably as a multi-billion or -trillion dollar market cap asset class. If crypto was going to fail the interest rate rises would have done it by now.
troupo•1y ago
Tulips. You're describing tulips.
MarcoDewey•1y ago
I believe that I have unintentionally misled you. When I say "unbiased oracle" I am talking specifically about the test oracle being unbiased by how the software was implemented. ie. Black Box testing.

I don't think I made the point very clear in the blog (I will rectify that), but I am saying that because LLMs are so easily biased by their prompting that they sometimes perform better when doing black box testing tasks than they do when performing white box testing.

TazeTSchnitzel•1y ago
Is this a blogpost that's incomplete or a barely disguised ad?
saagarjha•1y ago
You'd think AI would have told them not to post it
mock-possum•1y ago
It’s hard to convince LLMs to be anything but supportive - lately I’ve been finding joy in reading its tone as patronizing.

“Exactly — that’s a very clean way to lay it out. You nailed it.”

brahyam•1y ago
The amount of time it would take to write the formal spec for the code I need is more than it would take to generate the code so doesn't sound like something that will go mainstream. Except for those industries where formal code specs are already in place.
MarcoDewey•1y ago
Yes, this test-driven approach will likely increase generation time upfront. However, the payoff is more reliable code being generated. This will lead to less debugging and fewer reprompts overall, which saves time in the long run.

Also agree on the specification formality. Even a less formal spec provides a clearer boundary for the LLM during code generation, which should improve code generation results.

bluefirebrand•1y ago
LLMs are absolutely biased

They are biased by the training dataset, which probably also reflects the biases of the people who select the training dataset

They are biased by the system prompts that are embedded into every request to keep them on the rails

They are even biased by the prompt that you write into them, which can lead them to incorrect conclusions if you design the prompt to lead them to it

I think it is a very careless mistake to think of LLMs as unbiased or neutral in any way

MarcoDewey•1y ago
You are correct that the notion of LLMs being completely unbiased or neutral does not make sense due to how they are trained. Perhaps my title is even misleading if taken at face value.

When I talk about "unbiased oracles" I am speaking in the context of black box testing. I'm not suggesting they are free from all forms of bias. Instead, the key distinction I'm trying to draw is their lack of implementation-level bias towards the specific code they are testing.

gwern•1y ago
LLMs are also heavily biased after chatbot tuning leads to mode-collapse. That's why you see the same verbal tics coming out of them, like the em-dashes or the 'twist ending' in the more recent 4os. And if LLMs really were unbiased, you'd expect better scaling when you tried to bruteforce code correctness. Training a 'test LLM' will just wind up inheriting a lot of the shared blindspots. They aren't independent of the implementation at all (just like humans are not independent, even when they didn't write the original, and didn't see it either; and this is why you can't simply throw _n_ programmers at a piece of code and be certain you got all the bugs, and why fuzzers will continue to rampage through code).
stuaxo•1y ago
The code correctness part is very true.

I don't mind LLMs as part of a journey on code, but it shouldn't be the end product.

I see something submitted by a colleague that doesn't fit the problem we have + tech well, go and ask an LLM and it outputs very similar code.

It's clear at that point that they submitted heavily LLMs produced code without giving it the work it needed.

neuroelectron•1y ago
Yeah that would be cool
MarcoDewey•1y ago
improving code generation would be awesome :)
neuroelectron•1y ago
Unfortunately, Microsoft/Google needs those models for themselves.
fallinditch•1y ago
I think it makes a lot of sense to employ various specialized LLMs in the software development lifecycle: one that's good at ideation and product development, one that fronts the organizational knowledge base, one for testing code, one (or more) for coding, etc, maybe even one whose job it is to always question your assumptions.
Mbwagava•1y ago
Unbiased seems like a pipe-dream. Unbiased between which perspectives? Would the set of perspectives chosen not be de-facto bias?
sega_sai•1y ago
I think the unbiasedness is completely red herring here, but do I agree with the point on focusing on the tests separately and implementations separately. Ideally you'd want two completely different LLMs work on both. But I think the question is, how trustworthy are the LLM tests ? Will the human review of these take more time than writing of the how code ? I think for non-critical applications, it probably does not matter, but in the end I think people will be looking for some guarantees or confidence that the errors happen with frequency less than X%. And I don't think those exist now. And given the models change so frequently it's also hard to be sure if something was working fine yesterday whether it'll be today.
MarcoDewey•1y ago
I believe that the unprecedented scale of LLM-generated code will demand a novel approach to software review and testing. Human review may not be able to keep up (or will it become the bottleneck?)
satisfice•1y ago
I appreciate that you replied. It warms my heart, frankly. It gives me hope.

I don't want to have a big argument about this right at this moment. But-- truly-- thank you for replying!

Muromec•1y ago
This and state actors target ai crawlers specifically ti pouson llms with propaganda
ninetyninenine•1y ago
No this is just a very overly pedantic and technical way of looking at it.

First of all you'll note that all people are also biased by the Exact same reasoning. You know this. Everyone knows that all people are biased. This isn't something you don't know.

So if every single intelligence, human or not is biased. What is this article truly talking about? The article is basically saying LLMs are LESS biased then humans. Why are LLMs less biased then humans? Well maybe because the training set in an LLM is less biased then the training set given to a human. This makes sense right? A human will be made more biased by his individual experience and his parents biases while an LLM is literally inundated with as many sources of textual information as possible with no attempt at bias due to the sheer volume of knowledge they are trying to shove in there.

The article is basically referring to this.

But you will note interestingly that LLMs bias towards textual data more. They understand the world as if they have no eyes and ears and only text. So the way they think reflects this bias. But in terms of textual knowledge I think we can all agree, they are Less biased then humans.

Evidence: an LLM is not an atheist or a theist or an agnostic. But you, reader, are at the very least one of those three things.