frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Training a model to identify AI web content from structure alone

https://arxiv.org/abs/2609.15369
24•jochenmadler•8h ago
Hey HN! We’re Vincent and Jochen from Sitefire (https://sitefire.ai). We have been working together for years, with backgrounds in RL/optimization at Stanford and software engineering from Technical University Munich (TUM).

With Sitefire (YC W26), we help marketing teams get recommended by AI Search (ChatGPT, Google AI Overviews, AI Mode, Claude, etc.). Our software monitors prompts, sees which web pages get cited, and uses these insights to help marketing teams take action, e.g. create YouTube videos or write the right blog posts.

This means we have a commercial stake in AI-generated web content. And for now, high-information, AI-generated content works great to get cited and recommended in AI Search.

But after talking to hundreds of marketing teams, it became clear that everyone despises AI-generated content (“AI slop”). And yet, everyone still wants to leverage AI to create content. So we asked ourselves: what characterizes AI slop? Can we train a model to identify it from human-generated web pages?

Researchers from the University of Maryland and Google DeepMind already asked this question for fiction. Their paper StoryScope (Russell et al., 2026) showed that you can tell AI-written stories from human ones by their structure alone, without looking at the words.

We ported their pipeline to commercial web pages. Using the Wayback Machine, we collected 2,250 blog posts from 268 B2B company websites that were written before ChatGPT existed. For each blog post, five AI models (GPT-5.4, Claude Sonnet 4.6, Gemini 3 Flash, DeepSeek V3.2, Kimi K2.5) wrote their own version.

Instead of looking at the words, we looked at how each post is built. We had an AI model answer 214 questions about every post, e.g. how hard it pushes its own product, whether it backs up its claims with sources, or whether it quotes a named expert. Then we trained a classifier on these answers.

On blog posts it had never seen before, our classifier told AI-generated and human posts apart with 98% accuracy, getting only 19 of 1,740 wrong.

Why does it work so well? Because all five AI models write in a similar shape. Mapping every AI model’s values for these features, we see they cluster together, while the human values sit apart and spread out much more. Of the 1% most unique blog posts in our data set, 149 are human, only 4 are AI.

So what characterizes AI slop? It tells you the same thing three times. The title already promises what you'll get ("How to Cut Onboarding Time in Half"), the intro lays out what's coming, and the ending says it all again. 77% of the AI posts end by repeating their main point, compared to only 12% of the human posts. We call it the tidy, self-announcing blog post.

Still, each AI model has its own accent. We trained a second classifier to tell which of the five AI models wrote a post, or whether a human did. It picks the right author 79% of the time, where random guessing (1 in 6) would get 17%. Almost all of its mistakes are mix-ups between the AI models, not between human and AI.

The cool thing about structural features is that you can't simply reword your way out of it. We had each AI model rewrite its own posts until, on average, 73% of their original 13-word sequences were gone, and the AI slop classifier still worked just as well.

We're building this into Sitefire: our agents get a structural understanding of text, so the posts they write go deeper and vary the way human writing does.

There's a lot we haven't tested yet, like the myriad of humanizer tools, human rewriting, restructuring a post, or prompting an AI model to explicitly avoid these habits. And our human posts are mostly from 2020 to 2022, while the AI posts were generated in August 2026. Structure can't really tell when a human post was written, but it's still not a same-year comparison.

We published the study with all the figures on arXiv: https://arxiv.org/abs/2609.15369. The code is on GitHub: https://github.com/pulse-energy-eu/slopshape

We're pretty sure your own blog isn't AI slop, is it? We built a checker that runs one of your posts through the ten features from the paper, so you can see for yourself (the full report asks for a work email): https://sitefire.ai/slop-checker.

Think you can tell AI slop from human writing? We also made a little game to see if you can keep up with our model, which gets all five rounds right: https://sitefire.ai/spot-the-slop.

Comments

asdff•3h ago
The idea is interesting but in looking at methods and github I feel the tooling leaves me wanting. I mean you are trusting LLMs here to establish, vet, and detect your various thresholds that were then used to train the classifier. I'd rather see this sort of thing done deterministically with actual code vs lossy human english prompts and a dependency on token spend to a single third party (who will probably pull the underlying model used in what a few short years probably) to replicate the results or try and use different training data.
pooploop64•3h ago
Is the goal of this to help AI pick corn kernels out of it's own shit for the purposes of slightly raising the bar on how sloppy the slop is? Or are you trying to trick AIs into eating a higher amount of their own shit than they already are? This feels like a factory built specifically to manufacture pollution.
blackboxdev•3h ago
[flagged]
dang•1h ago
Can you please not post AI-generated or AI-edited comments to HN? It's not allowed here - see https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079.

Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.

jumploops
•
51m ago
Funnily enough, that comment, when passed to the OP's slop detector[0], returns "80% human"

It's very clearly AI-generated, and thus a bit ironic (:

[0]https://sitefire.ai/slop-checker/r/UkRAuqW1y-1T2qhbRQfLt2Jq7...

pooploop64•27m ago
Maybe they ran it through OP's de-sloppifier to see how well it works. And look how well it did! The slop detector doesn't suspect a thing! I can already smell the value this thing is adding to the world.
bryanrasmussen•39m ago
The idea is that AI slop is non-creative boring crap, to really determine if you are able to identify AI slop then it should be determined if you misidentify human slop as AI slop.

Idea 1: Identify the worst most boring human marketing, organizational, bureaucratic texts from the a time before AI was writing it, anonymize this text to make sure there is no reference to current events that can be used to determine that it is not AI. And then see if the AI will say hey, that is not AI slop.

Idea 2: Have people parody AI slop. Can it determine the parody is still not AI slop?

beepbooptheory•9m ago
Do you have an opinion on this point about identifying larger structures as tells instead of at more of a word level? I guess I am not sure here how this follows from what is posted?

Show HN: JevBench, a reproducible benchmark for typed decision models

https://benchmarkheaven.com/jev-models
36•florianstandhar•8h ago•4 comments

Show HN: Training a model to identify AI web content from structure alone

https://arxiv.org/abs/2609.15369
24•jochenmadler•8h ago•8 comments

Show HN: Drop – A rootless Linux sandbox with gVisor support

https://droprun.sh/
147•mixedbit•8h ago•50 comments

Show HN: AI·rete·RAG – a Rete rule engine decides, RAG explains why

https://ai-rete-rag.com/
32•ZaharaHussain•5h ago•2 comments

Show HN: LARP Capital – Prestige as a Service

https://www.larpcapital.us/
2•vknar•40m ago•0 comments

Show HN: NetM8 OS

https://netm8.com/
2•goodknightleo•53m ago•0 comments

Show HN: InstinctFlash – High-Performance Serving Runtime for Robotics Models

https://github.com/General-Instinct/InstinctFlash
17•guanming0717•6h ago•2 comments

Show HN: ToolHub – Hierarchical nav for 30 MCP tools without context bloat

https://github.com/Talos-popcorn/toolhub
2•soidkeo•1h ago•0 comments

Show HN: FreeCoffee – Self-hostable donation tool with crypto support

https://github.com/freecoffee-bio/freecoffee.bio
2•jacksun788•2h ago•2 comments

Show HN: graf (1000x faster graphify in Rust)

https://github.com/ctxrs/graf
5•ripped_britches•5h ago•2 comments

Show HN: Notes on Agentic AI – A text-first guide for practicing engineers

https://github.com/mrsachindixit/agentixit
2•tbaadm•3h ago•0 comments

Show HN: RUUN – Tor Browser with a second, non-Tor mode in the same app

https://gitlab.com/varad_7/ruun
2•opentestudox07•3h ago•0 comments

Show HN: Hack – my Hacker News mobile client now lets you block AI posts

4•busymom0•7h ago•0 comments

Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM

https://github.com/volotat/mini-AGI/
273•volotat•1d ago•72 comments

Show HN: Riftri – copy-on-write Git worktrees for running agents in parallel

https://twitter.com/KinfishT/status/2102093551601684897
2•kinfish•1h ago•0 comments

Show HN: OpenMCP – An open, code-first fork of the Model Context Protocol

https://github.com/enclawed/omcp
2•enclawed•6h ago•0 comments

Show HN: Foremerge – Catch intent conflicts between parallel coding agents

https://github.com/naw103/foremerge
45•naw103•1d ago•15 comments

Show HN: A competition for small neural networks that play strategy games

https://tinybrains.dev
110•codetiger•2d ago•40 comments

Show HN: Lossless-memory – a personal AI memory that never summarizes

https://github.com/aru-labs/lossless-memory
64•aru-labs•1d ago•29 comments

Show HN: Venya lets AI agents use secrets without seeing them

https://github.com/tabith-llc/venya
2•tabith•8h ago•0 comments

Show HN: Radius – A Meetup.com Alternative

https://radius.to/
162•radius89•2d ago•80 comments

Show HN: Vellum, the best diagram editor you'll ever use

https://vellum.blueprintr.io
22•JoshJamesAnthon•1d ago•15 comments

Show HN: Matcha – fresh job postings pulled directly from company career pages

https://matchajobs.co/
2•jpercy•10h ago•3 comments

Show HN: Watch all the AI agents on your machine

https://github.com/markwylde/all-your-agents
5•turblety•10h ago•2 comments

Show HN: An atlas of system designs with interactive architecture diagrams

https://atlas-sysdes.vercel.app/
3•mertkahyaoglu•11h ago•2 comments

Show HN: jevals – replacing LLM judges with typed Jev decisions

https://github.com/openlayer-ai/jevals
45•gbayomi•1d ago•6 comments

Show HN: Sigabrt.dev – cronjob monitor with an SSH TUI

https://sigabrt.dev
81•4815162342•3d ago•35 comments

Show HN: CUA-S1 – A System One Model for Computer Use

https://github.com/trycua/cua
92•frabonacci•3d ago•10 comments

Show HN: Gdocs-me-up: a high-fidelity Google Docs exporter

https://github.com/behdad/gdocs-me-up
20•behdad•1d ago•8 comments

Show HN: Share your AI Setup, Learn from others

https://mysetup.ai/
249•steveybrown•5d ago•139 comments