frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Pulpie – Models for Cleaning the Web

https://usefeyn.com/blog/pulpie-pareto-optimal-models-for-cleaning-the-web/
34•snyy•2h ago
Hey HN, I'm Shreyash, founder of Feyn. We built Pulpie, a family of Pareto optimal models for cleaning the web. Pulpie strips boilerplate (ads, footers, sidebars) from raw HTML and returns just the main content as HTML or Markdown.

We match SOTA extraction quality while being 20x cheaper. Cleaning 1 billion webpages costs $7,900 with Pulpie versus $159,000 with Dripper, the current leading extractor.

The gains come from architecture. Today's leading extractors are decoders that generate output one token at a time. Each step reads the full model from memory to produce a single token. Conversely, Pulpie models are encoders. They run one forward pass over the full input HTML and label each block as boilerplate or content. As a result, Pulpie is compute-bound while decoders are memory-bound. Cheaper GPUs have relatively more compute than memory bandwidth. This makes Pulpie easy to run optimally.

Here's Pulpie and Dripper cleaning the same pages side by side: https://www.youtube.com/watch?v=ibd-tIiQECo. You can try a side-by-side comparison yourself: https://huggingface.co/spaces/feyninc/pulpie

Our motivation for Pulpie came from building a deep research harness. Every search API returns noisy content that contains ads, nav elements, and sidebars. In one instance, an ad for "Gemini on Pixel" slipped into our search results, got passed into LLM context, and ended up in the final answer served to the user. Pretty embarrassing moment for us but it helped us realize how bad data kills model intelligence. We built Pulpie to get clean data for cheap.

All models are open source on Hugging Face. You can read about our training process and how to use Pulpie here: https://usefeyn.com/blog/pulpie-pareto-optimal-models-for-cl...

Happy to answer any questions!

Comments

lnenad•59m ago
Very nice! Thank you for building this.
rambambram•57m ago
Leeches.
kocamaz•21m ago
It's good looking, and I liked it. The trial page accessed from the hugging face website is a very inefficient experience when I use Mozilla and the dark theme, FYI.
esafak•17m ago
Why does the 'Quality vs Cost of Web Content Extraction' chart not have zero cost at the origin? Up to the right does not have to mean better; we can read.
snyy•7m ago
Funnily enough, that wasn't my first choice either. I A/B tested it with a small group and people understood "up and to the right is better" faster.
rishav2580•14m ago
ongrats on the release! The architectural insight to switch from a bandwidth-bound decoder (token-by-token generation) to a compute-bound encoder (single forward pass over 8k chunks) is brilliant—the 20x speedup and cost drop from $159k to $7.9k per billion pages is massive for web-scale pipelines.

As someone building AI developer utilities and document tools, I have two quick technical questions:

How well does the <|sep|> block-marker architecture handle heavily obfuscated HTML or adversarial SEO spam where boilerplate is styled to look like semantic body text? Have you tested running pulpie-orange-small (210M) quantized (e.g., INT8/FP8) on consumer edge GPUs or CPU-only setups for local RAG pipelines? Amazing work on open-sourcing the teacher and distilled weights on Hugging Face!

Show HN: Pulpie – Models for Cleaning the Web

https://usefeyn.com/blog/pulpie-pareto-optimal-models-for-cleaning-the-web/
34•snyy•2h ago•6 comments

Show HN: I Built LangGraph for Swift

https://github.com/christopherkarani/Swarm
17•christkarani•3h ago•3 comments

Show HN:I Made a Mac Native Excalidraw (Open Sourced)

https://github.com/zpphxd/whitespace
4•z_phxd•1h ago•3 comments

Show HN: Silicon Physics Sandbox

https://silicon.bergkv.ist/
5•bergkvist•1h ago•0 comments

Show HN: Nook – a sticky note on the edge of your monitor

https://nook.nostock.studio/
2•phinoda•1h ago•1 comments

Show HN: Cona – design your real room in 3D, in the browser

https://cona.design
3•Losenok•1h ago•0 comments

Show HN: I build a screenshot Mac app for Pi

https://pilens.dev/
2•nava•1h ago•1 comments

Show HN: Char-width – final answer to measure terminal text width

https://www.npmjs.com/package/char-width
2•dawson0•1h ago•0 comments

Show HN: A Wordle game that insults you as you play

https://smugexe.trydoff.dev
2•shreyasjk•1h ago•0 comments

Show HN: A website that shows every pro triathlete's swim, bike, and run gear

https://racekit.pro
2•mpc75•2h ago•2 comments

Show HN

https://vorcigernix.github.io/rohrpost/
2•vorcigernix•2h ago•2 comments

Show HN: Voltius – open-source SSH/SFTP client, built in Rust

https://voltius.app
4•kipavy•2h ago•0 comments

Show HN: Scan your AI agents for dangerous capabilities

https://github.com/makerchecker/MakerChecker
38•smashini•5h ago•19 comments

Show HN: Built a Website for People Who Miss the Old Web

https://www.dailicle.com/
2•lucky-solanki•2h ago•0 comments

Show HN: Mazzap, a Level 4 Open Source Digital Twin Engine

https://github.com/zymazza/mazzap
3•zymazza123•2h ago•1 comments

Show HN: Paint the Earth on a live, interactive globe (collaborative art.)

https://earth.tattoo
17•earth-tattoo•7h ago•9 comments

Show HN: Causari – Content-addressable ledger for AI agent code edits

https://causari.dev/#quickstart
3•CroviaTrust•2h ago•0 comments

Show HN: Tetris where every pixel is another Tetris (23,040 games in GLSL)

https://thiagolira.blot.im/_projects/fractetris/index.html
2•farcaster•3h ago•0 comments

Show HN: Nectar, a Rust-like React that compiles to WebAssembly

https://buildnectar.com
3•blakeburnette•3h ago•0 comments

Show HN: Keyhole – Share secrets with your AI agents

https://keyhole.maferland.com
2•maferland•3h ago•1 comments

Show HN: Agent Draw: An agent draws while you talk, built on TLDraw

https://techstackups.com/articles/tldraw-agent-draw/
2•jameswhitford•3h ago•0 comments

Show HN: Xalgorix – Autonomous AI Pentesting Agent

https://github.com/xalgord/xalgorix
4•xalgord•3h ago•0 comments

Show HN: Homegames. An open-source game platform I've been making for 8 years

https://homegames.io
214•homegamesjoseph•20h ago•52 comments

Show HN: AI harness for C/C++ with GDB, sanitizers, perf and compile tools

https://byteask.ai/
3•anirudhak47•3h ago•2 comments

Show HN: Query Companion (QCP) for your database

https://v0-qcp.vercel.app/
3•sjashwin•3h ago•0 comments

Show HN: Osint tool that finds exposed files on domains

https://search.cerast-intelligence.com/
55•PatchRequest•21h ago•19 comments

Show HN: BloomDisk – free open-source alternative to paid Mac disk analyzers

https://github.com/Nabilhassan12345/BloomDisk
3•nabilhassan•3h ago•1 comments

Show HN: Tool for Auditing Embedded Linux Hosts

https://github.com/nstarke/embedded_linux_audit
3•bootbloopers•3h ago•0 comments

Show HN: Messageboard – A simple API-first dashboard for human readable messages

https://messageboard.tech/
2•dchanco•3h ago•0 comments

Show HN: Otaku – one terminal client for Ollama, LM Studio, and MLX

https://github.com/enclavum/otaku
2•enclavum5•3h ago•0 comments