The Problem

I’ve been using BeautifulSoup for sometime. It’s the standard for ease-of-use in Python scraping, but it almost always becomes the performance bottleneck when processing large-scale datasets.

Parsing complex or massive HTML trees in Python typically suffers from high memory allocation costs and the overhead of the Python object model during tree traversal. In my production scraping workloads, the parser was consuming more CPU cycles than the network I/O. Lxml is fast but again uses up a lot of memory when processing large documents and has can cause trouble with malformed HTML.

The Solution

I wanted to keep the API compatibility that makes BS4 great, but eliminates the overhead that slows down high-volume pipelines. It also uses html5ever which That’s why I built WhiskeySour. And yes… I *vibe coded the whole thing*.

WhiskeySour is a drop-in replacement. You should be able to swap from "bs4 import BeautifulSoup" with "from whiskeysour import WhiskeySour" and see immediate speedups. Your workflows that used to take more than 30 mins might take less than 5 mins now.

I have shared the detailed architecture of the library here: https://the-pro.github.io/whiskeySour/architecture/

Here is the benchmark report against bs4 with html.parser: https://the-pro.github.io/whiskeySour/bench-report/

Here is the link to the repo: https://github.com/the-pro/WhiskeySour

Why I’m sharing this

I’m looking for feedback from the community on two fronts:

1. Edge cases: If you have particularly messy or malformed HTML that BS4 handles well, I’d love to know if WhiskeySour encounters any regressions.

2. Benchmarks: If you are running high-volume parsers, I’d appreciate it if you could run a test on your own datasets and share the results.

Survival Is the Only Success

DeepSeek V4 in vLLM: Efficient Long-Context Attention

Show HN: I got tired of boring SQL tutorials, so I built a game

Multi-player agents don't fit in the sandbox

Show HN: Duckville, a persistent-world life SIM where you're a duck

GPT hallucinated a bug in my code, so I 'fixed' it

UK to permanently ban future generations from buying cigarettes

How People Smuggle the Internet Through DNS [video]

Claude Code Tips I Wish I'd Had from Day One

20 Years Ago, I Spent $8 on This. My Life Was Never the Same

TSMC Says ASML's Latest Chipmaking Gear Is Too Pricey to Use

New 21-character nuclear command message observed during April exercise window

Show HN: I built an automatic micropayment-ish system to support the Web

Cloaca (art installation)

Show HN: MultiTable – I built a dashboard so I could vibe-code from my phone

AlphaGo versus Lee Sedol

Show HN: SherifDB, a databe written in Golang under 500 LOC

Jumping into cold water can stop your heart

A Guide to Reducing Cognitive Load

Show HN: I built a free LLMs.txt generator

DrP: Meta's Root Cause Analysis Platform at Scale

Show HN: PlayDrop – A home for vibe coded browser games

Fedora 44 Go/No-Go meeting

Music of the BBC Microcomputer System

Government watchdog urges FAA to address Boeing MAX engine issue

United States of America vs. Matthew David Keirans [pdf]

Surf-CLI – a CLI for AI agents to control Chrome

Radar Laboratory – Interactive Radar Phenomenology

Show HN: WhiskeySour – A 10x faster drop-in replacement for BeautifulSoup

Onyx Pro, a local desktop utility for re-evaluating AI IDEs before paying

Show HN: A games website with a swipe feature like TikTok