Hey HN, I built VibeScrape — it takes a website URL and a JSON schema describing the data you want, then analyzes the page, writes real Python code to extract that data, and refines the code until the output is accurate.

While there's lots of tools these days (e.g. Firecrawl) that feed the entire HTML of a webpage to an LLM to extract data from it, this always seemed like a really slow & expensive approach to me.

On the other hand, handwriting web-scraping code seems archaic at this point. This type of code is incredibly tedious to write, and immediately becomes throwaway code once the webpage's layout changes even a little.

VibeScrape aims to automate the process of writing this type of code.

1. Grabs the rendered HTML — the same view a browser sees. 2. Has an LLM extract data from the HTML into your target JSON schema (the “ground truth”). 3. Generates Python scraper code to reproduce that "ground truth" output. 4. Runs and compares results against the ground truth. 5. Refines the code automatically until the outputs match.

I've found that letting the LLM take control of both the code generation and iteration process e2e has worked pretty well for producing working scraper code for many of the websites I've tested it on!

It still has some limitations in terms of handling pagination, captchas, infinite scrolling, etc. Hoping to get some early feedback from the HN community to see if this is a valuable tool. There's a promo code FIRST5 on the site that gets you 5 credits for free, but am happy to give more credits to anyone that reaches out at contact@vibescrape.ai !

Thanks!