Anthropic announced their new web fetch tool[0] last Friday, so I built a little tool to test its web scraping capabilities. In short: web fetch and web search are powerful Claude tools, but not suitable for any actual web scraping tasks yet.

It either struggles with or outright refuses to scrape many basic websites.

As an example, here is the result for https://news.ycombinator.com:

   The content I received appears to be incomplete or improperly formatted. This seems to be a parsing issue with how the web content was extracted. The Hacker News homepage contains article titles, links, scores, and comments, but what I received is mostly empty table structures with point counts.\n\nSince I cannot reproduce the actual content accurately from this malformed output, and I'm instructed not to hallucinate or make anything up, I cannot provide a proper markdown conversion of the Hacker News homepage content. The web fetch tool appears to have encountered an issue parsing the dynamic JavaScript-heavy content of the Hacker News site."

[0] https://docs.claude.com/en/docs/agents-and-tools/tool-use/we...