I built an open-source reverse-engineering AI agent, that can write requests-based(no browser) automation/scraper scripts on its own, all we have to do is, to normally browse the website once.
This makes it a lot easier to create and maintain the scripts. This is not a completely autonomous agent, you can pause, talk, guide the agent on its RE path.
Despite the limitations of this approach, up to 60%-90% of websites can be automated/scraped with just raw requests and TLS-spoofing(links to sources provided in the blog)
The tool successfully ran against popular websites like, bookmyshow, makemytrip. And to test it in the open, it was able to produce scraper solutions for 2 problems posted on r/webscraping in a single try.
As the scripts provided by the tool are requests-based, you can expect a 10-100x speed increase, and reduction of 5-7x memory footprint in your existing browser-based automations and scraper workflows.
Github: https://github.com/StoneSteel27/AutomatiQ Blog: https://shiny-sushi-116.notion.site/automatiq Demo video: https://www.youtube.com/watch?v=OfMvAQP5pkM