Ask HN: How would you use an AI-powered browser agent?

3•hamzamairaj•4mo ago

I’m exploring the idea of AI powered browser agents tools that can navigate the web like a human: clicking, typing, scraping, filling forms, or even completing multi step workflows. All you do is prompt the task.

If you had such an agent, what would you use it for?

- Automating repetitive tasks? - Gathering and cleaning data? - Handling customer or internal workflows? - Something else entirely?

I’d love to hear how the HN community imagines using this.

Comments

ttoinou•4mo ago

All of this and more. A lot of what I'm doing with the browser could be automated.

I don't understand why this isn't a solved problem already, I tried a few options but none could control the browser properly. For example, I asked Claude Code Chrome extension to look for bookings at specific dates, it would never be able to use the calendar UI widgets to select dates automatically.

hamzamairaj•4mo ago

Thanks for sharing your experience. I've built most parts of the agent already though I haven't tested it with any calendar apps yet. One option I'm working on is enabling vision so the agent can see the browser state via screenshots and interact with visual elements like date pickers.

It also has an orchestrator that can split a larger task into multiple sub agents running in parallel. Browser profiles are supported too. I'm using OpenAI models (gpt-4.1-mini works really well so far) and the project is written in node js.

I plan to open source it once it's more polished but right now I'm especially curious about how people from different backgrounds would want to use such a tool.

ttoinou•4mo ago

Couldn't it rather know the HTML source and JS state ? Like if it was plugged into Chrome DevTools. It might be more straightforward and better inputs for AI to interact with

hamzamairaj•4mo ago

That's already there, by default the agent can see the HTML elements in the current viewport and interact with them properly. It works quite well for most cases but in some special scenarios (Like your calendar app example) we still need vision as a fallback.

USS Preble Used Helios Laser to Zap Four Drones in Expanding Testing

Show HN: Animated beach scene, made with CSS

An update on unredacting select Epstein files – DBC12.pdf liberated

Was going to share my work

Pitchfork: A devilishly good process manager for developers

You Are Here

Why social apps need to become proactive, not reactive

How patient are AI scrapers, anyway? – Random Thoughts

Vouch: A contributor trust management system

I built a terminal monitoring app and custom firmware for a clock with Claude

Tiny C Compiler

Y Combinator Founder Organizes 'March for Billionaires'

Ask HN: Need feedback on the idea I'm working on

OpenClaw Addresses Security Risks

Apple finalizes Gemini / Siri deal

Italy Railways Sabotaged

Emacs-tramp-RPC: high-performance TRAMP back end using MsgPack-RPC

Nintendo Wii Themed Portfolio

"There must be something like the opposite of suicide "

Ask HN: Why doesn't Netflix add a “Theater Mode” that recreates the worst parts?

Show HN: Engineering Perception with Combinatorial Memetics

Show HN: Steam Daily – A Wordle-like daily puzzle game for Steam fans

The Anthropic Hive Mind

Just Started Using AmpCode

LLM as an Engineer vs. a Founder?

Crosstalk inside cells helps pathogens evade drugs, study finds

Show HN: Design system generator (mood to CSS in <1 second)

Show HN: 26/02/26 – 5 songs in a day

Toroidal Logit Bias – Reduce LLM hallucinations 40% with no fine-tuning

Top AI models fail at >96% of tasks