I built and open-sourced Carve to handle repetitive work in windows you already use.
Choose the window in any Mac app where you want it to work, press Option-D, and watch it do what you want. For example, in the demo I show an example CRM and Carve entering contact details and drafting a follow up. It also has the ability to annotate and explain items on the screen.
This runs with your own API keys, and can even support OpenRouter, though OpenAI is the best tested. Experimental adapters also for Bedrock, Azure. No reason it couldn't expand beyond experimental support to support more local models.
I built this because I found that Claude and Codex were rather clunky when using computer use, I like having a sleek window that just pops up and shows me what it's doing. Also it has the 3 similar modes (approve plan, step by step manual approval, or act directly mode).
Local currently covers browser actions via vision models, computer use support is required for operating other native mac apps. This is an ongoing area of work.
I'm seeing a lot of demand for local model support and building out relevant tooling so wanted to contribute. We're just now at the point where models are getting fast and smart enough to make rapid computer use possible. You can take a look at the benchmarks in the repository, Carve currently can rapidly and cheaply handle explaining, summarizing, basic navigation, and extraction. Complex, long horizon tasks are a work in progress, though I'm seeing promising results. Failures resemble patterns, and you can identify patterns and fix them by running evaluations in batches.
I'd love any feedback on how well the tool works, what you tried, and any issues that you found. Thank you!