It's actually kinda amazing just watching it swipe, open the app launcher, and find issues to fix later.
I wrote an APK that gives access to the accessibility service via an in-Android http server, which talks to a daemon running on the host machine which can, in turn, talk to Hermes. It's designed to be used with an emulator called Waydroid but you could probably run it on your device too. It does need to fall back to ADB when the accessibility tree is incomplete/inaccurate (which is often). And, going back to my question, I haven't found a means of interacting via ADB other than by coordinate/UIAutomator. To be clear, I am not smart or patient enough to be an Android dev, so I use it for ordering things on Amazon.
alas it needs manual intervention between each of those steps to copy and paste or publish results or comment on bugs because qa is outsourced and the outsourcing company refuses to not have someone do something manually in order to avoid losing revenue on their end.
the outsourcing company charges the client per person so they want to have as many people working as possible and the client, an airline, wants the exact opposite. they only hire software development students for the job so we are entirely able to fully automate the workflow and replace ourselves. people from the airline are too busy to do it themselves.
I am very sure most companies will just cobble together something with Claude and some skill.
oneounceman•4h ago
Check it out and let me know what you think. I’d really appreciate any feedback or suggestions. Thanks!
app.deltix.ai
shermozle•2h ago