fp.

Hey HN - we’re Oskar, Szymon, and Piotr, and we’re building TesterArmy (https://tester.army). TesterArmy is an agentic testing platform that runs end-to-end checks before deployment and in production. Instead of wasting hours on manual testing or maintaining static scripts, we let you specify your tests in natural language and handle everything in between. We've built the platform fully around agents. Our agent will reliably execute the tests, but your coding agent can manage everything in our platform, from defining tests in natural language to running them on your behalf.

Check out our demo video: https://www.youtube.com/watch?v=291IkUbPrlk.

We started TesterArmy because testing is still far too painful. AI coding tools have made it dramatically faster to write and ship code, but testing is still a bottleneck. Traditional E2E tests are slow to set up and expensive to maintain. Managing auth and test users is painful. Setting up staging environments is painful. Running tests reliably is painful.

We think most teams do not actually want to spend their time writing selectors or maintaining test infrastructure. They just want confidence that their core flows work. With TesterArmy, an engineer can sign up, give an agent our CLI, and let it handle creating tests and running them on schedule or on GitHub.

When something breaks, TesterArmy alerts your team through Slack or Discord.

Over the past few months, we scaled from 0 to 30+ teams using our product every day. We caught bugs in critical flows, including onboarding, checkout, and AI chat. We've got many of our customers migrating from already established competitors to us because of the quality and reliability of our agents.

Here are a few of the recent bugs that our agent found (there were quite a lot of them!):

1) Timezone bug that affected the booking flow in one of our clients' apps, the dashboard was very complex and hard to catch by a human. 2) Regression in agent orchestration that caused a sandboxed environment to be stuck on loading, thanks to TesterArmy, the team was able to resolve it before it hit production. 3) Incorrectly counting the order amount in a complex dashboard flow with checkout, thanks to TesterArmy, the team was able to resolve it before it affected revenue 4) Catching a regression in an AI chat flow that would result in a user not being able to retrieve their data due to broken tool calling.

And many more, mostly related to some incorrect API calls, 404s, unhandled errors, etc.

If this sounds useful, we would love your feedback at https://tester.army. We have a bunch of free test runs for you to try. And don’t worry, we won’t make you do sales calls, and we don’t have long onboarding or annoying setup. Our goal is an it-just-works experience.

If you're looking for an end-to-end testing solution, we'd love to hear your feedback!

Comments

yohguy•1d ago

Does it work of mobile native applications or expo apps that have native modules?

Pricing question, the usage on the plans seems low considering in the demo you said that you have 25 tests per pr which would mean you get only 10 PRs per month on the hobby plan?

okwasniewski•1d ago

Yes, it works for any framework. We just get the built native binary and run it in the cloud.

Regarding pricing, the self serve options are currently only for lower usage. We will add more plans further down the line. Currently the most popular one is the startup plan. If you need more usage I’m happy to discuss it on a call!

msencenb•1d ago

Have you been able to nail down a loop where your tool can take an open pr, guess the code path and do some testing?

We use cypress heavily for our core flows which has a similar ai prompt thing but it’s not quite ad hoc enough for smaller fixes which is where the bottleneck still comes in for us.

okwasniewski•1d ago

Yes! We spent quite a lot of time on this, and we are currently creating a test plan based on PR changes and sending an agent to verify it. We have some customers who are only using this feature.

dbbk•1d ago

"Traditional E2E tests are slow to set up and expensive to maintain." I don't really understand this. If I'm already using Opus to write the code, surely it would know best what E2E tests to write to be able to verify its own output? This seems like an unnecessary external step.

Laurel1234•1d ago

Seems interesting, but I wonder about this

> Traditional E2E tests are slow to set up and expensive to maintain.

Isn't this just using agents to create e2e tests or is there some better new approach I'm missing?

okwasniewski•1d ago

We use agents to navigate the app, making real-time decisions based on its state. I prefer to compare it more to a manual QA engineer than to static e2e tests. We spent a lot of time on the harness to make sure the results are reliable. This allows you to assert on dynamic content like AI-generated content. We also support validation of email flows since the agent can read its own email.

jaggederest•1d ago

Fable (rip) is absurdly good at this, great time to build a product around it, you definitely need the harness, but it feels like it just turned the corner to be able to do really in depth and edge case work.

Do you handle heterogenous environments and network connectivity simulation as well? I am working on a mobile app and occasionally having users just lose a request or two can put the state machine into unusual modes.

okwasniewski•1d ago

I feel like new AI model releases will only allow our agents to do more in-depth testing; the space still has a lot of room to grow. Quality assurance is way more complicated than just clicking around a UI.

Regarding the other question: not yet. For now, we have Chromium, iOS, and Android (latest versions of each), but we are working on adding more. Regarding network connectivity, it's coming soon (I have an open PR).

Laurel1234•20h ago

> We use agents to navigate the app, making real-time decisions based on its state.

This still leads me to my original question of how though. If you're not using locators are you just passing page contents to the LLM? Or using a multi modal model and say screenshotting? My experience with that has been pretty poor and worse than proper e2e scripts, and is fairly expensive to boot.

Sorry for the insistence haha, just interested because it could be pretty groundbreaking if done well.

_pdp_•1d ago

Great presentation

On a slight tangent, since we are all here...

Does anyone still believe there is a long-term future in traditional UI/UX?

It feels like a lot of attention is still going into landing pages, dashboards, and CRUD apps, while overlooking a bigger shift where fewer people will actually need to interact with those interfaces directly when the same tools can perform the underlying tasks automatically, without much UI at all.

So the bigger question is does UI/UX evolve into something else, or does a large part of it simply disappear?

I might be a bit too early. Recently I started a project and decided to skip all of that and focus to make it more friendly to AI agents and frankly so far it has been great purely from user experience but also what it delivers.

altmanaltman•1d ago

How can it perform tasks automatically? It's not magic, there has to be an UI/UX for interacting with it. Will that UI/UX be more optimized and easier to use is the question. Like would you prefer saying "close window computer" or press alt+f4 or just click on the little cross thing or equivalent. Why are we assuming all AI automagic UI/UX will be better for all tasks?

_pdp_•1d ago

AI agents can perfectly do a lot of the data entry tasks and build dashboards. You practically need to build none of that when you can ask an AI agent to pull the data and build a chart or provide a file or a paste to insert into a database.

Basically that.

If the app requires a mouse then it should have UI, if not, unless critical, it can be driven by an agent.

That's my point.

collingreen•1d ago

Perfectly is a wild word to use here

Eridrus•1d ago

Is there a long term future in hand-crafted UI/UX? Maybe not.

Is there a future where we still have traditional UX? Absolutely.

I don't want to write a whole dissertation on this topic, so I'm just going to mention that we tried to build AI voice assistants for a decade, and while LLMs have basically solved understanding, they have not solved the UX portion.

ed_mercer•21h ago

Aside from entertainment/marketing purposes, I think UI will become useless. Why should I interact with UI when an agent can (and will) interact with it?

Eridrus•8h ago

Because you do not know everything and information (possible inputs, actual outputs) still needs to be presented to you. Because text is not the best way to present everything. Because consistency of presentation makes it easier to absorb. Because everything is social and standardization is useful.