Anyway, the ball is currently in the employer's court and the idea of exchanging one faceless master for another doesn't immediately sound appealing, so I figured I'll try to solve a pain point that I've experienced for a while.
Full transparency: I don't have the MVP working just yet. But what I'm trying to do is gauge genuine demand for an idea before I go all in.
*What if we could generate documentation from tests?*
Having documentation become stale sucks. Keeping docs up to date is hard.
Tests already verify the actual behavior of your system, so they can't lie about what your code does. They're living documentation that's always in sync with reality.
What if we could turn tests into docs that non-technical team members can actually use, or even the public?
It'd be great for onboarding new team members, giving product documentation for everything that's already been implemented, and–assuming we can come up with some best practices on how to write these tests–can even help reduce help desk calls as public facing documentation can self update on every deploy.
And I think we can. I'm currently playing around with this, but the theory is I can use Playwright, create a custom reporter for it, and it'll generate markdown you can use in something like Docusaurus.
That's not the paid product. That'll be an open source library that I'll give away.
But what I want to know is, would you be interested in paying for a SaaS platform that will host the docs and have integrations with:
* Github - allow non-technical to make PRs to update copy (code is the source of truth)
* JIRA – Link to the original requirements and vice versa
* Google Doc style comments for collaborative feedback
* On-prem support if you're paranoid and want to keep your secret docs away from public eyes
Check out my totally original unique landing page if these pain points are something you can relate to and I'm looking for feedback on this idea. Does it have legs? Does this address a problem you see at your company? Do you want help writing better tests to have better documentation for your codebase?
evrflx•7mo ago
I like the idea!
We use it actually for a financial application we develop for a bank.
We use spring test docs with tests to create example api calls with answers, run reference calculations as part of the test and record the outcome and decisions
Both become part of the documentation rendered with asciidoc.
We added custom annotations to add documentation snippets thorough the code in addition to using drools and recording the ruleset as well.
Feedback is great! But it is no generic approach and involved quite some effort for infrastructure and ongoing maintenance.
But well worth the effort given the stakes involved.
Perhaps this helps you as feedback. I am curious how your approach will turn out.
dethstrobe•7mo ago
That's great to know. I was thinking about tackling this from the highest level of abstraction first, so the user interface. I plan on supporting Playwright first for MVP, and then expanding in to Cypress, and maybe even unit testing frameworks.
I feel like backend API documentation is kind of handled with things like Swagger.
How do you think your in house solution compares to something like Swagger or Javadoc?
One of my personal fears (which might be a bit unfounded) since Swagger and Javadoc are generated based off of code comments, and not tests, and there is a possibility that they could get out of sync with implementation. But that might be unfounded. When I worked on Java and wrote unit tests and generated Swagger docs, we never actually ran in to the problem of these things becoming out of sync.
I theorize that the front end isn't as well disciplined as it is in the backend world as well. Which is where I think this idea of Test2Doc will really shine.
dethstrobe•7mo ago
Anyway, the ball is currently in the employer's court and the idea of exchanging one faceless master for another doesn't immediately sound appealing, so I figured I'll try to solve a pain point that I've experienced for a while.
Full transparency: I don't have the MVP working just yet. But what I'm trying to do is gauge genuine demand for an idea before I go all in.
*What if we could generate documentation from tests?*
Having documentation become stale sucks. Keeping docs up to date is hard.
Tests already verify the actual behavior of your system, so they can't lie about what your code does. They're living documentation that's always in sync with reality.
What if we could turn tests into docs that non-technical team members can actually use, or even the public?
It'd be great for onboarding new team members, giving product documentation for everything that's already been implemented, and–assuming we can come up with some best practices on how to write these tests–can even help reduce help desk calls as public facing documentation can self update on every deploy.
And I think we can. I'm currently playing around with this, but the theory is I can use Playwright, create a custom reporter for it, and it'll generate markdown you can use in something like Docusaurus.
That's not the paid product. That'll be an open source library that I'll give away.
But what I want to know is, would you be interested in paying for a SaaS platform that will host the docs and have integrations with:
* Github - allow non-technical to make PRs to update copy (code is the source of truth)
* JIRA – Link to the original requirements and vice versa
* Google Doc style comments for collaborative feedback
* On-prem support if you're paranoid and want to keep your secret docs away from public eyes
Check out my totally original unique landing page if these pain points are something you can relate to and I'm looking for feedback on this idea. Does it have legs? Does this address a problem you see at your company? Do you want help writing better tests to have better documentation for your codebase?
evrflx•7mo ago
Perhaps this helps you as feedback. I am curious how your approach will turn out.
dethstrobe•7mo ago
I feel like backend API documentation is kind of handled with things like Swagger.
How do you think your in house solution compares to something like Swagger or Javadoc?
One of my personal fears (which might be a bit unfounded) since Swagger and Javadoc are generated based off of code comments, and not tests, and there is a possibility that they could get out of sync with implementation. But that might be unfounded. When I worked on Java and wrote unit tests and generated Swagger docs, we never actually ran in to the problem of these things becoming out of sync.
I theorize that the front end isn't as well disciplined as it is in the backend world as well. Which is where I think this idea of Test2Doc will really shine.