I was thinking about something like this for years, few trys before this. Started this repo last year and I think I got something that usable now.

Async framework, mix sync/async python functions, compose them into DAGs, run them, schedule them, persist data between steps or let it flow just in memory.

GitHub: https://github.com/walnutgeek/lythonic

Docs: https://walnutgeek.github.io/lythonic/

PyPI: pip install lythonic

It is dataflow. So theoretically you can compose it with pure functions only. Lythonic requires annotations for params and returns to wire up outputs with inputs. All data saved in sqlite as json for now, and it would work for some amount of data ok.

You may use it as task flow keeping params and returns empty and maintaining all data outside of the flow.

But practically you may do well with some middle ground, just flow metadata thru, enough to make your function calls reproducible and keep some system of records that you can query reliably.

Anyway I will stop rambling ... soon.

Python 3.11+ MIT License. Minimal dependencies: Pydantic, Pyyaml, Croniter

Prepping for v0.1. Looking of feedback. v0.0.14 is out. Claude generated reasonable docs. Sorry, I would not be able to do it better. I am working on Web UI and practical E2E example app as well.

Thank you. -Sergey