- Python developers interested in server tech, especially if already using Envoy
- Envoy users to see use cases enabled by dynamic modules
- Rust fans to see an example of it bridging two non-Rust ecosystems, Envoy and Python
Note, this is my first real project in Rust - while I tried to study idioms while working on it, I doubt it is an example of a fully idiomatic Rust project.
I was interested in a Python app server with support for HTTP/2 trailers to be able to serve gRPC as a normal application, together with non-gRPC endpoints. When looking at existing options, I noticed a lot of complexity with wiring up sockets, flow control, and similar. Coming from Go, I am used to net/http providing fully featured, production-ready HTTP servers with very little work. But for many reasons, it's not realistic to drive Python apps from Go.
Coincidentally, Envoy released support for dynamic modules which allow running arbitrary code in Envoy, along with a Rust SDK. I thought it would be a fun experiment to see if this could actually drive a full Python server - I thought it wouldn't. But after exposing some more knobs in dynamic modules - it actually worked and pyvoy was born, a dynamic module that loads the Python interpreter to run ASGI and WSGI apps, marshaling from Envoy's HTTP filter. There's also a CLI which takes care of running Envoy with the module pointed to an app - this is definitely not net/http level of convenience, but I appreciate that complexity is only on the startup side. There is nothing needed to handle HTTP, TLS, etc in pyvoy, it is all taken care of by the battle-hardened Envoy stack, and we get everything from HTTP, including trailers and HTTP/3.
With support for trailers, pyvoy drives the gRPC protocol support on the server for connect-python (https://github.com/connectrpc/connect-python), allowing them to be served along an existing Flask or FastAPI application as needed. Notably, it is the only server that passes all of connect's conformance tests with no flakiness. It's important to note that uvicorn also passes reliably when disabling features that require HTTP/2. It's a great server when bidirectional streaming or gRPC aren't needed - unfortunately others we tried would have unreliable behavior handling client disconnects, keepalive, and such. This doesn't surprise me as I have seen quite some time ago how hard it is to implement especially HTTP/2 reliably, and I appreciate pyvoy can rely on Envoy to just take care of it.
It seems that pyvoy is a fast (always benchmark your own workload), reliable server not just for gRPC but any workload. It also can directly use any Envoy feature, and could replace a pair of Envoy + Python app server. I currently use it in production at low scale serving Django, FastAPI, and connect-python.
Happy to hear any thoughts on this project. Thanks for reading!