Hi HN, we built an extension to Envoy proxy that parses and sends contents of requests to an LLM at runtime.

We do that by adding a deterministic SQL parser, gGRPC payload inspection and LLM analysis of the payload before upstream call.

Users are proxying headless agents traffic with it so that sec teams can have their prompts blocking harmful actions via network traffic before it leaves the agent.

When you tell the LLM analyzing the traffic to block writes, it will spot it across the different protocols by looking at the structured payload we build from parsing the binary protocol.

It runs as a UDS upstream, and we're exploring bundling it inside envoy to remove the UDS hop.

Envoy is incredible, but it has limited support for anything outside http. Basic database query parsing, little policy enforcement on request contents, most of the stuff happens on HTTP and gRPC headers or metadata.

Would love to hear any feedback: https://github.com/hoophq/hoop/tree/main/sidecar