"Code interpreters" are incredibly powerful tools for agents as it allows them to process large amounts of data without actually having to move the tokens through it's context window.
However, I don't actually see what any of this has to do with MCP. It's more-so just tool calling + code interpreter design patterns. If anything, the MCP hype has resulted in a lot of BAD tools being written that return ridiculous number of tokens.
SQL is really a perfect solution for allowing the agent to access data, but in most applications, it's not realistic to provide it with a db connection. Your either need RLS and user connection pooling (like supabase) or strict application tenant filtering (which is tricky) and even then, you still can't efficiently join data from multiple sources.
I recently built a system using tenant isolated S3 "dataponds", populated with parquet files, that the agent queries with duckdb.
The VM (agent core) gets an short lived "assume role" STS token injected so it can only access buckets the user is entitled to (also VPC networking so no other ingress/egress). Each location has a `manifest.json` file that describes the contents and schema. The agent can query the parquet files using duckdb, do additional processing with pandas and then share files back to user by writing it to S3 in special predefined display formats (data-table, time-series, etc).
The file ids, along with a sample of the data is pass back the LLM. It can then embed the file in it's response using a special tag and a custom markdown renderer displays it back to the user.
From what I can tell, this is basically what Chat-GPT has done for a long time (minus the preconfigured "datapond" and special response formats), but it's pretty awesome how "low-effort" a system like this can be built today.
cjonas•1h ago
However, I don't actually see what any of this has to do with MCP. It's more-so just tool calling + code interpreter design patterns. If anything, the MCP hype has resulted in a lot of BAD tools being written that return ridiculous number of tokens.
SQL is really a perfect solution for allowing the agent to access data, but in most applications, it's not realistic to provide it with a db connection. Your either need RLS and user connection pooling (like supabase) or strict application tenant filtering (which is tricky) and even then, you still can't efficiently join data from multiple sources.
I recently built a system using tenant isolated S3 "dataponds", populated with parquet files, that the agent queries with duckdb.
The VM (agent core) gets an short lived "assume role" STS token injected so it can only access buckets the user is entitled to (also VPC networking so no other ingress/egress). Each location has a `manifest.json` file that describes the contents and schema. The agent can query the parquet files using duckdb, do additional processing with pandas and then share files back to user by writing it to S3 in special predefined display formats (data-table, time-series, etc).
The file ids, along with a sample of the data is pass back the LLM. It can then embed the file in it's response using a special tag and a custom markdown renderer displays it back to the user.
From what I can tell, this is basically what Chat-GPT has done for a long time (minus the preconfigured "datapond" and special response formats), but it's pretty awesome how "low-effort" a system like this can be built today.