ALSO there's a bunch of interesting questions around security and permission models when code is pulled on demand. Running arbitrary tools in a sandbox is neat, BUT you still need to think about what those tools can access… environment variables, network, local file system… limiting that scope could make these experiments more viable for larger teams.
i'd be curious to see benchmarks for cold starts and memory usage in this model… as well as patterns for caching compiled tools so they aren't reloaded every time. Discovering tools on demand is one thing, making them feel instantaneous is another. HOWEVER it's exciting to see folks pushing on this area and sharing their work.
on your second point, check out how we lock down the sandbox with a custom deno runtime! https://github.com/portofcontext/pctx/tree/main/crates/pctx_...
on third, will def get some benchmarks out... we setup OTEL so we have the data
One question: why is Deno used? I thought that it was a JavaScript runtime. Can pctx only run sandboxed JavaScript code? If so, what do you do if you need the agent to run a Python script? If not, I don't understand how using a sandboxed JavaScript runtime allows you to sandbox other things.
we chose typescript because it's the most token efficient way to pass types and signatures to an LLM, with Python and Pydantic there are extra characters passed around
thinking a native connection to cloud auth managers is the best way to do this (clerk, auth0, etc.)
> pctx optimizes this communication by presenting MCP servers as code APIs
Would be nice to have examples of how this is reduced, if some information was lost in the process and what the tradeoff is
And the original Anthropic post that inspired both: https://www.anthropic.com/engineering/code-execution-with-mc...
This is the real problem in my opinion.
There are a ton of great sounding MCP but in practice they have too many individual tools and way too much documentation for each tool. It inflates processing time and burns tokens.
I find MCP is the opposite of the Unix design philosophy. You want fewer tools with more options surfaced via schema, shorter documentation, and you want to rely on convention as much as possible.
You don’t want a create file, write file, and update file tools, you want one write file tool with the ability to do all of those things. Instead of ls and find you want your list files tool to support regex and fuzzy matching with a metadata list.
This is based on building these things for most of this year, so it’s anecdotal and ymmv.
As an example rust-mcp-filesystem has 24 tools, many with completely overlapping functionality: `head_file`, `tail_file`, `read_file_lines`, `read_text_file` plus multi-file variants; or there's `list_directory`, `list_directory_with_sizes`, `calculate_directory_size`, `search_files`, and `directory_tree`. I think that whole server could be 4-6 mcp tools and it would accelerate things.
Someone please build this with lightweight containers so it's not limited to JS services
Fannon•2mo ago
thecopy•2mo ago
(shameless plug: im building an cloud based gateway where the set of servers given to an mcp client can be controlled using "profiles": https://docs.gatana.ai/profiles/)