What it does - Connects any LLM to existing ROS robots via the Model Context Protocol (MCP) - Natural language → ROS topics, services, and actions (And the ability to read any of them back) - Works without changing robot source code
Why it matters - Makes robots accessible from natural language interfaces - Opens the door to rapid prototyping of AI-robot applications - We are trying to create a common interface for safe AI ↔ robot communication
This is too big to develop alone — we’d love feedback, contributors, and partners from both the robotics and AI communities.
avasan•3h ago
r-johnv•3h ago
For the industrial robot (in the video on the main readme page) I intentionally gave Claude no context beforehand. All of the inferences that you see there are based on information that it got through the MCP tools that let it analyze the ROS topics and services on the robot.
In fact, I had a starting prompt to ignore all context from previous conversations because this looked to be like an example of emergent behavior and I wanted to confirm that it was not picking things from my earlier conversations!
avasan•45m ago
r-johnv•40m ago
avasan•29m ago
I sometimes face issues with LLMs running out of tokens or only using partial contacts from previous conversations - thereby repeating or compounding on previous incorrect responses.
Any thoughts on how to tackle that? Or is that too abstract a problem/beyond the scope to address at the moment?
r-johnv•19m ago
So far, I've observed that for Claude and Gemini, which are what we've been testing most with, the Language model has been pretty good at recognizing its a faulty initial interpretation when it queries more information from the system.
Running out of tokens is a more significant issue. We saw it a lot when we queried imaged topics, which led us to try writing better image interpreters within the MCP server itself (credit to my collaborators at the Hanyang university in Korea) to defend the context window. Free tiers of the language models also run out of tokens quite quickly.
PS - Thank you for the questions, I'm enjoying talking about this here on HN with people who look at it critically and challenge us!
r-johnv•40m ago
The demo video with the Unitree Go (robot dog) uses this approach to give the LLM additional context of the custom poses available to it.