This time two years ago, I was lucky enough to have the opportunity to work on one of the very first enterprise grade ChatGPT applications, delivering internal data through a natural language chat interface.
With the help of a Microsoft Early Adoption Team, we ingested data from a variety of sources, and ran the data through a RAG pipeline. This allowed our virtual assistant to answer company specific questions without performing any machine learning on our end. At this point the technology was still very new, and no standards existed for delivering this kind of tool.
Needless to say, we hit a lot of walls:
- Frequent API failures and outages left our application slow, and error prone.
- The lack of persistent chat history storage often led to confusing or irrelevant LLM responses.
- Our RAG pipeline took weeks to configure, and native integration left us unable to make use of tool calls.
- New model releases and API changes were hard to keep up with, and kept us siloed within Azure OpenAI, unable to try models like Claude, Grok, or Llama.
- And tweaking a models system message, or temperature required redeploying the entire application.
Having learned all the challenges of setting up an LLM powered chat interface, I set out to create an API that would vastly simplify this process. Which leads me to the reason for this post today...
Introducing, The Intelligence Hub, you're all in one API for managing anything a chat bot could need. With just a little set up, The Intelligence Hub allows you to:
- Save agentic 'Profiles,' simplifying client request payloads.
- Swap between Azure AI Foundry, OpenAI, and Anthropic models with a single request.
- Use tools in parallel with RAG databases.
- store or load message history by simply providing a "ConversationId" in the request body.
- Configure tool execution URLs to send requests directly to your APIs.
- Consume RAG databases hosted within Azure AI Search, or Weaviate.
- Generate images through Azure while using an OpenAI or Anthropic model, and vice versa.
- Provide "ReferenceProfiles," allowing for one LLM to pick up where another left off.
- Ensure requests aren't dropped with built in resiliency handling.
- And much, much more
I have plans to release a managed service with a robust free tier this coming Tuesday, but in the mean time, you can take a look at the below GitHub project if you'd like to start building.
Contributions are not just welcome, but highly encouraged! Interfaces are used to ensure extending the interoperability of the service to additional RAG or LLM providers is a breeze, but feel free to reach out to me with any questions, or wild ideas you'd like implemented.
Jacob-J-Thomas•6mo ago
With the help of a Microsoft Early Adoption Team, we ingested data from a variety of sources, and ran the data through a RAG pipeline. This allowed our virtual assistant to answer company specific questions without performing any machine learning on our end. At this point the technology was still very new, and no standards existed for delivering this kind of tool.
Needless to say, we hit a lot of walls: - Frequent API failures and outages left our application slow, and error prone. - The lack of persistent chat history storage often led to confusing or irrelevant LLM responses. - Our RAG pipeline took weeks to configure, and native integration left us unable to make use of tool calls. - New model releases and API changes were hard to keep up with, and kept us siloed within Azure OpenAI, unable to try models like Claude, Grok, or Llama. - And tweaking a models system message, or temperature required redeploying the entire application.
Having learned all the challenges of setting up an LLM powered chat interface, I set out to create an API that would vastly simplify this process. Which leads me to the reason for this post today...
Introducing, The Intelligence Hub, you're all in one API for managing anything a chat bot could need. With just a little set up, The Intelligence Hub allows you to:
- Save agentic 'Profiles,' simplifying client request payloads. - Swap between Azure AI Foundry, OpenAI, and Anthropic models with a single request. - Use tools in parallel with RAG databases. - store or load message history by simply providing a "ConversationId" in the request body. - Configure tool execution URLs to send requests directly to your APIs. - Consume RAG databases hosted within Azure AI Search, or Weaviate. - Generate images through Azure while using an OpenAI or Anthropic model, and vice versa. - Provide "ReferenceProfiles," allowing for one LLM to pick up where another left off. - Ensure requests aren't dropped with built in resiliency handling. - And much, much more
I have plans to release a managed service with a robust free tier this coming Tuesday, but in the mean time, you can take a look at the below GitHub project if you'd like to start building.
Contributions are not just welcome, but highly encouraged! Interfaces are used to ensure extending the interoperability of the service to additional RAG or LLM providers is a breeze, but feel free to reach out to me with any questions, or wild ideas you'd like implemented.