So here's the story of my long way here.
Most AI chatbots are basically just wrappers around an LLM. Some of them can juggle context and maybe trigger pre-coded actions, but that's it. For an online store it's almost impossible to find a bot that can actually handle a whole catalog, recommend, search properly... And the few that can - they don’t optimize context at all, which blows up inference costs.
What I hacked together:
1. Built a crawler-bot that fully auto-indexes a site and structures info using sitemap / JSON+LD / OGP to make it cheaper and simpler.
2. Dumps everything into a vector DB, split into collections (products, faq, links...).
3. Each collection gets queried separately during convo, sorted by semantic closeness before hitting the prompt.
4. Topics get deduped with the same semantic search, so no flooding context with dupes.
5. Did some synthetic distillation to train the agent on conv style and specific rules.
6. Every conversation gets analyzed → notes → clustered semantically → grouped into threats, strengths & opportunities.
7. During conversation agent can choose a task you define and complete it.
User dialogs are pure gold for sales. Step 6 lets me see the aggregate picture: could be a product missing from the catalog or just a broken button — either way, it shows up at the top of the dashboard and makes it dead simple to spot where revenue is leaking.
So, what I've built?
Basically it's yet another chatbot, that is easy to use but does almost nothing until you assign some task.
Stage: live, pre-launch, demo w/o signup: https://whilio.com