- How are you currently monitoring the performance, behavior, and output quality of your chatbot? - What metrics or KPIs matter most to you (e.g., hallucination rate, latency, user satisfaction, retention)? - Are you using any specific tools or dashboards? Any homegrown solutions? - How do you handle error tracking, feedback loops, or regression when updating prompts/models?
We’re building something in this space and want to better understand real-world practices, pain points, and what “good monitoring” looks like for LLM products.
Would love to learn from your experiences—thanks in advance!