One problem I kept running into was that a model could continue running normally while something underneath was starting to go wrong. By the time you notice it, you may already have spent hours trying to figure out what changed.
MLSentinel monitors model health and surfaces issues through monitoring, reports, and alerts.
The software is free to use.
The platform is not open source, but the SDK is publicly available here: https://github.com/Narasimha440/mlsentinel
You can try the platform here: https://mlsentinel.dev
I'd especially like feedback from people who have deployed ML models: what do you currently monitor in production, and what tends to get missed?