Other providers probably don't even know/care that this is happening
kolanos•58m ago
Other providers might have better isolated KV stores?
6thbit•54m ago
Last year it happened to me more than once that dictating something in chatgpt would become a completely different message after processing, not in a misheard type of way but just a separate conversation more easily explained by the same type of KV cache issue here.
kami23•52m ago
I think in the last year I've had it happen once where I seemingly got someone else's response, it was very odd. I rewound and got what I expected. It was correlated to Anthropic's roll out of new models at the time which always seems to cause instability in their apis.
kami23•50m ago
I like this RCA a lot more than most. I appreciate the ownership and clear explanation of why their testing did not catch this bug.
Also impressed with the speed to address the issue unplug the model and then pretty quick remediation, the speed to remediation matters less to me, but good to see they can act quickly on their platform to kill the serving of the model.
8cvor6j844qw_d6•49m ago
Interesting. That might explain the few times I've received completely unrelated responses from a certain provider's API.
thehamkercat•1h ago
kolanos•58m ago
6thbit•54m ago
kami23•52m ago