If you found something wrong in the post, I'd really appreciate hearing about it.
I just hope HN gets over the "but it might be AI!!" crap sooner rather than later and focuses on the actual content because these types of posts are never going away.
The entire "Gradual recovery" part of the post makes absolutely no sense, and is presumably an LLM fabrication. That's just... not how anything works. And deploying three different weird little mitigations flies in the face of the earlier "We couldn't just restart production. Too many active users."
Debugging becomes, hit the debug endpoint, get a list of all goroutines and their call stack. A leaked goroutine shows up a lot with the same call stack and that's it. There is also a fancy graph you can make the visualizes allocated objects if you have a mem leak and aren't sure if it is me or goroutine.
* Anti-pattern because it is so easy to forgo good program design (like a solid state machine) and reach for a goroutine and communicate back with channels. Do that a few times and your code becomes spaghetti.
But, I haven't touched Go. unexciting .net dev...
Channels are buffered in Go, they will not grow unbounded.
> Tickers Are Not Garbage Collected
It used to be necessary in older versions to call ticker.Stop(), but in recent versions it's no longer necessary.
// Start goroutines
go s.pumpMessages(ctx, sub)
go s.heartbeat(ctx, sub)
// Monitor the connection
go s.monitorConnection(ctx, sub)
The "fixed" code is still using the fire-and-forget pattern for goroutines which encourages this kind of leak. Go makes it easy to add concurrency on the caller side, so it's usually better to write blocking functions that clean up all their goroutines before returning.In general this article screams AI with most of the conclusions being hallucinated. Goroutine leaks are real, but it's hard to trust any of the article's conclusions
Uriopass•1h ago
They do once you restart the server. Unsure what the "phase 3 monitoring" shows where the goroutines go down gradually. If you have new code to deploy you must recompile and therefore restart and those goroutines are gone anyway.
This feels like an AI made-up story but I'm not sure I understand the point of making this up.
However, goroutine leak is interesting! I hope what I learned from this post isn't hallucination. For example, how could the subscriber send messages/heartbeats to the closed goroutine without an error...
cassonmars•1h ago
WJW•1h ago
(and lo, the BEAM does indeed allow hot code reloads. I don't think this is commonly used in BEAM-based Erlang/Elixir web services though. Certainly the Gleam people don't seem to like hot reloads very much...)