I couldn't find any suitable libraries for implementing SIP in Go, so I used a library just for the message parser, and reimplemented the protocol logic myself. It's somewhat incomplete at this time, but complete enough to work with at least three SIP providers (self-hosted Asterisk, Linphone and OVH).
If somebody ever told you that SIP is horrible, they were telling the truth.