Wouldn’t someone start using AGI to create spam seconds after it was configured to filter spam?

For conventional spam, no, I don't think so. Conventional spam requires spewing out vast numbers of untargeted emails, and it's the quantity of spew that makes it spam. Owning botnets allows you to hide that somewhat, but that seems to be a solved-ish problem, and I don't think AI models will significantly change that.

What you will get, instead, is much more targeted unsolicited email. Instead of spewing billions of messages, and then turning the conversation over to a human being when it finds a target, the AI will be able to find likely targets, determine what will work on them, and con the target out of money.

So, less spam, but more scams. AI might be able to help with that: "This person probably isn't actually your grandson looking for cash bail. You probably don't want to send money to Nigeria."