Every browser has 2 zones: website controlled, browser controlled. There are many reasons why don't want any dynamic, website controlled content outside of website zone, inside browser zone.

I'm supposing Safari's SVG implementation when moved to supporting favicons meant there were security holes, probably scripting exploits, but also potential XML exploits, so they removed until they could fix these, with a probable low priority.

on edit: ok evidently that was a stupid assumption on my part, as it got a downvote - why is it stupid though? SVG inline needs to support scripting, SVG is XML - if Safari's SVG implementation meant that SVG favicons were open to either XML exploits or scripting exploits that were not adequately handled in the first release (because sitting in the browser chrome part of code instead of web site part of code) then they might have pulled it back quickly until they could fix that.