Author here. The one that cost me the most: I quantized my input images down to 6 colors to fit under an API size limit, assuming it only degraded the two frames I was sending. It degraded every frame generated in between — 30–32 colors instead of 63–64, against a static sprite with 49. Counting unique colors per frame ended up being the cheapest defect detector I had.

Happy to answer anything, including the two defects I couldn't build a metric for.