Every agent step takes both a visual snapshot and a memory read, the memory gives more consistent tileset and location parsing and also has some stuff like party state/status conditions that isn't usually visible.