At this point this is penalizing those that came too late for crawling. In my opinion there can only be two option. You want content to be crawled or not. There isn't a mechanism to check if data was used or not, so enforcement is more or less impossible as well.

The article is light on details in how this block is implemented.