DeepMind recently introduced DataRater, a meta-learning approach that rates the value of training data points using meta-gradients. It promises more efficient training by filtering/re-weighting low-value data instead of relying on heuristics.

Sharing an implementation of the DataRater where I tried to reproduce the results on MNIST!