self-supervised learning

self-supervised learning is a form of unsupervised learning where the data itself provides the supervision without access to handmade training labels.

In general, withhold some part of the data and task the model with predicting the masked portion. The model generates a useful representation (Representation Learning) of the input data via its parameters which can then be used in some downstream task. This is an example of Transfer learning.

The loss of the self-supervised learning task serves as a proxy for the downstream task that the model will be fine-tuned on.


References

eonline