Unlock: BERT and the Pretrain-Finetune Paradigm

BERT introduced bidirectional pretraining with masked language modeling. The pretrain-finetune paradigm it established, train once on a large corpus then adapt to many tasks, became the default approach for NLP and beyond.