In 2018 Google published a paper with an ungainly name, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," and briefly rewrote the leaderboard for almost every task in natural language processing. Its method was disarmingly childlike: hide some of the words in a sentence and train the model to guess them from the words around them, on both sides. That simple game, played across billions of sentences, produced a model that grasped language well enough to be repurposed, with minimal effort, for question answering, sentiment analysis and much else.

Your free preview ends here
Keep reading with a free account
Sign up and redeem two free articles per month. No credit card required.
Already a member? Log in
