Before you start:
π Complete the pre-class exercise. [30 min]Β
π Download the class slides here.
In the previous class, we learned how to predict "the next word". Now, we focus our attention on small language models, and specifically n-gram models.
How can a model understand the essence and context of a word? In this video, we explore word embeddings and their importance in language models.
Let's dive deeper: in this video, we'll get inspiration from biological neurons to understand how artificial neurons in deep neural networks communicate information between each other.
Architectures, weights, activations: in this video, we go into the details of how this large models work and unpack the training process (warning: cute cats and dogs are involved).Β