Generative AI language Modeling with Transformers

Overview

What you’ll learn

1. - Explain the concept of attention mechanisms in transformers, including their role in capturing contextual information.
2. - Describe language modeling with the decoder-based GPT and encoder-based BERT.
3. - Implement positional encoding, masking, attention mechanism, document classification, and create LLMs like GPT and BERT.
4. - Use transformer-based models and PyTorch functions for text classification, language translation, and modeling.