Neural Machine Translation

Objectives

• Explain how an Encoder/Decoder model works
• Apply word alignment for machine translation
• Train a Neural Machine Translation model with Attention
• Develop intuition for how teacher forcing helps a translation model check its predictions
• Use BLEU score and ROUGE score to evaluate machine-generated text quality

Text Summarization

• Describe the three basic types of attention
• Name the two types of layers in a Transformer
• Define three main matrices in attention
• Interpret the math behind scaled dot product attention, causal attention, and multi-head attention
• Use articles and their summaries to create input features for training a text summarizer
• Build a Transformer decoder model (GPT-2)

Question Answering

• Gain intuition for how transfer learning works in the context of NLP
• Identify two approaches to transfer learning
• Discuss the evolution of language models from CBOW to T5 and Bert
• Fine-tune BERT on a dataset
• Implement context-based question answering with T5
• Interpret the GLUE benchmark