Coding a ChatGPT-style Language Model from Scratch in PyTorch

About

Currently, ChatGPT is captivating people worldwide. In this GenAI session, we’ll dive into coding, training, and deploying a ChatGPT-style language model from scratch using PyTorch. Through this hands-on experience, we'll explore the strengths and weaknesses of models like ChatGPT and discuss alternative design strategies. Additionally, we’ll learn how to fine-tune a production language model using a custom dataset. This fine-tuning process grants us greater control over the model's behavior, enhancing its reliability and tailoring it to specific needs. Join us to understand the intricacies of building and refining powerful language models.

Key Takeaways:

  • Understand how to code positional encoding.
  • Learn to code attention.
  • Get to know how the different parts of the language model fit together.
  • How to format data for a Decoder-Only Transformer?
  • How do you train and use a language model?
  • Learn to fine-tune and use a production large language model.



Speaker

video thumbnail
Book Tickets
Stay informed about DHS 2025

Download agenda

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details