Mastering LLMs: Training, Fine-tuning and Best Practices

10 AUGUST 2024 | 09:30AM - 05:30PM | location RENAISSANCE :- Race Course Rd, Madhava Nagar Extension

About the workshop

In this workshop, you will get a comprehensive introduction into the world of training and fine-tuning Large Language Models (LLMs) and Generative AI. Over the course of five modules, you will explore the essentials of LLMs, deep-dive into training and fine-tuning methodologies, and learn about parameter-efficient techniques like Low rank Adaptation (LoRA), Quantized low rank Adaptation (QLoRA) and instruction-based fine-tuning using techniques like Supervised Fine-tuning and Reinforcement Learning with Human Feedback. You will also learn about the advantages, disadvantages and best practices for these techniques. This is not a university lecture. We will follow a hybrid approach where you will learn the concepts behind these techniques and also spend a lot of time with live hands-on sessions, actually training and fine-tuning LLMs using tools like the HuggingFace ecosystem and Unsloth AI.

Additional Points:

  • Prerequisites: Basic understanding of Python, NLP and Deep Learning is helpful
  • Content Provided: Slides, complete code notebooks, datasets
  • Infrastructure: We will be running everything on the cloud on a GPU server, you will be provided with runpod.io credits or you can use Colab for some of the hands-on also
  • Important Note: Some of the hands-on, especially SFT and RLHF on LLMs usually take a minimum of 30+ hours so we might pre-run some of them and do a live walkthrough. Rest most of the hands-on everyone can run live along with the instructor in the session
  • Session will be interactive so feel free to keep it interesting and ask questions and treat it as a open-ended discussion
video thumbnail

Instructor

Modules

In this module, we will take a rapid tour of Generative AI and dive into the specifics of Large Language Models. You will learn about different model architectures, such as encoders, decoders, and encoder-decoder in the context of transformers, along with state-space and multi-modal models. We will also cover various types of LLMs and their applications including prompting, summarization, and question answering. A hands-on introduction to the HuggingFace ecosystem will allow participants to experiment with transformer-based LLMs for various NLP tasks.

  • Whirlwind Tour of Generative AI
  • Introduction to Large Language Models
    • What are LLMs?
    • Key architectures (e.g., Encoder, Decoder, Encoder-Decoder, State-space Models, Multi-modal models)
  • Types and Applications of LLMs
    • Foundational and Fine-tuned LLMs
    • Applications - Prompting, Summarization, Extraction, Q&A, RAG
    • Hands-on: Introduction to the HuggingFace ecosystem. NLP applications with foundational and fine-tuned LLMs

This module focuses on the nuts and bolts of training LLMs from scratch and the fine-tuning process. You will understand the stages of data preparation, tokenization, pre-training, and fine-tuning. We will also explore the differences between training a model from scratch and fine-tuning a pre-trained model, learning about the benefits and challenges of each. The module includes hands-on sessions on training an LLM from scratch for text \ code generation and fine-tuning for popular NLP tasks such as classification \ translation.

  • Understanding the LLM Training Process
    • Data preparation, tokenization, pre-training and fine-tuning language models
    • Understanding pre-training vs. fine-tuning
  • Training an LLM from Scratch
    • Understand the key steps to train an LLM from scratch
    • Introduction to HuggingFace datasets, tokenizers, Trainer constructs
    • Hands-on: Training an LLM from scratch - Text and code generation
  • Full Fine-tuning a pre-trained LLM
    • Understand the key-steps to fine-tune all layers and parameters of an LLM
    • Key advantages and limitations of full-fine tuning
    • Methods to improve and accelerate fine-tuning performance
    • Hands-on: Fine-tune an LLM for a popular NLP task - Classification \ Translation etc.

In this module, we will explore parameter-efficient fine-tuning techniques that help in fine-tuning LLMs without expensive computational infrastructure as well as compute time. Techniques such as LoRA, QLoRA, and prompt-based methods like prompt tuning and prefix tuning will be covered. The session will introduce HuggingFace's parameter-efficient fine-tuning (PEFT) library and include hands-on sessions in fine-tuning LLMs for tasks such as classification and question-answering using these advanced methods.

  • Understand what is parameter-efficient fine-tuning, advantages and disadvantages
  • Key parameter-efficient fine-tuning techniques:
    • Decomposition methods: LoRA, QLoRA, DoRA and more
    • Prompt-base methods: prompt tuning, prefix tuning, p-tuning
    • Activation-based methods: IA3
  • Introduction to HuggingFace PEFT and key constructs
  • Hands-on: Fine-tune an LLM using LoRa for a simple NLP task e.g. Classification
  • Hands-on: Fine-tune an LLM using QLoRA for a simple NLP task e.g. Classification
  • Hands-on: Fine-tune an LLM using Prompt-tuning for a simple NLP task e.g. Classification
  • Hands-on: Fine-tune LLMs using above techniques on more complex tasks like Question-Answering
  • Hands-on: Learn how to switch and merge different adapters using PEFT

This module  will deep-dive into instruction-based fine-tuning using approaches like Supervised Fine-tuning (SFT), Reward Modeling (RM), and Reinforcement Learning with Human Feedback (RLHF). Participants will learn about different reinforcement learning policies and their application in fine-tuning LLMs. Hands-on sessions will include fine-tuning models like LLaMA 3, Mistral using HuggingFace TRL and Unsloth AI.

  • Understand what is instruction-based fine-tuning of LLMs and key components and steps
    • Supervised fine-tuning (SFT)
    • Reward modeling (RM)
    • Reinforcement learning with human feedback (RLHF)
    • Understand different RL policies like Proximal Policy Optimization (PPO), Direct preference optimization (DPO) etc.
  • Introduction to HuggingFace TRL and key constructs
  • Introduction to Unsloth AI for even more efficient PEFT
  • Hands-on: Fine-tune LLaMA 3 on prompt-based instructions using Supervised Fine-tuning with HuggingFace TRL and Unsloth AI
  • Hands-on: Go through the complete process of SFT + RM + RLHF on an instruction-based task for fine-tuning LLMs like LLaMA, Mistral etc.

The final module will summarize key topics covered in the workshop and discuss best practices for fine-tuning LLMs. It will address the fine-tuning of both open-source and commercial LLMs like ChatGPT. The session aims to provide a roadmap for future learning and application of the skills gained during the workshop.

  • Understand the best-practices w.r.t fine-tuning LLMs
  • How to fine-tune closed commercial LLMs like ChatGPT or Gemini
  • Future scope, where to go from here?

  • Proficient in Python programming (as explicitly stated in the outline)
  • Familiarity with Natural Language Processing (NLP) concepts (as explicitly stated)
  • Basic knowledge of Deep Learning principles (as explicitly stated)
  • Experience with machine learning frameworks, particularly PyTorch or TensorFlow
  • Familiarity with transformer architectures and their applications in NLP (very important)
  • Experience with Jupyter Notebooks or similar interactive development environments like Google Colab or VS code
  • Familiarity with the Hugging Face ecosystem (recommended) - transformers, peft, trl (we will cover them anyway in the hands-on so you just need to know what they are)
  • We will provide ample GPU credits to ensure a seamless and productive workshop experience.
*Note: These are tentative details and are subject to change.
Stay informed about DHS 2025

Certificate of Participation

Receive a digital (blockchain-enabled) and physical certificate to showcase your accomplishment to the world

  • Earn your certificate
  • Share your achievement
Book Tickets
Book Tickets

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details