Training Stable Diffusion Models with Industry Best Practices

10 AUGUST 2024 | 09:30AM - 05:30PM | location La Marvella :- 2nd Block, Jayanagar, Bengaluru

About the workshop

Have you ever used groundbreaking technologies such as MidJourney and Stable Diffusion for your professional and personal work? I am sure you did. These technologies have taken the world by storm and have become part of our lives. Wondering how these technologies work and what makes them so incredibly effective? The answer lies in the power of Diffusion Models.

The diffusion models have become the backbone of modern computer vision! From Dall-E 2 to Midjourney, these powerful models have revolutionized the way machines understand and process information. But what exactly are Diffusion Models, and how do they work? Don’t worry! You are at the right place. Welcome to the workshop on Diffusion models!

video thumbnail

Instructor

Modules

This section is to bootstrap your interest in Generative AI and mentally orient yourself in the landscape of Stable Diffusion. 

  • Revisit the Prerequisites: DL, ML, Optimization fundamentals needed. 
  • What is Generative AI: Introduction, Landscape, Domains, Sub-Fields.
  • Why Generative AI: Why it matters, How to augment it, How to Capitalize on it.
  • History of Generative AI: Short but Significant history, Milestones, Ahha moments.   
  • Why should you care about the history of Stable Diffusion: Professional Impact, Societal Implications, Technology Reshaping Information Exchange.  
  • Applications of Generative AI: An Umbrella look at the Concrete applied Use Cases. 
  • Gauging the State of Art in Stable Diffusion: What is the forefront and Who is at the forefront. 

This module will be most important in terms of low level understandings of Stable Diffusion and  offers an in-depth comprehension of Diffusion Methods. This module will be extremely hands-on and It encompasses:

  • Explanation of Diffusion Models and their purpose.
  • Acquire the Intuition behind Stable Diffusion Model.
  • Paper Review: Fergus & Zeiler :Visualizing and Understanding CNNs Gradients
  • Paper Review: CLIP (Contrastive Language–Image Pre-training)
  • Understanding Text and Image Embeddings and Their Mutual Relation.
  • Tokens as Embedding: Understanding Nuances. 
  • Detailed analysis of the inner workings of Diffusion Models. Understand Math and how it looks into the code. 
  • Setup the Stable Diffusion Development Environment and attaining GPU/vCPU stage.
  • Understand the training Paradigm of Stable Diffusions.
  • Hugging Face’s Diffusers library, Setup of Hugging Face Spaces and API Key.
  • Quick overview of Google Colab for Stable Diffusion. 
  • How to Generate Images using State of Art Stable DIffusion Model. Code and Exercise. 

This module builds on top of module 1 and explains various Terminologies and Parameters of Stable Diffusion, which are essential to know while building your own stable diffusion solutions. This module will be much more deeper in terms of explanation of inner workings of Stable Diffusion and will male you think critically about the method of procedures to be followed while building Image based Gen AI solutions. This module will make you learn following:

  • Paper Review: The VAE (variational autoencoder)
  • What it means by Denoising Diffusion and Reverse Diffusion. 
  • Predicting noise with the UNet.
  • Removing noise with schedulers.
  • Understanding Critical Key Concepts:
    • Pre-trained pipelines
    • Guidance scale
    • Negative prompts
    • Finite differencing
    • Analytic derivatives
    • Textual inversion
    • Latents
    • U-Nets
    • Text encoders and image encoders
    • Contrastive loss function
    • CLIP text encoder
    • Deep learning optimizers
    • Perceptual loss
  • How it all fits into code. Complete Code walkthrough of Stable Diffusion Method. 
  • Train a Stable Diffusion Model on GPU Cloud. 
  • Test your own Diffusion Model For Clothing Articles.
  • Question and Answers: To make sure, Everybody got the Stable Diffusion Method fully.  

This module takes us through the applications, which are created using Stable Diffusion and How can we create our own applications using Stable Diffusion methods and solutions. This Module includes: 

  • How Do We Train Stable Diffusion at Scale.
  • Paper Review: Progressive Distillation for Fast Sampling of Diffusion Models
  • Paper Review: On Distillation of Guided Diffusion Models
  • Ethical Implications of Training a Stable Diffusion Models.
  • Biggest Player contributing to Open Source Gen AI. 
  • How to Capitalize and Contribute to Open Source Stable Diffusion.
  • Should you train Stable Diffusion from Scratch? Yes and No!
  • What is considered valuable in Stable Diffusion Domain.
  • Stability.ai: OpenAI, but a better and much more supporting approach. 
  • DreamStudio and StableStudio: Blessings of Stable Diffusion.
  • Stable Diffusion WebUI Introduction and Purpose.
  • Running Automatic1111 WebUI on Kaggle or any GPU Environment.
  • Checkpoint:  Everybody is able to get to WebUI? Provide Assistance.

This module introduces the concepts of Various Stable methods and models, which are essential to know and augment in the tools, which can be built for Stable Diffusion. Also, the central idea of this module is to try things by themselves and get comfortable with concepts. In this module we will discuss:

  • Analyzing Prompts, Prompts Matrix.
  • Prompt Strengths and Weights. 
  • Prompt Editing and Blending for Stable Diffusion. 
  • Understanding and Using XYZ Plots for Stable Diffusion. 
  • Hands-On Exercises and Encouraging People to Try the ideas themselves. 

This module will introduce and discuss various Stable Diffusion Applications and tasks which are built on this technology with little bit of applied understanding of this technology. This module will consist of:

  • Introduction: img2img. How to use it?
  • Image In-Painting: Various forms and factors.
  • Image Editing with img2img Sketch.
  • Image Editing with In-Paint Sketch.
  • CIVITAI: Introduction and fine-tuned models. Model Zoo of Stable Diffusion
  • Stable Diffusion Extensions. 
  • MidJourney: The Storm, The Story, The Inspiration and Healthy Serendipity. 
  • MidJourney: How to exploit MidJourney for your purpose at hand. 
  • Adobe Firefly: Introduction and Demo
  • Microsoft Designer: Introduction and Demo
  • OpenAI DallE-2: Introduction and Demo
  • Blue Willow: Introduction, Demo and Warnings. 
  • Lexica Art Prompt Learnings. 

  • Most Important:  Get your work reviewed by the Instructors post workshop.
  • Bonus: One month of Free mentorship by Instructors over the email.
  • Where to go from here?
  • How to keep a tab on this space.
  • Join our Slack or Discord Community if not done already.

  • Strong foundation in deep learning and machine learning concepts
  • Proficiency in Python programming
  • Familiarity with neural network architectures, especially CNNs and transformers
  • Basic understanding of computer vision principles
  • Experience with deep learning frameworks (e.g., PyTorch or TensorFlow)
  • Familiarity with GPU computing and cloud platforms
  • Basic knowledge of natural language processing (for understanding text-to-image concepts)
*Note: These are tentative details and are subject to change.
Stay informed about DHS 2025

Certificate of Participation

Receive a digital (blockchain-enabled) and physical certificate to showcase your accomplishment to the world

  • Earn your certificate
  • Share your achievement
Book Tickets
Book Tickets

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details