Building Large Language Models for Code
IntermediateLevel
1 hrs 0 minsDuration
About this Course
- Here, you will learn how to train LLMs for Code from Scratch, covering Training Data Curation, Data Preparation, Model Architecture, Training, and Evaluation Frameworks.
- Explore each step in-depth, delving into the algorithms and techniques used to create StarCoder, a 15B code generation model trained on 80+ programming languages.
- Understand and learn the best practices to train your own StarCoder on the data.
Learning Outcomes
LLM Training
Master training Large Language Models for code from the ground up.
StarCoder Insight
Gain a deep dive into the StarCoder development process.
Techniques Mastery
Learn techniques used in StarCoder through our Hands-On.
Essential Frameworks
Explore code LLM architecture and evaluation frameworks.
Course Curriculum
Learn the complete process of building Large Language Models for code, covering data curation, model training, fine-tuning, evaluation, and deployment strategies.
1. Introduction to LLM Tricks
2. Recap of Core LLM Concepts
3. Understanding LLM Building Blocks
4. Practical LLM Application Strategies
5. LIMA- Less is More Alignment Insights
6. Distil Step by Step Implementation
7. Instruction BackTranslation Techniques
8. Textbooks as Essential Resources
9. Reducing Sycophancy in LLMs
10. Gorilla- API Following for LLMs
11. ToolLLMs and Their Uses
12. Comprehensive Course Summary
1. Techniques of Large Language Models
Meet the instructor
Our instructor and mentors carry years of experience in data industry
Get this Course Now
With this course you’ll get
- 1 hour
Duration
- Loubna Ben Allal
Instructor
- Intermediate
Level
Certificate of completion
Earn a professional certificate upon course completion
- Globally recognized certificate
- Verifiable online credential
- Enhances professional credibility
Frequently Asked Questions
Looking for answers to other questions?
Foundational machine learning and deep learning knowledge is essential. Proficiency in Python and software development practices are also crucial.
t's an intermediate-level course, so beginners should establish foundational knowledge beforehand. It requires prior experience with ML and coding.
Yes, you'll learn practical insights and best practices for training similar models. The training pipeline, from data curation through model evaluation, will be thoroughly covered.
The training process encompasses data curation, preprocessing, model architecture, and evaluation. Specific techniques and frameworks used in StarCoder's development are explored in detail.
The core instructional content spans 38 minutes, yet additional time for practice and exploration is recommended. Learning is self-paced, allowing you to delve into topics as needed.
Yes, you will receive a certificate of completion after successfully finishing the course and assessments.
Related courses
Expand your knowledge with these related courses and expand way beyond
1 Hours6 Lessons4.6
Getting Started with Large Language Models
0 Hours1 Lessons4.8
Navigating LLM Tradeoffs: Techniques for Speed, Cost, Scale & Accuracy
1 Hours6 Lessons4.5
Framework to Choose the Right LLM for Your Business
Popular Categories
Discover our most popular courses to boost your skills
Popular free courses
Discover our most popular courses to boost your skills
Contact Us Today
Take the first step towards a future of innovation & excellence with Analytics Vidhya
Unlock Your AI & ML Potential
Get Expert Guidance
Need Support? We’ve Got Your Back Anytime!
+91-8068342847 | +91-8046107668
10AM - 7PM (IST) Mon-Sun[email protected]
You'll hear back in 24 hours