• Markov Decision Processes
• Bellman Equation and Dynamic Programming
• Value Iteration
• Policy Iteration
• Hands-on Experience: Jupyter notebook with a simple numpy-based tutorial on Value Iteration and Policy Iteration, including solutions

Expected Outcome: The emphasis will primarily be on understanding the necessary mathematical tools for RL. This will lay out the foundations for things to come in the next set of modules. We will see some basic examples of optimizing an organization's decision-making and eventually solving them using value and policy iteration algorithms.

• Temporal Difference (TD) Learning and Monte Carlo (MC) Methods
• RL Framework: OpenAI Gym Environment
• Exploration vs Exploitation in RL
• Actor-Only, Critic-Only, and Actor-Critic Algorithms
• Q-Learning
• SARSA
• REINFORCE
• Hands-on Experience: Jupyter Notebook tutorial on TD, MC, Q-Learning, SARSA, and REINFORCE, including solutions

Expected Outcome: This is when we will start looking at basic RL algorithms. We will better understand objects like state-action value functions (or the Q in Q-learning). Different types of RL algorithms will be covered and used to solve some toy problems.

• Introduction to Linear Function Approximation
• DQN and Variants
• PPO and Variants
• DDPG, TD3, SAC
• Hands-on Experience: Jupyter Notebook tutorial with OpenAI Stable-Baselines3-based solutions

Expected Outcome: Most practical problems suffer from the curse of dimensionality, and that's when we will need to combine RL with Deep Learning (DL), popularly known as DRL. We will explore multiple state-of-the-art algorithms and their applications in Gym environments.

• When to Use RL
• RL for Real-Time Control of Power Networks
• RL for Real-Time Inventory Management
• Decision-Making in the Presence of Delays
• Incorporating Delays in Inventory Management

Expected Outcome: This is when we will dive deeper into solving practical, real-world problems using RL.

• Prompting Using LLMs
• RL with Human Feedback (RLHF)
• Direct Preference Optimization (DPO)
• Reward Restructuring Using LLMs
• Hands-on Experience: Jupyter Notebook tutorial on learning anthropomorphic policies in autonomous navigation

Expected Outcome: We will look to generate human-centric behavior in RL-learned policies using the power of LLMs.

• Strong foundation in mathematics, particularly linear algebra, calculus, and probability theory
• Proficiency in Python programming
• Basic understanding of machine learning concepts
• Familiarity with deep learning frameworks (e.g., PyTorch or TensorFlow)
• Experience with data analysis and visualization libraries (e.g., NumPy, Pandas, Matplotlib)
• Basic knowledge of optimization techniques
• Familiarity with Jupyter Notebooks and version control systems (e.g., Git)