Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Overview of MLOps With Open Source Tools

Amit
Amit Last Updated : 15 Mar, 2022
5 min read

This article was published as a part of the Data Science Blogathon.

Overview

The core of the data science project is data & using it to build predictive models and everyone is excited and focused on building an ML model that would give us a near-perfect result mimicking the real-world business scenario. In trying to achieve this outcome one tends to ignore the various other aspects of a data science project, especially the operational aspects. As ML projects are iterative in nature, tracking all the factors, configurations and results can become a very daunting task in itself.

With the growing and distributed data science teams, effective collaboration between teams becomes critical. The open-source tools that we will explore on this blog namely DVC Studio and MLFlow will help us to address some of these challenges by automatically tracking the changes/results from every iteration. To add more, both the tools give us a very interactive UI where the results are neatly showcased and of course, the UI is customizable !!.

Any Prerequisites?

All we need is the basics of machine learning, python, and version control account eg: Github. In this blog, Kaggle’s South Africa Heart Disease dataset will be used for running our experiments. Our target variable will be CHD (coronary heart disease).

Note: We will be using the same dataset and building the same model for sake of simplicity for both DVC Studio and MLflow use cases. Our aim is to understand the features of both the tools and not fine-tune the model building.

DVC Studio

DVC is an open-source tool/library which can be plugged into version control tools like Github, Gitlab, Bitbucket, etc to import ML projects for experimenting and tracking. The studio has a UI to track the experiments/metrics. To know more about DVC explore its features.

Installation & setup: The pip install dvc should be fine for installation. For more information on the installation of the windows version, please refer to install DVC. You can download/clone the code from Github for quick reference. You can download/clone the code from Github for quick reference.

We will load the dataset, split it and then carry out the model building step. All the code/files can be found under the src folder – skipping the code walkthrough of this section to keep the blog relatively short. You can access the complete code and walkthrough from the blog.

Before we move further, let’s take a glimpse of how the experiment and KPI’s are tracked in DVC.

DVC studio for MLOps
Experiment tracking in DVC

Source: Author

As you would have noticed above, the output is in the console, definitely not interactive, what about the plots that are so crucial for ML models?. Is there a way to filter results on a defined threshold? eg: If we need to view the experiments which have an accuracy of higher than 0.7. The only way is to write a piece of code to filter that specific data. That’s where the DVC studio makes it very simple and interactive. We will look into UI in the next sections.

DVC Studio: Set up the DVC studio by following the below steps.

Step 1: Navigate to the URL https://studio.iterative.ai, Sign in with your Github and you will be able to see Add a view at the top right of the screen.

Step 2: The GitHub repository that we have should be mapped to the DVC studio by clicking on Configure Git Integration Settings.

Step 3: Once step 2 is completed, it will open the Git Integrations section. Select the repository and provide access.

Git integration section | MLOps
Map and grant access

Source: Author

Step 4: Once mapped, the repo will be available for the creation of a view as below.

Add a view | MLOps
Creating a view for the DVC repository

Source: Author

Step 5: Once the above steps are completed, click on the repo and open the tracker.

repo and open tracker | MLOps

DVC studio experiment tracker UI

Source: Author

Model Comparison: Select the models of your choice and click on Compare to view the results.

 

Model comparison

Model Comparision

Source: Author

Run the experiments:  There are two ways to run the experiments.

1. Make all the changes and check the code into the Github repository. The DVC studio automatically pulls the metrics on the studio for tracking.

2. The other way is to make changes on the DVC studio UI, run experiments, and then push it to Github.

MLflow

MLflow is an open-source tool for tracking ML experiments. Similar to DVC studio it helps with collaboration, carry out a varied range of experiments and analysis. To know more about MLflow explore its features.

Setting Up Work Environment:

You can access the code repository for download/clone from Github. Install the mlflow and other libraries for model building, set up the config file to improve the code readability. The files can be found under the MLflow folder.

Monitoring Model Metrics: The last and the current metrics are tracked and listed as in snapshot. If the number of iterations increases then keeping track of the changes becomes a challenging process and rather than focussing on improving the model performance, we will spend a lot of time tracking the changes and resulting metrics.

Path                Metric             Old       New       Change
reportscores.json  Logistic Accuracy  0.62069   0.65517   0.03448
reportscores.json  roc_auc            0.65093   0.72764   0.07671
reportscores.json  test_score         62.06897  65.51724  3.44828
reportscores.json  train_score        71.96532  74.27746  2.31214

MLflow gives us a nice MLflow UI which helps us to track everything on the UI. Once, we are ready to run our experiment (classification.py), the metrics are tracked and displayed in the UI with the below piece of code command.

mlflow ui## Here is the output
INFO:waitress:Serving on http://127.0.0.1:5000

The URL is of the localhost, by clicking on the URL, we will be able to view the results in the UI. The UI is very user-friendly and one can easily navigate and explore the concerning metrics. The section highlighted in the red box below shows the tracking of metrics that are of our interest.

MLFLow

MLflow experiment tracker UI

Source: Author

Closing Note

In this blog, we had a look at an overview of MLOps and implemented it with open source tools namely DVC Studio and MLflow. These MLOps tools make tracking the changes and model performance hassle-free so that we can focus more on domain-specific tuning and model performance.

MLOps will continue to evolve in the future with more features added to the tools making the lives of data science teams that much easier in managing the operational side of machine learning projects.

If you liked the blog then here are articles on MLOps. Keep experimenting!

DevOps & MLops

Tracking ML experiments with DVC

My Linkedin & Github

References

https://dvc.org/

https://www.mlflow.org/docs/latest/quickstart.html

Amit
Amit

I am a Data Science enthusiast with experience in building predictive models, data processing, and data mining algorithms to solve challenging business problems. Involved in open source community and passionate about building data apps.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details