Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Kubernetes -Let’s Learn The Basic Tenets

Nibedita Dutta
Nibedita Dutta Last Updated : 10 Jul, 2021
5 min read

This article was published as a part of the Data Science Blogathon

Introduction

Kubernetes is an open-source container orchestration tool that helps in managing containerized applications in different development environments. Kubernetes enables building application services that consist of multiple containers, scheduling these containers across a cluster, scaling the containers, and manage their performance over time.

What problems does Kubernetes solve?

If we consider microservices, which typically each run in their own containers, a containerized application might become hundreds or thousands of containers when building and operating a large-scale system. Significant complexity would be involved to manage these containers manually. Hence, this is where container orchestration can come in handy for making that operational complexity manageable for development and operations. Container orchestration technologies like Kubernetes automatically and continuously monitor the cluster of containers and make adjustments as required ensuring that there is no downtime in a production environment. For instance, if a container goes down, another container automatically takes its place without the end-user ever noticing.

Kubernetes vs Dockers:

Docker refers to a specific platform meant for building containerized applications, while Kubernetes is a container orchestration tool that aids in managing the container’s lifecycle. Docker Swarm is Docker’s own container orchestration tool that enables managing multiple containers deployed across multiple host machines.

docker vs kubernetes
Source: https://ubuntu.com/blog/kubernetes-versus-docker

 

Basic Architecture of Kubernetes:

Like other distributed computing platforms, a Kubernetes cluster has at least one master and multiple compute nodes. The master node is the node that works on exposing the application program interface (API), scheduling the deployments, and managing the overall cluster. Each of the worker nodes runs a container runtime, such as Docker, along with an agent that communicates with the master. The nodes can either be virtual machines (VMs) running in a cloud or physical servers running within a data centre.

basic architecture kubernetes
Source: https://platform9.com/blog/kubernetes-enterprise-chapter-2-kubernetes-architecture-concepts/

 

Kubernetes Master Node:

The master node receives input from a CLI or UI via an API. This input can be the commands that a developer provides to Kubernetes. For example, which container images are to be run, which ports are to be exposed, parameters of the desired state of the applications running in the cluster are defined by the developer.

1) API Server: Both internal & external components communicate through the same API server.

2) Key-Value Store (etcd): The Key-value store acts as the single source of truth for all components present in the Kubernetes cluster. The master node asks the etcd for retrieving parameters of the state of the nodes and containers.

3) Controller: A controller checks the state of the cluster and makes changes attempting to move it from the current state to the desired state.

4) Scheduler: A Scheduler looks for requests coming from the API Server and is responsible for assigning them to healthy nodes. It assesses the quality of the nodes and deploys pods to the best-suited node.

Kubernetes Worker Node: Worker Nodes are responsible for executing the work assignments from the master Node are and reporting back the results to the master node.

1) Kubelet: Every node of the cluster has a ‘Kubelet’ which is the primary Kubernetes agent. Kubelet watches for tasks from the master node, executes them and then reports back the results to the master node. Additionally, it monitors the pods and reports back to the Master if a pod is dysfunctional.

2) Container Runtime: A container runtime executes containers and managers container images on a node. For example, Docker is a popular container runtime.

3) Kube-Proxy: The Kube-proxy maintains network rules on each node and ensures that each node has an IP.

4) Pod: A pod constitutes one or more containers and includes shared storage (volumes), IP address, and information about how to run them.

Running Kubernetes on Local

We can run Kubernetes on our local system, instead of a cloud service. For this, we would need to install two programs – Minikube and Kubectl. Minikube allows us to run a single-node Kubernetes cluster on our local computer. Kubectl is a command-line tool for Kubernetes.

Minikube uses a hypervisor driver that varies by the operating system. This link has the list of drivers that one could use for setting up Minikube- https://minikube.sigs.k8s.io/docs/drivers/

We will be using Docker as the hypervisor drive in this illustration. The hypervisor drive acts as an abstraction layer to separate the virtual machine from the system hardware.

 

virtual machine

Source: https://www.padok.fr/en/blog/minikube-kubeadm-kind-k3s

Starting the cluster

To start the cluster, we can run the following command from a terminal with administrative access:

minikube start

In case Minikube fails to start, you can choose some other appropriate driver for setting up a compatible container or virtual machine manager.

As seen in the output, Minikube has used the docker driver for our case.

command prompt

Interacting with the cluster

We can interact with the created Kubernetes cluster using kubectl by running the following command:

kubectl get po -A

This displays all the components of the Kubernetes cluster that have been created like etcd – minikube, Kube-apiserver-minikube, Kube-controller-manager-minikube, Kube-scheduler-minikube, etc. These, as the name indicates, are the components that we had discussed in the previous sections.

interacting with cluster | kubernetes

Creating a sample deployment 

We can use the following command to create a deployment that manages a pod. This pod manages a container based on the Docker image provided.

kubectl create deployment hello-world --image=k8s.gcr.io/echoserver:1.4

We can view the deployment using the command below:

kubectl get deployments

To view the pods, we can run this command:

kubectl get pods
creating sample deployment

We can expose the deployment on a port:

kubectl expose deployment hello-world --type=NodePort --port=8080

To access the deployment, we can run the following command:

kubectl port-forward service/hello-world 7080:8080

 

About Author

Nibedita Dutta

Nibedita completed her master’s in Chemical Engineering from IIT Kharagpur in 2014 and is currently working as a Senior Consultant at AbsolutData Analytics. In her current capacity, she works on building AI/ML-based solutions for clients from an array of industries.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.
Nibedita Dutta
Nibedita Dutta

Nibedita completed her master’s in Chemical Engineering from IIT Kharagpur in 2014 and is currently working as a Senior Data Scientist. In her current capacity, she works on building intelligent ML-based solutions to improve business processes.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details