Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Getting Started With Docker Image

P
Paul Last Updated : 23 Jun, 2022
7 min read

This article was published as a part of the Data Science Blogathon.

Introduction on Docker Image

I created this flow diagram for you to get a simpler idea before deep diving into docker. By looking at this diagram you can understand what is really happening while we are using docker. After docker is installed I simply call the docker run command, docker is looking for the image in our local machine, if it is installed, it runs that image, if not Docker search for the image from our docker hub (You will learn about docker hub in the following article). Once the image is found by docker it downloads the image to our local machine. Then it creates a local container and starts the program. Do you think it is looking like a software installation? Logically yes, but technically not. You will get the answer after reading this article.

Docker

Do We Really Need Docker?

Before getting into the Docker, let’s look at the traditional system we are using instead of docker and understand why we are moving to Docker.

Do We Really Need Docker

In this picture we can see the traditional web browsing method, let’s think about what will happen if millions of user access the same server at once, the server gets slower and slower right. To solve this issue let’s see the classic strategy.

Do We Really Need Docker 2

Problems that arise in this strategy are, that if one server or hardware shutdown for any reason the user will face difficulties. To solve this there were multiple solutions came, and one of the best solutions is the docker.

Do We Really Need Docker 3

The reason for moving the virtual machine to docker is, that the virtual machines completely running with a full operating system, which requires high memory requirements.

If we use docker Containers rather than Virtual machines.

  1. Lightweight
  2. Isolation
  3. Quick deployment
  4. Re-usability

What is Docker?

Wow! finally we arrive into our topic. Docker is an open source platform for building, deploying, and managing containerized applications. This ecosystem around creating and running containers. Docker is a software using as a container runtime.

What is Docker?

For example, the ecosystem contains:
1. Docker Client
2. Docker Server
3. Docker Machine
4. Docker Images
5. Docker Hub
6. Docker Compose

Containers simplify the delivery of distributed applications and have become increasingly popular as organizations shift to cloud-native development and hybrid multi-cloud environments.

Simply says, Docker makes it really easy to install and run software without worrying about setup or dependencies.

Simple Architecture

Simple Architecture of Docker

Here the client talks to the server (docker ps). If the client needs a particular image it will ask that image (docker pull ) to the server, the server will find the image in the registry and give it back to the client. (The registry contains images uploaded by various developers freely)

Simple Example for  Client-server Methodology

Simple Example for Docker Client-server Methodology
Simple Example for Docker Client-server Methodology 2

What is Container?

  1. Actual problem in normal system
What is Container?

2. Solution in normal system

2. Solution in normal system

3. How the Container Works

How the Container Works
How the Container Works 2
How the Container Works 3

Install Docker

Windows: https://docs.docker.com/desktop/windows/install/

Ubuntu: https://docs.docker.com/engine/install/ubuntu/

Mac: https://docs.docker.com/desktop/mac/install/

Docker Layers, Create Image and Sharing

Docker Layers, Create Image and Sharing
Docker Layers, Create Image and Sharing 2

Here you can see the IMAGE ID, it is considered the top latest id of the created image. The docker always updates the latest layer to the image. Exactly what it means is, take an image (layer) and do modifications, finally, you will get the modified image (top most layer). For every modification docker is giving different ids to its layer, so the image is getting the latest layer id at last.

Docker Layers, Create Image and Sharing 3

Last added layer should be at the top
Docker Layers, Create Image and Sharing 4
Copy the AB image and Create C image

Add C image layer at the top of AB layer
Add C image layer at the top of AB layer 4
Now the C image on the top of the AB layer

You can simply understand the layered mechanisms that we are using in Photoshop.

Now we will create one website and run it via one server.

  1. Pull a server (httpd image) — Here I already have this image
Pull a server (httpd image) — Here I already have this image

2. Create a container with a name & run it in a detached mode (background process), to access via the local machine we need to expose our port number. The Default web server port is 8080, but I am forwarding it to 9090.

Pull a server (httpd image) — Here I already have this image 2

Then we can check our web server in localhost:9090

Pull a server (httpd image) — Here I already have this image 4
Pull a server (httpd image) — Here I already have this image 5

We can see the index.html file inside our docker image

We can see the index.html file inside our docker image

Let’s create a new mypage.html. once created to exit from the bash use ctrl + D

Let’s create a new mypage.html. once created to exit from the bash use ctrl + D
Local host

Now we can see the layer modification by using the docker diff command. C means changed, A means added

Docker file

Note: It is a big disadvantage because while we are doing modification all the previous unwanted layers are also behind the layer. To overcome this issue we can use the docker file. Before going to the docker file, By using this we will create the image.

Let’s create a new myp ge.html. once created to exit from the bash use ctrl + D
Let’s create a new mypage.html. once created to exit from the bash use ctrl + D

Now, stop the web server, and create the new container by using the same image, with a different port.

Let’s create a new mypage.html. once created to exit from the bash use ctrl + D 4

To identify the image in the future, let’s give the tag & repository name.

Share the  Image by Saving the File

Now we can share our images with our friends by saving and redirecting the file.

Share the Docker Image by saving the file
Share the Docker Image by Uploading the Docker Hub

Alternatively, we can save image by using the following command

Share the Docker Image by Uploading the Docker Hub 1

After saving the file we can share the file with the friend. Once the friend receives the file he can load the file using the following command

Share the Docker Image by Uploading the Docker Hub 2

Share the  Image by Uploading the Docker Hub

  1. Create an Account in hub.docker.com
  2. Create a repository at the hub.docker.com — can push the docker image that you created
Share the Docker Image by Uploading the Docker Hub

You should use the same repository name that you created earlier.

Image 1
Image 2

After creating your repository you will get the command to upload.

docker push lmcshub/myhttpd:v1 (this is just an example command)

We can see the index.html file inside our docker image 2
We can see the index.html file inside our docker image 3

You have to log in to the docker account using the docker login command. You will ask for the username & password. After login success, use docker push lmcshub/myhttpd:v1 (this is just an example command), here you will get an error because the container name is different from the docker hub repository. So we will need to create the container with that name (the prefix — lmcshub/ is used to avoid the name duplication from different accounts) with the same image. Finally push it will work.

Conclusion on Docker Image

I hope that you learned why we are using docker, why we moved to docker from our traditional systems, how to install docker, how to pull images, and how to run containers. From my simple example, you also got clear knowledge about docker commands. So please install docker and ready for my next article. My next article will be on Docker Using Docker file.

What we learned So far on Docker Image

  • Docker images and containers.
  • Traditional Strategies we used before Docker and the Drawbacks.
  • Push and Pull Images from Docker Hub.
  • Docker architecture and Layered principle.

 The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

P
Paul

I am a Software Engineer working in Arimac Digital in Sri Lanka. I graduated from the University of Kelaniya with a BSc (Hons) in Software Engineering. Also specialized in Business Intelligence (Data Science) and web development. I am much interested in learning new technologies.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details