Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

K Means Clustering in Python | Step-by-Step Tutorials for Clustering in Data Analysis

Pranshu Sharma
Pranshu Sharma Last Updated : 14 Oct, 2024
8 min read

Introduction

K Means is one of the most popular Unsupervised Machine Learning Algorithms used for solving classification problems in data science, making it a crucial skill for those aspiring to excel in a data scientist role. K Means segregates unlabeled data into various groups, known as clusters, by identifying similar features and common patterns within the dataset. This tutorial aims to provide a comprehensive understanding of clustering, with a specific focus on the K Means clustering algorithm and its implementation in Python. By delving into the nuances of K means clustering in Python, you will gain valuable insights into how to effectively organize and analyze data. Additionally, the tutorial will guide you on determining the optimum number of clusters for a dataset, enhancing your ability to apply K means clustering in practical scenarios.

K Means Clustering in python image

Learning Objectives

  • Understand what the K-means clustering algorithm is.
  • Develop a good understanding of the steps involved in implementing the K-Means algorithm and finding the optimal number of clusters.
  • Implement K means Clustering in Python with scikit-learn library.

This article was published as a part of the Data Science Blogathon.

Table of contents

What Is Clustering?

Suppose we have N number of unlabeled multivariate datasets of various animals like dogs, cats, birds, etc. The technique of segregating these datasets into various groups on the basis of having similar features and characteristics is called clustering.

The groups being formed are known as clusters. Clustering techniques are used in various fields, such as image recognition, spam filtering, etc. They are also used in unsupervised learning algorithms in machine learning, as they can segregate multivariate data into various groups, without any supervisor, on the basis of common patterns hidden inside the datasets.

What Is K-Means Clustering Algorithm?

The k-means clustering algorithm is an Iterative algorithm that divides a group of n datasets into k different clusters based on the similarity and their mean distance from the centroid of that particular subgroup/ formed.

K, here is the pre-defined number of clusters to be formed by the algorithm. If K=3, It means the number of clusters to be formed from the dataset is 3.

Implementation of the K-Means Algorithm

The implementation and working of the K-Means algorithm are explained in the steps below:

Step 1: Select the value of K to decide the number of clusters (n_clusters) to be formed.

Step 2: Select random K points that will act as cluster centroids (cluster_centers).

Step 3: Assign each data point, based on their distance from the randomly selected points (Centroid), to the nearest/closest centroid, which will form the predefined clusters.

Step 4: Place a new centroid of each cluster.

Step 5: Repeat step no.3, which reassigns each datapoint to the new closest centroid of each cluster.

Step 6: If any reassignment occurs, then go to step 4; else, go to step 7.

Step 7: Finish

What is K-Means clustering method in Python?

K-Means clustering is a method in Python for grouping a set of data points into distinct clusters. The goal is to partition the data in such a way that points in the same cluster are more similar to each other than to points in other clusters. Here’s a breakdown of how to use K Means clustering in Python:

Import Libraries:

  • First, you need to import the necessary libraries. In Python, the popular scikit-learn library provides an implementation of K-Means.
from sklearn.cluster import KMeans

Prepare Your Data:

  • Organize your data into a format that the algorithm can understand. In many cases, you’ll have a 2D array or a pandas DataFrame.
import numpy as np
data = np.array([[1, 2], [5, 8], [1.5, 1.8], [8, 8], [1, 0.6], [9, 11]])

Choose the Number of Clusters (K):

  • Decide on the number of clusters you want the algorithm to find. This is often based on your understanding of the data or through techniques like the elbow method.
kmeans = KMeans(n_clusters=2)

Fit the Model:

  • Train the K-Means model on your data.
kmeans.fit(data)

Get Results:

  • Once the model is trained, you can get information about the clusters.
# Get the cluster centers
centroids = kmeans.cluster_centers_

# Get the labels (cluster assignments for each data point)
labels = kmeans.labels_

In this example, n_clusters=2 indicates that we want the algorithm to find two clusters. The fit method trains the model, and then you can access information about the clusters, such as the cluster centers and labels. Visualizing the results can be helpful to see how well the algorithm grouped your data points.

How K Means Clustering in Python Works?

Here is Step-by-Step Explanation that How K-means Clustering in Python works:

Initialize Centroids:

  • Randomly choose K data points from the dataset to be the initial centroids. K is the number of clusters you want to create.

Assign Data Points to Nearest Centroid:

  • For each data point in the dataset, calculate the distance to each centroid.
  • Assign the data point to the cluster whose centroid is the closest (usually using Euclidean distance).

Update Centroids:

  • Recalculate the centroids of the clusters by taking the mean of all the data points assigned to each cluster.

Repeat:

  • Repeat steps 2 and 3 until convergence. Convergence occurs when the centroids no longer change significantly or after a predefined number of iterations.

Final Result:

  • The algorithm converges, and each data point is assigned to one of the K clusters.

Here’s a simple example using Python with the popular machine learning library, scikit-learn:

from sklearn.cluster import KMeans
import numpy as np

# Sample data
data = np.array([[1, 2], [5, 8], [1.5, 1.8], [8, 8], [1, 0.6], [9, 11]])

# Specify the number of clusters (K)
kmeans = KMeans(n_clusters=2)

# Fit the data to the algorithm
kmeans.fit(data)

# Get the cluster centroids and labels
centroids = kmeans.cluster_centers_
labels = kmeans.labels_

print("Centroids:")
print(centroids)
print("Labels:")
print(labels)

Diagrammatic Implementation of K-Means Clustering

Step 1: Let’s choose the number k of clusters, i.e., K=2, to segregate the dataset and put them into different respective clusters. We will choose some random 2 points which will act as centroids to form the cluster.

Step 2: Now, we will assign each data point to a scatter plot based on its distance from the closest K-point or centroid. It will be done by drawing a median between both the centroids.

Step 3: points on the left side of the line are near the blue centroid, and points to the right of the line are close to the yellow centroid. The left forms a cluster with the blue centroid, and the right one with the yellow centroid.

Step 4: Repeat the process by choosing a new centroid. To choose the new centroids, we will find the new center of gravity of these centroids, as depicted below.

Step 5: Next, we will reassign each data point to the new centroid. We will repeat the same process as above (using a median line). The yellow data point on the blue side of the median line will be included in the blue cluster.

K Means Clustering in python 2

Step 6: As reassignment has occurred, we will repeat the above step of finding new k centroids.

K Means Clustering step 6

Step 7: We will repeat the above process of finding the center of gravity of k centroids, as depicted below.

K Means Clustering in python step 7

Step 8: After finding the new k centroids, we will again draw the median line and reassign the data points, like the above steps.

K Means Clustering step 8

Step 9: We will finally segregate points based on the median line, such that two groups are being formed and no dissimilar point is to be included in a single group.

K Means Clustering step 9

The final cluster formed is like this:

Cluster, k means clustering

Choosing the Optimal Number of Clusters

The number of clusters that we choose for the algorithm shouldn’t be random. Each and every cluster is formed by calculating and comparing the mean distances of each data point within a cluster from its centroid.

We can choose the right number of clusters with the help of the Within-Cluster-Sum-of-Squares (WCSS) method. WCSS stands for the sum of the squares of distances of the data points in each and every cluster from its centroid.

The main idea is to minimize the distance (e.g., euclidean distance) between the data points and the centroid of the clusters. The process is iterated until we reach a minimum value for the sum of distances.

Elbow Method

Here are the steps to follow in order to find the optimal number of clusters using the elbow method:

Step 1: Execute the K-means clustering on a given dataset for different K values (ranging from 1-10).

Step 2: For each value of K, calculate the WCSS value.

Step 3: Plot a graph/curve between WCSS values and the respective number of clusters K.

Step 4: The sharp point of bend or a point (looking like an elbow joint) of the plot, like an arm, will be considered as the best/optimal value of K.

Python Implementation:

Importing relevant libraries

import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()
from sklearn.cluster import KMeans

Loading the data

data = pd.read_csv('Countryclusters.csv')
data
data

Plotting the data

Python Code for K-Means Clustering

mport pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv('Countryclusters.csv')

print(data.head())

plt.scatter(data['Longitude'],data['Latitude'])
plt.xlim(-180,180)
plt.ylim(-90,90)
plt.show()
plotting data

Selecting the feature

 x = data.iloc[:,1:3] # 1t for rows and second for columns
x
select feature

Clustering

kmeans = KMeans(3)
means.fit(x)

Clustering results

identified_clusters = kmeans.fit_predict(x)
identified_clusters
array([1, 1, 0, 0, 0, 2])
data_with_clusters = data.copy()
data_with_clusters['Clusters'] = identified_clusters 
plt.scatter(data_with_clusters['Longitude'],data_with_clusters['Latitude'],c=data_with_clusters['Clusters'],cmap='rainbow')
plot cluster

WCSS and Elbow Method

wcss=[]
for i in range(1,7):
kmeans = KMeans(i)
kmeans.fit(x)
wcss_iter = kmeans.inertia_
wcss.append(wcss_iter)

number_clusters = range(1,7)
plt.plot(number_clusters,wcss)
plt.title('The Elbow title')
plt.xlabel('Number of clusters')
plt.ylabel('WCSS')
cluster

This method shows that 3 is a good number of clusters.

Conclusion

To summarize everything that has been stated so far, k means clustering in python is a widely used unsupervised machine learning technique that enables the grouping of data into clusters based on similarity. It is a simple algorithm that can be applied to various domains and data types, including image and text data. k-means can be used for a variety of purposes. We can use it to perform dimensionality reduction also, where each transformed feature is the distance of the point from a cluster center.

Key Takeaways

  • K-means is a widely used unsupervised machine learning algorithm for clustering data into groups (also known as clusters) of similar objects.
  • The objective is to minimize the sum of squared distances between the objects and their respective cluster centroids.
  • The k-means clustering algorithm is limited as it can not handle complex and non-linear data.

Frequently Asked Questions

Q1. What is meant by n_init in k-means clustering?

A. n_init is an integer and represents the number of times or the number of iterations the k-means algorithm will be run independently.

Q2. What are the advantages and disadvantages of K-means Clustering?

A. Advantages of K-means Clustering include its simplicity, scalability, and versatility, as it can be applied to a wide range of data types. Disadvantages include its sensitivity to the initial placement of centroids and its limitations in handling complex, non-linear data. k-means is also sensitive to outliers.

Q3. What is meant by random_state in k-means clustering?

A. In K-Means, random_state represents random number generation for centroid initialization. We can use an Integer value to make the randomness fixed or constant. Also, it helps when we want to produce the same clusters every time.

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

blogathonK Means AlgorithmK means clusteringk means clustering in python
Pranshu Sharma
Pranshu Sharma

Aspiring Data Scientist | M.TECH, CSE at NIT DURGAPUR

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Norman
Norman

In part 2 - "What is K Means Algorithm" - you forgot a very important step, which is to determine the new cluster center by computing the average of the assigned points

123

imran hussain
imran hussain

hi it is a very good article. Please can you also write an article with coding how to choose cluster head based on battery power in each cluster?

123

Hosein
Hosein

thanks for your helpful article I just wanted to say there are some parts that needs to be corrected : first one : 5. Python Implementation >> Clustering >> means -> kmeans second one : in section "Trying different method ( to find no .of clusters to be selected) WCSS and Elbow Method" after "for" loop indent is forgotten

123

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag
Congratulations, You Did It!
Well Done on Completing Your Learning Journey. Stay curious and keep exploring!
K-Means: Getting the Optimal Number of Clusters
K-Means: Getting the Optimal Number of Clusters
Understanding K-means Clustering in Machine Learning(With Examples)
Understanding K-means Clustering in Machine Learning(With Examples)

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details