Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

A Quick Overview of Data Science Universe

Sriniketh
Sriniketh Last Updated : 28 Jan, 2021
7 min read

This article was published as a part of the Data Science Blogathon.

What is Data Science (DS)?

Data Science is a blend of various fields like Probability, Statistics, Programming, Analysis, Cloud Computing, etc; which are used to extract value from the data provided. It is a vast field which is a  booming field and all people are learning these skills to become a professional in this domain. A skilled person in all the above-mentioned sub-domains of data science is called a Data Scientist. He/She is a very skilled person in all the areas of DS.

 

Roles in Data Science:

Data science universe role

Roles of a Data Scientist

These are the major roles in Data Science. So it’s clear from the venn that a person has an opportunity to become either a Data Engineer, Data Scientist or Data Engineer, etc;. So which means there are enormous choices and opportunities.

 

Who is a Data Scientist?

A data scientist is a professional responsible for collecting, analyzing, and interpreting extremely large amounts of data. The data science role is entirely different from several traditional technical roles, including mathematician, scientist, statistician, and computer professional. This job requires the use of advanced analytics technologies, including machine learning and predictive modeling.

A data scientist requires large amounts of data to develop hypotheses, make inferences, and analyze customer and market trends. Basic responsibilities include gathering and analyzing data, using various types of analytics and reporting tools to detect patterns, trends, and relationships in data sets.

In business, data scientists typically work in teams to mine for information that can be used to predict customer behavior and identify new revenue opportunities. In many organizations, data scientists are also responsible for setting best practices for collecting data, using analysis tools, and interpreting data.

The demand for data science skills has grown significantly over the years, as companies look to glean useful information from big data, the voluminous amounts of structured, unstructured, and semi-structured data that a large enterprise or IoT produces and collects.

 

Types of Data:

A person who knows a bit of programming would have known about data-types. But data-types are different from Types of Data as the data-types deal with programming and the latter deals with DS.

There are two types of Data:

-> Structured Data,

-> Unstructured Data.

Structured Data:

data science universe data types

Structured Data

The above image looks very cozy and unknown things mentioned, but let’s not bother about it, but what’s clear from the above image is that data is represented in a table and they are sorted and are grouped. So, therefore, structured data is one that is represented in a table format or any other easily distinguishable form, from where we can retrieve data and it is highly sorted. It may or may not have any outliers.

 

Unstructured Data:

data sciecne universe Unstructured Data

Unstructured Data

Let’s just analyze the points plotted in the graph without considerations of the scale, what’s on the X or Y axis etc; In the graph, we have a point named X(in red). So this is the point where the data is supposed to lie. With respect to this point then we can find that very few are in close range with this point.

So similarly when it comes to data when we may have all the data which is absolutely correct but they may be randomized. So with respect to the above example, unstructured data is one that is not sorted and is very much randomized from which analyzing the data is difficult. It will have many outliers.

And in the recent decade, most of the data is unstructured and is quite clear from the graph. So it is very much necessary to know the difference between structures and unstructured data.

trends

Trends

Role of Machine Learning in DS:

As said above, DS is a vast domain and has a correlation with many fields and one such field is Machine Learning(ML). So let’s look at the impact of ML in DS:

>For Pattern Finding: The ultimate goal with the data we have is to predict the pattern and users’ choice and then give a conclusion regarding the particular data model available. The companies are mainly focused on pattern-finding as this helps to find the users likings say for a particular period and the company mainly focuses on that kind of products which are liked by the customers and give offers to improve and increase their business.

> For making Predictions: So from a given data set we are required to find the desired output like a future trend, so we can make use of ML algorithms as these are best suited. This is based on the concept of supervised learning. As humans learn from their mistakes it is the same in the case of machines too. So with the help of an already available data model, we would train the machine to perform a specific task and this will help the machine to learn and would feature itself in the task by repeatedly performing the same task.

Many people usually get confused with data science and data analysts. So with a given tabulation, it is easily distinguished.

 

Machine Learning in DS

DS vs Data Analyst

So hope the tabulation makes the point clear.

Business Intelligence(BI):

From the skills which a Data Analyst must possess we came across the term BI. Therefore,

BI is a set of processes, architectures, and technologies that convert raw data into meaningful information that drives profitable business actions.It is a suite of software and services to transform data into actionable intelligence and knowledge. So it mainly deals with the intention to strive the business in the best possible way. Many tech-giants are really using this technology to boost their business.

 

Domains of Data Science:

As mentioned earlier DS is a vast domain and has many co-relations with many other fields and there is a wide range of opportunities available if you are a really skilled person.

domain data science

Domains

The vast nature makes many people in different fields have a hand at DS as it helps them boost their career.

To get a more clear understanding of DS it is necessary to know the life cycle of DS.

 

Life Cycle of Data Science:

data science life cycle

1 -> Discovery: Before starting the project, it is necessary to know the requirements, priority, budget, specifications and budget, etc;. We must also prepare an initial hypothesis, understand the problem and get a clear idea of the project beforehand so that we are not stuck in the middle of the project.

2 – > Data Preparation: In this phase, we need to get the required data model and the data sets to perform Data Analytics for the whole project. We need to perform ETLT[Extract, Transform, Load, and Transform] on the available data to keep the data ready for the next stage.

3 -> Model Planning: We would develop models and techniques to draw relationships with the data available. These relationships will be the building blocks that will be useful in the later stages.

There are various tools to perform Model Planning:

 

Life Cycle of Data Science: automl

These are the various model planning tools in which a few are open platforms whereas others are paid and these tools can be used to perform model planning.

4 -> Model Building: This uses a set of algorithms being applied on the previous fetched results and tries to interpret the patterns and predict the future trend. This acts as a base for the whole business as it helps the companies to depict their growth in the upcoming years. With the help of existing data sets, we would train the machines and predict the model. We would learn association, classification, and clustering, etc; in this stage.

5 -> Communication Results: In the penultimate stage, we are required to interpret the results obtained with the required results made in the initial stages. If the results match then we are on the right track and the goal is achieved 90%.

6 -> Operationalise: In the ultimate stage if the goal is achieved the model is being made to operationalize in real life and tested. If the desired output is achieved then the goal is achieved cent percent. And it is also necessary to maintain the data model for future reference.

Major areas of Data Science:

 

Major Areas

Major Areas

Hope to see many Data Scientists in the near future.

Thanks to one and all for reading my blog and hope this helps. Positive or negative comments are appreciated because this is my first blog and everyone learns from mistakes :).

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Sriniketh
Sriniketh

A seasoned software and ML developer with likes to share knowledge on the latest skills, frameworks, and technologies. Writes about Data Science and Machine learning and love to build and ship projects.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Sriniketh J
Sriniketh J

Very nice and quick intro to DS and very tempting to do DS

123

Hariprasad
Hariprasad

Fundamentals are well defined and good pictorial represntation to make the concepts understand with ease.

123

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details