Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Introduction and a New Convection About Data Science

Y
YAMINI Last Updated : 26 Apr, 2021
5 min read
This article was published as a part of the Data Science Blogathon.

Introduction

Today Data Science has become a huge topic and also stands at the top of the list of jobs in the market. People learning it and doing it as well as demand has grown to peak stage. What is it that is making this job so desirable and wanted by many people and how has this got to this stage of huge demand.

The companies have come up with many new things also the data has increased enormously that people like to know what is it and one man or a group can’t do it. It requires a lot of work and crunching all the data to get insights from it has become the ultimate goal of the companies so that they can grow and bring many innovative products. All the companies, industries, fields, researchers everyone is trying to get things out of the data so that it brings a new era, new transformations, and developments in the respective domains.

Introduction to Data Science and related technologies involved in it

Data Science is the science of dealing with all the data growing in every field, industry, and domain. It is the process of extracting the data from structured, unstructured data using the Data mining technique (for getting the information from raw data). It is a lot of science involving getting the necessary information out from tons and millions of tons of products, services, and their data in it for making better products, developments, and many more. Data is never-ending and the same is for data science technologies and related things.

What are the technologies used in it? What is data science made up of?

data science venn diagram

The above picture gives the basic subjects to be known in data science. Then what are new technologies to be learned and get acquainted with into the broader picture of data science? The below picture gives how data science is dealing with new technologies to get the information from the data. It excites to be part of new technology and also can see a big mountain in front to cross and go to next stage.

Along with all these technologies a lit bit of domain knowledge/business knowledge would help in making better insights into the data. There is nothing impossible, it is the courage that counts. Small things make big differences.

data science 2 venn

Now let’s get into ML, What is Machine Learning?

Machine Learning is the method of dealing with data and automating the tasks by training it so that it gives new suggestions and detects when a similar type of data is provided to it. It comes under the AI umbrella and it identifies the data and makes the decisions faster and saves time and human effort so that there is less need for human intervention in it.

Did anyone get a question about automation and machine learning?

 

RPA vs ML

Artificial intelligence and RPA are different. RPA is a software robot that mimics human actions, whereas AI is the simulation of human intelligence by machines. RPA is a rule-based software that has no intelligence and automates repetitive tasks. RPA just does what task is assigned to it and decreases the time taken by humans whereas AI brings new things and evolves and yes RPA is also part of AI then it is not used in Data Science.

The machine learns about data and RPA just performs the repetitive tasks. Many people might have come across the bots and would have known what do they do. They just do the task given to them and still, RPA developers are helping data scientists make their work easier. Machine learning gives new insights from the data trained to it and in RPA it is not like that.

 

Machine Learning its Types and Use of it

Machine Learning is classified into Supervised, Unsupervised, Semi-supervised, and Reinforcement Learning. The first two play a major role in the data science industry and also the others come under Data Science. Supervised Algorithms deals with the labeled data, unsupervised with unlabeled data, Semi-supervised combines where both types of data exist (labeled and unlabelled), Reinforcement Learning or algorithms is like trial and error method where when the task is done the correct way it gets rewarded else it gets penalized or punished.

Supervised Learning is further divided into Classification and Regression problems, Unsupervised is divided into Clustering and Dimensionality Reduction problems. Some of the algorithms used in Supervised are Linear Regression, Logistic Regression, KNN, Decision Tree, Random Forest, SVM, Boosting techniques, etc. Some of them used in Clustering are Kmeans, Market Basket Analysis, DBSCAN(Density-based spatial clustering of applications with noise), Hierarchical clustering, etc and Dimensionality Reduction can be done using PCA, SVD, LDA, t-SNE…etc.

The below diagram describes the use and benefit of applying these algorithms to the data and also explains which type of algorithm or technique has to be used while solving the problem based on its requirements.

Machine Learning

Deep Learning

Deep Learning is the extension of ML also sometimes called advanced ML algorithms. Still is there is a lot of difference between ML. Deep Learning works based on neural networks as the human brain does. There are a lot of networks in Deep Learning. Some of the DL algorithms are CNN, RNN, LSTM’s, GAN, RBFN, MLP, SOM’s, DBNs, RBMs, Autoencoders and Decoders, etc.

NLP(Natural Language Processing) is mainly used in text extraction and understanding it.

All these technologies combine ML, DL, NLP comes under AI (Artificial Intelligence) umbrella.

 

Computer Programing Languages used for Data Science

Many programming languages can be used for Data Science. The most popular data science languages used nowadays are Python and R. Even though all the other programming languages can be used for data science. Python has become the most popular and data science language because of the in-built libraries it has for data science.

data science programming

Data Science has many topics and it has many things which can be applied everywhere and so this is making a huge impact everywhere and making the data easier to understand and use so that there are new developments and directly proportional to the growth of the country.

This is about the short description and introduction on Data Science and its need in this big data world which is evolving every day without any pause. Even in times of pandemic the Data science analysis and its use have increased a lot which has made things easy for storing and making analysis on the numbers(for counts, infected people, recoveries, deaths, and all ..). Data Science is such an important part because of which we can to predict and take necessary actions. Thanks to Data Science, data scientists, and everyone who is being a part of eradicating the virus from our country and world.

Let me know if you have any queries or anything to say about the article. Thanks for reading.👩‍🔬👩‍💻👩

Have a nice day. 🕊

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion. 

blogathondata science
Y
YAMINI

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details