Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Data Science- The Go-To Tool for Solving Daily Problems and Taking Better Decisions

Sarthak
Sarthak Last Updated : 06 Oct, 2020
6 min read
This article was published as a part of the Data Science Blogathon.

Introduction

How can Data Science, besides helping you professionally, improve the quality of decisions you make daily?

Data science decision making

Data Scientist as a term sounds so intimidating, right? Like someone who invents data-related stuff to serve humanity. No! I mean Yes, obviously some people have done their doctorates in Statistics or Computing and who constantly work hard to come up with algorithms and all. But, do you know that ‘Data Science’ is a very generalized term which has been defined in the 21ˢᵗ century but has been out there since forever.

In fact, in the year 1962, John W. Tukey wrote in his book “The Future of Data Analysis”:

“For a long time, I thought I was a statistician, interested in inferences from the particular to the general. But as I have watched mathematical statistics evolve, I have had cause to wonder and doubt. I have come to feel that my central interest is in data analysis, and the parts of statistics which adhere to it must take on the characteristics of science rather than those of mathematics.”

We use data to come to conclusions daily. I entered college three years ago and within a few days, subconsciously categorized people whether they would good or bad for me to know which ones should I befriend. This is what is known as Biased Analysis. It was such a novice mistake- using 10 Day data-points to predict the behavior of people, who have lived over 20 years. A very small sample indeed!

I am still learning these techniques and while I am doing it, I keep on thinking and searching about how I can use them to make my life better, besides helping me to fetch a job. 😛

We waste a lot of time thinking about what to watch when we sit to watch a Movie/TV Show, or when you have to buy something. Well, whatever we watch or buy, it should be worthy of the amount of time or money we spend on making that decision.

In this article, I will talk about what tools you can use to solve the kind of problems I am talking about and how a beginner can apply them.

 

Here’s how you can Data Science for Better Decision Making-

  1. Personal Movie Recommender System

    Data science decision making

    It’s 9 P.M. I am not in a mood to study. Let’s watch a movie. But, I have to sleep by 12 as well.

    Many a time, it happens that I want to watch a movie, and when I sit down to browse, I spend most of my time browsing and by the time I pick up something, it’s time to sleep.

    Netflix recommends you watch new movies based on your viewing history. But, there’s a limitation. Netflix can only recommend the movies that it has in its library. Let’s create a bigger list of movies we have watched and

    By making a personal movie recommender system, you can avoid all of this hassle. This may be a bit complicated for a beginner.

    But, if you have an idea of basic ML models you can try this out in a different way. Create a dataset by feeding-in all the movies you have watched along with their IMDB scores, genre, major actors, language, director, etc. and give them a personal rating out of 10. Use this personal rating as the target variable, pick a validation approach, and then use the appropriate modeling technique on it.

    In a similar manner, gather a list of all the movies that you want to watch and using the above-fitted model, try to get the predicted ratings on each of them!

    What’s next? Start watching these movies in descending order and enjoy it!

  2. Doing a Self-Analysis

     

    Data science decision making

    I have always been this guy who would avoid his emotions and would distract himself if something serious would pop into his head. The pandemic made me realize how important it is to face your emotions, scrutinize them and then let them go.

    So, I am thinking about creating a journal wherein I would write the majority of things I did throughout the day followed by a review of the day. You can do this too!

    Learn some basic techniques on how to analyse textual data, also known as Natural Language Processing or NLP and let’s try to put it into action. A sentimental analysis can be done to understand how you were feeling when you wrote that message and how often do you feel like that.

    Here is a comprehensive guide on how to perform sentiment analysis.

    Also, you wrote down what you did throughout the day, right? You can associate those set of activities with the emotions you feel at the end of the day and then understand what kind of activities make your day better.

  3. Creating a Chatbot to answer your friends when you’re busy

     

    When this COVID pandemic outbroke, everything got affected, offices had to be closed because we were all supposed to stay inside our homes and of course call centres and service centres were closed as well. With limited number of people present at their disposal, companies had to look for alternatives which could make their jobs easier. Many companies used chatbots to talk to us and record our problems. If the solution was easy and could be solved without any human assistance, the chatbot itself would recommend some solutions.

    You can also make a chatbot that answers for you when you’re busy. Let’s say that you’re at office and someone texts you asking where you are, you don’t have to leave your work and sit back to reply to that person. The chatbot that you program can handle that effectively for you.

    You can check this out to know more about Chatbots and how to implement them.

  4. Researching efficiently before buying anything

     

    Data science decision making

    You are thinking about buying a smartphone and you google “Best phones under 35k”. You open the first link and the blog lists 5 choices for you, and the confusion game starts. Also, it doesn’t talk about the features that matter to you.

    To be honest, only you know what matters to you and those blogs are written from a generic point of view. So how do you choose the best (for yourself) among thousands of different options?

    Let me tell how an analyst i.e. you could do that job. Prepare a list of all kinds of smartphones available in the market along with the latest prices of the same. You could do this activity efficiently if you are aware of Web Scraping Techniques.

    Now, you have a structured dataset in front of you! What are you waiting for? Start Digging! Apply all sorts of filters on Battery, Storage, Price etc. and viola! You have ‘Best smartphones under 35k.’

    For these shortlisted smartphones, you can gather the top 500 or 1000 reviews and then analyse using NLP techniques.

    Here is how you can do that.

  5. Investment

    You have a great job but it is a task for you to save money. And to be very honest, your money will not multiply if you keep the majority of it in your Savings Account. To be honest, you will not be able to catch up with the inflation rate.

    Stock Investing sounds very fancy right? By the way, it’s not just for investment bankers. Anyone can do it! If you choose the right set of stocks, in just one month, you could earn an amount of money equivalent to what you would get had you put this money in a Savings Bank account.

    People do it by conventional methods- qualitative analysis, looking at charts etc. But, you’re comfortable with computers and have analytical skills too. Why don’t you try Algorithmic Trading? Where your computer does most of your job, analyzing the historical data and picking the right time to invest your money.

    Pick up various stocks, diversify your portfolio and make efficient use of your Data skills.

 

End Notes

These are just a few of the tangible things that can be done using Data Analytics. But, there are numerous benefits that you won’t even realize you’re gaining when you’re learning these skills.

You are developing structural thinking where nothing is chaotic anymore. Making rational decisions would become a part of your life. Indecisiveness would be a thing of the past.

Let’s keep this discussion going- Talk about how Data Science, besides helping us professionally, would help us in our day to day tasks?

You can connect with me by clicking here. Let’s talk about things that can be achieved with Data Science!

Sarthak
Sarthak

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Apurva
Apurva

Very informative

123

Prageesha Chawla
Prageesha Chawla

Such an informative article Sarthak ! I really liked how you described the use cases that we encounter in our daily routines. A great article for newbies and professionals as well.

123

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details