Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

5 Statistical Tests Every Data Scientist Should Know

Aayush Tyagi
Aayush Tyagi Last Updated : 22 Jul, 2024
9 min read

Introduction

In data science, having the ability to derive meaningful insights from data is a crucial skill. A fundamental understanding of statistical tests is necessary to derive insights from any data. These tests allow data scientists to validate hypotheses, compare groups, identify relationships, and make predictions with confidence. Whether you’re analyzing customer behavior, optimizing algorithms, or conducting scientific research, a solid grasp of statistical tests is indispensable. This article explores the essential statistical tests every data scientist should know.

Table of contents

Role of Statistical Tests in Data science

  • Hypothesis validation: Statistical tests allow data scientists to objectively assess whether observed patterns in data are likely to be real or just due to chance.
  • Decision making: They provide a quantitative basis for making decisions, helping to remove subjectivity and gut feelings from the process.
  • Comparing groups: Tests enable meaningful comparisons between different groups or conditions in a dataset.
  • Identifying relationships: Many tests help uncover and quantify relationships between variables.
  • Model validation: Statistical tests are crucial in assessing the validity and performance of predictive models.
  • Quality control: They help in detecting anomalies or significant changes in data patterns.

5 Statistical Tests Every Data Scientist Should Know

Z-test

A z-test is a statistical test used to determine whether there is a significant difference between sample and population means or between the means of two samples when the variances are known and the sample size is large (typically n > 30). It is based on the z-distribution (also known as the standard normal distribution), which is a normal distribution with a mean of 0 and a standard deviation of 1.

Formula

For a single sample z-test, the test statistic (z) is calculated as:

z = (x̅ - μ) / (σ / √n)

Where:

  • x̅ is the sample mean.
  • μ is the hypothesized population mean.
  • σ is the population standard deviation (assumed to be known).
  • n is the sample size.

Steps for Conducting a Z-Test:

Here are the steps for conducting a z-test:

1. State your hypothesis:

  • Null hypothesis (H₀): This is the default assumption you aim to disprove. In a z-test, it typically states that there’s no significant difference between the means you’re comparing.
  • Alternative hypothesis (H₁): This is what you believe to be true and what the z-test will help you assess. It can be one-tailed (specifies a direction for the difference) or two-tailed (doesn’t specify a direction).

2. Choose your significance level (α): This value, denoted by alpha (α), represents the probability of rejecting the null hypothesis when it’s actually true (a type I error). Common choices for alpha are 0.05 (5%) or 0.01 (1%). A lower alpha indicates a stricter test, requiring stronger evidence to reject the null hypothesis.

3. Determine the appropriate z-test type: Select the z-test that aligns with your research question:

  • One-sample z-test: Compares one sample mean to a hypothesized value.
  • Two-sample z-test: Compares the means of two independent samples.
  • Z-test for proportions: Used for data in proportions (less common).

4. Calculate the test statistic (z-score): Use the appropriate formula. This calculation involves the sample means, hypothesized population mean (for one-sample test), standard deviations (or estimated values), and sample sizes.

5. Find the critical value (z_critical): Look up the z-critical value in a standard normal distribution table based on your chosen significance level (alpha).

6. Interpret the results: Compare the absolute value of your calculated z-statistic (|z|) to the z_critical value. If the absolute value of your z-statistic is greater than the critical value, reject the null hypothesis (evidence of a difference).If not, fail to reject the null hypothesis (insufficient evidence for a difference).

T-Test

T-test is a statistical test used to determine if there is a significant difference between the means of two groups. It helps to determine if the differences observed in sample data are likely to exist in the population from which the samples were drawn.

There are three main types of T-tests:

  • One-Sample T-test
  • Independent (Two-Sample) T-test
  • Paired Sample T-test

Formula:

The formula for a t-test depends on the specific type of t-test you’re performing:

1. One-sample t-test:

This formula compares the mean of one sample (x̅) to a hypothesized population mean (μ). It’s similar to a one-sample z-test but uses the sample standard deviation (s) instead of the population standard deviation.

t = (x̅ - μ) / (s / √n)

Where:

  • x̅ is the sample mean.
  • μ is the hypothesized population mean.
  • s is the sample standard deviation.
  • n is the sample size.

2. Independent (two-sample) t-test:

This formula compares the means of two independent samples (x̅₁ and x̅₂). It considers the separate sample standard deviations (s₁ and s₂).

t = (x̅₁ - x̅₂) / √(s₁² / n₁ + s₂² / n₂)

Where:

  • x̅₁ and x̅₂ are the means of the two samples.
  • s₁² and s₂² are the variances of the two samples (estimated from sample data).
  • n₁ and n₂ are the sizes of the two samples.

3. Paired t-test:

This formula compares the means of paired differences (d) between two related groups.

t = (d̅) / (s_d / √n)

Where:

  • d̅ is the mean of the paired differences.
  • s_d is the standard deviation of the paired differences.
  • n is the number of pairs.

Steps for Conducting a T-Test:

Here’s a breakdown of the steps to calculate a t-test:

  1. State your hypotheses:
    • Null hypothesis (H₀): This is the “no difference” scenario you aim to disprove.
    • Alternative hypothesis (H₁): This is what you believe might be true.
  2. Choose significance level (α): This is the probability of rejecting a true null hypothesis (usually 0.05).
  3. Identify the appropriate t-test type:
    • One-sample t-test (comparing one sample to a hypothesized mean).
    • Independent (two-sample) t-test (comparing means of two independent groups).
    • Paired t-test (comparing means of paired or related samples).
  4. Collect and organize your data: Ensure your data is numerical and ideally follows a normal distribution.
  5. Calculate the relevant statistics:
    • Depending on the chosen t-test type, calculate the mean, standard deviation, and sample size for each group (or for the single sample).
    • If using a paired t-test, calculate the mean and standard deviation of the differences between paired samples.
  6. Determine the degrees of freedom (df): This value depends on the sample size(s) and varies with the t-test type. Refer to a t-distribution table guide for calculating df.
  7. Calculate the t-statistic: Use the appropriate formula (refer to previous explanation of t-test formulas) based on your chosen t-test type.
  8. Find the critical value: Look up the t-value on a t-distribution table corresponding to your chosen significance level (α) and the degrees of freedom (df) you calculated in step 6.
  9. Interpret the results:
    • If the absolute value of your calculated t-statistic is greater than the critical value from the table, reject the null hypothesis (evidence of a significant difference).
    • If not, fail to reject the null hypothesis (insufficient evidence for a difference).

ANOVA (Analysis of Variance)

ANOVA, or Analysis of Variance, is a statistical method used to compare the means of three or more groups to determine if there are any statistically significant differences between them. There are 3 types of ANOVA tests:

  1. One-Way ANOVA: Compares the means of three or more independent (unrelated) groups based on one factor.
  2. Two-Way ANOVA: Compares the means of groups that are split on two factors and can show interaction effects between the factors.
  3. Repeated Measures ANOVA: Used when the same subjects are used for each treatment.

Steps in Conducting ANOVA

1. Formulate Hypotheses:

  • Null hypothesis (H₀): All group means are equal (µ₁ = µ₂ = µ₃ = … = µₖ).
  • Alternative hypothesis (H₁): At least one group mean is different.

2. Calculate Group Means and Overall Mean: Compute the mean of each group and the grand mean (overall mean of all observations).

3. Calculate Sums of Squares:

  • Total Sum of Squares (SST): Measures the total variation in the data.
  • Between-Group Sum of Squares (SSB): Measures the variation between the group means.
  • Within-Group Sum of Squares (SSW): Measures the variation within each group.

4. Calculate Degrees of Freedom (df):

  • df between groups (df₁): k – 1 (where k is the number of groups).
  • df within groups (df₂): N – k (where N is the total number of observations).

5. Compute Mean Squares:

  • Mean Square Between (MSB): SSB / df₁
  • Mean Square Within (MSW): SSW / df₂

6. Calculate the F-Statistic:

F = MSB / MSW

7. Determine the p-Value:

Compare the calculated F-value with the critical F-value from F-distribution tables based on the degrees of freedom and chosen significance level (usually 0.05).

8. Make a Decision:

If the p-value is less than the significance level, reject the null hypothesis (indicating that there are significant differences between group means).

F-Test

F-test is a statistical tool used to compare the variances of two normally distributed populations. It helps determine if there’s a statistically significant difference in how spread out the data is between the two groups.

Formula:

F = σ₁² / σ₂²

Where:

  • F is the F-statistic (test statistic).
  • σ₁² (sigma squared) is the variance of the first population / sample.
  • σ₂² (sigma squared) is the variance of the second population / sample.

Steps to Conduct F-Test:

  1. State the null and alternative hypotheses:
    • Null hypothesis (H₀): The variances of the two populations are equal (σ₁² = σ₂²).
    • Alternative hypothesis (H₁): The variances of the two populations are not equal (σ₁² ≠ σ₂²).
  2. Calculate the sample variances (s₁² and s₂²) for each group.
  3. Compute the F-statistic using the formula F = s₁² / s₂². Place the larger variance in the numerator to ensure a right-tailed test (more common scenario).
  4. Determine the degrees of freedom: This considers the sample sizes of both groups. You’ll need to look up F-critical values in a table based on these degrees of freedom and your chosen significance level (usually 0.05).
  5. Interpret the results:
    • If the F-statistic is greater than the F-critical value, you reject the null hypothesis and conclude there’s a significant difference in variances between the two populations.
    • If the F-statistic is less than or equal to the F-critical value, you fail to reject the null hypothesis. There’s not enough evidence to say the variances are statistically different.

Chi-Square Test

The Chi-Square test is a statistical method used to determine if there is a significant association between two categorical variables. It’s widely used in hypothesis testing to assess the goodness of fit or the independence between variables.

There are two types of Chi-Square Tests:

  • Chi-Square Test for Independence
  • Chi-Square Test for Goodness of Fit

Chi-Square Test for Independence

The Chi-Square Test for Independence is a statistical test used to determine if there’s a relationship between two categorical variables. Here’s a breakdown of the test and its formula:

Formula:

The Chi-Square test statistic (Χ², chi-squared) is calculated using the following formula:

X^2 = Σ ( (O - E)² / E )

Where:

  • Σ (sigma) represents summation across all categories (i x j, where i is the number of rows and j is the number of columns in the contingency table).
  • O = Observed frequency for a particular category combination.
  • E = Expected frequency for the same category combination (calculated based on the assumption of independence).

Steps to Calculate Chi-Square Test for Independence

  1. Create a contingency table: Fill it with observed frequencies for each combination of variable categories.
  2. Calculate expected frequencies: Consider the row and column totals and the overall sample size to determine what the expected frequencies would be if the variables were independent.
  3. Compute (O-E) for each category: Subtract the expected frequency from the observed frequency for each cell.
  4. Square (O-E) for each category.
  5. Divide (O-E)² by E for each category.
  6. Sum all the values from step 5. This sum is your Chi-Square test statistic (Χ²).

Interpretation:

  • A higher Chi-Square value indicates a stronger evidence against the null hypothesis (variables are independent).
  • You need to compare the Chi-Square statistic to a critical value from the Chi-Square distribution table based on the degrees of freedom (calculated as (number of rows – 1) * (number of columns – 1)) and your chosen significance level (usually 0.05).
  • If the Chi-Square statistic is greater than the critical value, you reject the null hypothesis and conclude there’s a relationship between the variables.

Chi-Square Test for Goodness of Fit

The Chi-Square Test for Goodness of Fit is a different application of the Chi-Square statistic used to assess how well a sample distribution fits a hypothesized probability distribution.

Formula:

Similar to the Chi-Square Test for Independence, the Goodness of Fit test statistic (Χ², chi-squared) is calculated using the following formula:

X^2 = Σ ( (O - E)² / E )

Where:

  • Σ (sigma) represents summation across all categories (i, where i is the number of categories).
  • O = Observed frequency for a particular category.
  • E = Expected frequency for the same category (calculated based on the hypothesized probability distribution).

Steps to Calculate Chi-Square Test for Goodness of Fit:

  1. Define the expected distribution: Specify the theoretical distribution you’re comparing your data to.
  2. Calculate expected frequencies: Based on the chosen distribution and its parameters, calculate how often each category should occur in your sample size.
  3. Create a table: Organize your observed data frequencies and the calculated expected frequencies.
  4. Compute (O-E) for each category. Subtract the expected frequency from the observed frequency for each category.
  5. Square (O-E) for each category.
  6. Divide (O-E)² by E for each category.
  7. Sum all the values from step 6. This sum is your Chi-Square test statistic (Χ²).

Interpretation:

  • A higher Chi-Square value indicates a stronger deviation from the hypothesized distribution.
  • You need to compare the Chi-Square statistic to a critical value from the Chi-Square distribution table based on the degrees of freedom (calculated as the number of categories minus 1) and your chosen significance level (usually 0.05).
  • If the Chi-Square statistic is greater than the critical value, you reject the null hypothesis (data follows the distribution) and conclude there’s a significant difference between your data and the hypothesized distribution.

Conclusion

In data science, statistical tests are essential tools for uncovering insights and making informed decisions. The z-test, t-test, ANOVA, F-test, and chi-square test each play a crucial role in analyzing different aspects of data. By mastering these tests, data scientists can confidently validate hypotheses, compare groups, and identify relationships within their data. Remember, the key to success lies not just in knowing how to perform these tests, but in understanding when and why to use each one. Armed with this knowledge, you’ll be well-equipped to tackle complex data challenges and drive data-driven decision-making in any field.

Aayush Tyagi
Aayush Tyagi

Data Analyst with over 2 years of experience in leveraging data insights to drive informed decisions. Passionate about solving complex problems and exploring new trends in analytics. When not diving deep into data, I enjoy playing chess, singing, and writing shayari.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details