Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

What are the odds that you know about the odds

V
Vishwanath Last Updated : 16 Nov, 2023
8 min read

This article was published as a part of the Data Science Blogathon

Introduction

Reading the title, if you have guessed that the article is all about odds, you are right. This article is all about odds and its variants – log of odds, odds ratio, etc. I am not sure about you, but I have always been confused by the terms odds, odds ratio, etc. These terms were taught using lengthy formulas and whatnot. If you are as confused as I was, this article will clear all your confusion.

You might have come across the term odds while studying probability or statistics. It is also famous in the betting, horse racing industries, etc. You might have heard the sentence, “What are the odds, that my horse will win the race” or “What are the odds that I will win a lottery”. Odds are nothing but chance, when someone says, “What are the odds”, one can interpret it as “What are the chances”.

Table of contents

Odds:

Odds are basically the ratio of some event happening to some event not happening. It can also be defined as the ratio of the probability of an event happening to the Probability of the event not happening. Odds can be expressed as a Ratio or a Fraction.

odds

Now one should also note that Odds should not be confused with Probability. Probability is the ratio of an event happening to the total number of events (Event Happening + event Not Happening). Odds can be derived from Probabilities and vice versa. But it is especially important not to confuse odds with probabilities.

probability formula

We can consider an example to demonstrate the difference between Odds and Probability. Consider a team that played 100 matches and won 25 of them and lost 75 of them. Now we can calculate the Odds and Probabilities as follows,

odds team winning | odds and odd ratios

We can say that the Odds in favor of the team winning are 1:3 or 1/3 or 0.333. Since we have odds in favor of the team winning, we also have odds against the team winning which is the “multiplicative inverse” of the odds in favor of the team winning. As you might have worked it out, the odds against the team winning are 3:1 or 3/1 or 3.

odds team loosing

We can also calculate the probability of the team winning and losing as follows,

calculate probability | odds and odd ratios

As I had mentioned that Odds can also be calculated from the probabilities, we will see how it is done below,

final formula | odds and odd ratios

Few things to keep in mind about the odds,

  1. Odds are the ratio of the count of events happening to the count of events not happening.
  2. Odds can range from 0 to infinity, whereas probabilities range from 0 to 1.
  3. Odds are not probabilities.
  4. Odds can be calculated from Probabilities and vice versa.
  5. Odds can be in favor of an event happening or against the event happening.

Log Odds:

As we have seen in the example above, the odds in favor of the team winning were 0.33, and the odds against the team winning were 3. This is just a simple case of 100 matches being played, consider a hypothetical situation where 1000 matches are played, and a team won only 25 of those and lost 975. In such cases, the odds in favor of the team winning will be 0.0256 and the odds against the team winning (odds in favor of team losing) will be 39. Because of such a large gap in the magnitude of the two odds, it becomes necessary to normalize it. This is mainly the reason why we log transform the odds. One more advantage of Log transforming odds is that once transformed, the distribution turns out to be symmetrical and this is helpful in the case of binary classification problems.

If you consider the above example, after log transforming, the odds will be as follows,

log odds

This can also be calculated using the probabilities as below,

log odds with probabilities | odds and odd ratios

It is also interesting to note that the formula that we just saw to calculate the Log of odds using the probabilities can also be written as below,

log odds p

If you have seen the above formula somewhere, then you are right, it is the Logit transformation formula that we use in Logistic Regression. The need to Log transform the odds will be further illustrated using the example below.

Consider a team that played 1000 matches, and if we consider 1000 possibilities of the team winning and losing (both adding up to 1000) and then calculate the corresponding odds, and plot the graph of log odds, it will resemble the normal distribution. This is implemented using the python code below,

# importing necessary libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
# initializing the Win and Lose lists
Win = list(range(1,1000,1))
Lose = list(range(999,0,-1))
# creating empty data frame
df = pd.DataFrame()
# initializing the columns of the data frame with the two lists
df['Win'] = Win
df['Lose'] = Lose
# calculating the odds of winning and losing
df['Odds_Win'] = df['Win']/df['Lose']
df['Odds_Lose'] = df['Lose']/df['Win']
# calculating the log of odds of winning and Losing
df['Log_Odds_Win'] = np.log(df['Odds_Win'])
df['Log_Odds_Lose'] = np.log(df['Odds_Lose'])
# plotting the log odds of winning
sns.displot(df['Log_Odds_Win'],kde=True);
plt.title("Distribution of Log of Odds in Favor of Winning");
plt.xlabel("Log Odds");
plt.ylabel("Count");

You can see the histogram generated below,

distribution of log odds

Odds Ratio:

As the name suggests Odds Ratio is just a “Ratio of Two Odds”. Although “Odds” is also a ratio, “Odds” and “Odds Ratio” are not the same. Odds are the ratio of an event happening to an event not happening, but Odds Ratio is the ratio of two odds (Odds1 and Odds2). The Odds ratio is an important concept that is useful while interpreting the output of the Logistic Regression algorithm, it also measures the association between events. It is especially important to differentiate the terms “Odds” and “Odds Ratio” and not to get confused between the two. The Odds ratio is expressed by the formula given below ( using the probabilities ).

odds ratio | odds and odd ratios

Log of Odds Ratio:

As we have seen in the case of Odds, values of the Odds ratio range from 0 to infinity. When the numerator in an odds ratio is lesser than the denominator, the value of the Odds Ratio is less than 1 and when the numerator is greater than the denominator, the value is greater than 1 ( Up to infinity). Similar to Odds values, since there is a chance that the magnitude of these two values will be different, it is convenient if we normalize the ratio using Log normalization. Once we do that, the distribution of the odds ratios becomes normal. Log of odds ratio can be defined using the formula below,

log of odd ratios

Just for explaining the concept of Log of Odds Ratios, using the problem of win and lose described above, we can also calculate and plot the Log of Odds ratio using Python as below,

# calculating odds ratios
df['Odds_Ratio_Win_Lose'] = df['Odds_Win']/df['Odds_Lose']
df['Odds_Ratio_Lose_Win'] = df['Odds_Lose']/df['Odds_Win']
# calculating log of odds ratios
df['Log_Odds_Ratio_Win_Lose'] = np.log(df['Odds_Ratio_Win_Lose'])
df['Log_Odds_Ratio_Lose_Win'] = np.log(df['Odds_Ratio_Lose_Win'])
# plotting the log odds of winning
sns.displot(df['Log_Odds_Ratio_Win_Lose'],kde=True);
plt.title("Distribution of Log of Odds Ratio Win Over Lose");
plt.xlabel("Log Odds Ratio");
plt.ylabel("Count");

The histogram of Log of Odds ratios is as below,

distribution

Use case of Odds Ratios:

  1. In medical terms, the Odds ratio is used to define the relationship between exposure and outcome. For example, the effects of smoking and lung cancer. For example, the odds ratio, in this case, defines the odds that a person will suffer from Lung Cancer given that he smokes to the odds that he will still suffer from Lung Cancer given that he does not smoke.
  2. The Odds ratio is also commonly seen while interpreting the outcome of a Logistic Regression model. Here the Regression coefficient of a variable represents the increase/decrease in the Log Odds of the dependent variable based on one unit increase in the independent variable.

Example:

Consider the example of Smoking and its effects on Lung cancer, if we are to form a two by two table showing the effects of smokers and non-smokers in causing lung cancer, the table would look something like this,

CancerNon-CancerTotals
Smoker10060160
Non-Smoker34125159
Totals134185319

Out of the 319 patients,

  1. 160 are smokers and 159 are non-smokers.
  2. 134 have cancer and 185 do not have cancer.
  3. 100 smoke and have cancer, 60 smoke and do not have cancer.
  4. 34 do not smoke and have cancer and 125 do not smoke and do not have cancer.

From the above information, if you want to calculate the Odds Ration, you just have to cross multiply and take the ratio. Odds Ratio can be calculated as,

odds ratio value

Now, what this odds ratio means is that the odds of someone smoking and having cancer are 6.127 times the odds that someone who does not smoke and has cancer. To find out if this Odds Ratio is statistically significant or not, we need to calculate the Confidence intervals, which is not in the scope of this article. However, you can see a complete example of it in the link here.

Conclusion:

Odds and Odds Ratios play a very important role in the Medical domain, betting industries, etc. It becomes especially important in the medical domain to check the effect of certain exposures on certain outcomes. It is also important in the gambling industry as it relies heavily on the odds and probabilities. I hope I have been able to explain the intuition behind Odds, Odds Ratios, and Log effectively transforming the two.

As always any improvement tips, suggestions are always welcome.

I am a Software Test Engineer with a passion for Data Science, I am looking to explore opportunities in the field of Data Science and Machine Learning, You can connect with me on GitHub and LinkedIn.

Another article on Data Scraping using Python and Selenium can be found here.

Some excellent resources on Odds and Odds ratios can be found here and here.

FAQs

Q1. How do you convert log odds to probability?

To convert log odds to probability, apply the formula:

probability = e^log_odds / (1 + e^log_odds)

Q2. What is weighted log odds?

WLO is a modified version of the log odds ratio (LOR) that considers the frequency of both the feature and the target variable, making it less sensitive to rare features and more sensitive to common ones. It’s often used in text analysis and machine learning to identify relevant features.

Q3. What is log odds form of Bayes rule?

Bayes’ rule in log odds form simplifies calculations and provides a more intuitive understanding of relative likelihoods. It expresses posterior odds as the sum of prior odds and the log likelihood ratio.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

basics of probabilityblogathonstatistics
V
Vishwanath

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag
Congratulations, You Did It!
Well Done on Completing Your Learning Journey. Stay curious and keep exploring!
Logistic Regression using Python and Excel
Logistic Regression using Python and Excel
Simple Guide to Logistic Regression in R and Python
Simple Guide to Logistic Regression in R and Python

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details