Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Swiggy’s Hermes: AI Solution for Seamless Data-Driven Decisions

JanviKumari01
JanviKumari01 22 Aug, 2024
8 min read

Introduction

In today’s data-driven world, Swiggy, a leading player in India’s food delivery industry, is transforming how its team accesses and interprets data with Hermes, a generative AI tool. Recognizing the need for timely and accurate information for informed decision-making, Swiggy developed Hermes to make data retrieval fast and accessible across the organization.

Unlike many AI tools that focus on summarizing text, Hermes is designed to deliver precise numbers and detailed insights crucial for business decisions. Whether it’s assessing the impact of a telco outage on customer notifications or analyzing customer claims within a restaurant cohort, Hermes allows Swiggy’s teams to pose questions in natural language and instantly receive both SQL queries and results within Slack. This innovation empowers users with actionable insights, streamlining data access without requiring extensive technical expertise.

Overview

  • Swiggy developed Hermes, an AI-based workflow, to make data access and interpretation faster and more efficient for teams.
  • Hermes allows users to pose natural language questions and instantly receive SQL queries and results within Slack.
  • The introduction of Hermes V2 refined the system with a compartmentalized approach, improving data flow and query accuracy.
  • Hermes V2 utilizes a Knowledge Base and Retrieval-Augmented Generation (RAG) to enhance context and precision in SQL generation.
  • Since its launch, Hermes has been widely adopted across Swiggy, significantly reducing the time needed for data-driven decisions.
  • Hermes empowers product managers, data scientists, and analysts by streamlining data retrieval and enabling deeper insights with minimal technical expertise.

Table of contents

The Challenge of Swiggy

Swiggy encountered a challenge familiar to many organizations: providing employees from diverse departments with the ability to access crucial data without heavily relying on technical experts. Traditionally, obtaining specific business insights involved navigating through reports, crafting complex SQL queries, or waiting for an analyst to extract the data—tasks that could be both time-consuming and cumbersome. Such inefficiencies delayed decision-making and risked decisions based on incomplete or incorrect data.

Introducing Hermes

To overcome these hurdles, Swiggy developed Hermes, a sophisticated generative AI solution integrated with Slack. This innovative tool allows employees to pose questions in natural language and receive both the SQL queries and their results in real-time. For instance, a product manager might ask, “What was the average rating for orders delivered 5 minutes earlier than promised last week in Bangalore?” and promptly get the SQL query and data needed.

Previously, answering such a query could take minutes to days, depending on its complexity and resource availability. Hermes dramatically shortens this timeframe, enabling Swiggy’s teams to make swifter, data-driven decisions and boost overall productivity.

We have taken reference from this article by Amaresh M: Click Here.

Hermes V1: The Foundation

The first version of Hermes, or Hermes V1, was a straightforward implementation using GPT-3.5 variants. Users could bring their metadata, type a prompt in Slack, and receive a generated SQL query along with the results. Although the results were promising and aligned with industry benchmarks, Swiggy quickly realized the need for a more tailored solution. The complexity of users’ queries and the vast volume of data necessitated a more specialized approach.

Swiggy’s learnings from Hermes V1 led to a critical design decision: Compartmentalizing Hermes into distinct business units or “charters,” each with its own metadata and specific use cases. This approach recognized that tables and metrics related to different Swiggy services, like Food Marketplace and Instamart, while similar, needed to be treated separately to optimize performance.

Hermes V1
Figure: Implementation of Hermes V1

Hermes V2: A Refined Approach

Building on the insights gained from Hermes V1, Swiggy introduced Hermes V2, featuring an improved data flow and a more robust implementation. The revamped system includes several key components:

Hermes V2
Figure: Implementation of Hermes V2

1. User Interface

Slack continues to serve as the entry point, where users type prompts and receive both SQL queries and results.

2. Middleware (AWS Lambda)

This intermediary layer facilitates communication between the user interface and the generative AI model, processing and formatting inputs before sending them to the model.

3. Generative AI Model

Upon receiving a request, a new Databricks job fetches the relevant charter’s generative AI model, generates the SQL query, executes it, and returns both the query and its output.

4. Knowledge Base + RAG Approach

This approach helps the model incorporate Swiggy-specific context, ensuring the correct tables and columns are selected for each query.

Generative AI Model Pipeline

Swiggy’s implementation of a Generative AI model pipeline employs a Knowledge Base combined with a Retrieval-Augmented Generation (RAG) approach. This method is instrumental in embedding Swiggy-specific context, guiding the AI model to accurately identify and select the appropriate tables and columns for each query.

Hermes V2
The Gen AI model pipeline

5. Knowledge Base

This pipeline’s core is a comprehensive Knowledge Base, which stores key metadata for each specific business unit or “charter” within Swiggy, such as Swiggy Food or Swiggy Genie. This metadata includes essential information like metrics, tables, columns, and reference SQL queries. The importance of metadata in a Text-to-SQL model cannot be overstated, as it serves several critical functions:

  • Contextual Understanding

Metadata provides the model with crucial information about the data structure, such as table names, column names, and descriptions. This context is vital for the model to map natural language queries to the correct database structures accurately.

  • Disambiguation

Human language is often ambiguous and context-dependent. Metadata helps clarify terms, ensuring the model generates SQL queries accurately reflecting the user’s intent. For example, it can distinguish whether “sales” refers to a specific table, a column within a table, or another entity.

  • Accuracy

Detailed metadata significantly enhances the accuracy of the generated SQL queries. A thorough understanding of the data schema makes the model less likely to produce errors, reducing the need for manual corrections.

  • Scalability

A robust and standardized set of metadata allows the Text-to-SQL model to scale effectively across different databases and data sources. This scalability enables the model to adapt to new datasets without requiring extensive reconfiguration, ensuring it meets Swiggy’s evolving data needs.

The Model Pipeline

The improved model pipeline in Hermes V2 is designed to break down the user prompt into several stages, ensuring clean and relevant information is passed for the final query generation. 

The Model Pipeline
Low-Level Design of the Prompt Journey

These stages include:

  1. Metrics Retrieval: The first stage retrieves relevant metrics to understand the user’s question. This involves leveraging the knowledge base to fetch associated queries and historical SQL examples through embedding-based vector lookup.
  2. Table and Column Retrieval: The next stage uses metadata descriptions to identify the necessary tables and columns. This process combines LLM querying, filtering, and vector-based lookup. For tables with a large number of columns, multiple LLM calls are made to avoid token limits. Additionally, vector search matches column descriptions with user questions and metrics, identifying all relevant columns.
  3. Few-Shot SQL Retrieval: Swiggy maintains ground-truth, verified, or reference queries for several key metrics. A vector-based few-shot retrieval method fetches relevant reference queries to aid in the generation process.
  4. Structured Prompt Creation: The system compiles all gathered information into a structured prompt, which includes querying the database and collecting data snapshots. The system then sends this structured prompt to the LLM for SQL generation.
  5. Query Validation: Swiggy validates the generated SQL query by running it on its database. If errors occur, they relay them to the LLM for correction with a set number of retries. Once they obtain an executable SQL query, they run it and relay the results back to the user. If retries fail, they share the query and modification notes with the user.

Adoption and Impact

Hermes has quickly become a vital tool across Swiggy, with hundreds of users leveraging it to handle thousands of queries in under two minutes on average. Product managers use Hermes for swift metrics checks and post-release validations, while data scientists and analysts depend on it for detailed investigations and trend analyses.

The success of Hermes V2 highlights the critical role of well-defined metadata and a tailored approach in data management. By organizing data by charter and continuously refining its knowledge base, Swiggy has developed a robust tool that democratizes data access and significantly enhances team productivity.

Swiggy Hermes: Looking Forward

Swiggy’s ongoing innovation with Hermes sets a new benchmark for how businesses can harness generative AI to transform data accessibility. With a commitment to continual improvement and incorporating user feedback, Hermes is well-positioned to become a cornerstone of Swiggy’s data-driven decision-making process, ensuring the company remains at the forefront of the rapidly evolving food delivery industry.

Our Opinion

Swiggy’s approach with Hermes exemplifies how generative AI can streamline data processes and empower teams. By addressing specific business needs with a tailored solution, Swiggy has enhanced operational efficiency and set a precedent for leveraging AI in practical, impactful ways. It’s exciting to see how such innovations can shape the future of data accessibility and decision-making in the industry.

Conclusion

Swiggy’s journey with Hermes underscores the importance of making data accessible and actionable for all users. With the successful rollout of Hermes V2, Swiggy has improved its internal processes and set a new standard for how companies can democratize data access across their organizations. As Hermes continues to evolve, it promises further to enhance the speed and accuracy of decision-making at Swiggy, enabling teams to unlock the full potential of their data.

Dive into the future of AI with GenAI Pinnacle. Empower your projects with cutting-edge capabilities, from training bespoke models to tackling real-world challenges like PII masking. Start Exploring.

Frequently Asked Questions

Q1. What is Hermes, and how does it benefit Swiggy’s teams?

Ans. Hermes is Swiggy’s in-house developed generative AI-based workflow designed to allow users to ask data-related questions in natural language and receive both a SQL query and its results directly within Slack. It streamlines data access, enabling faster, more efficient decision-making by reducing the dependency on technical resources and minimizing the time needed to retrieve actionable insights.

Q2. How does Hermes V2 differ from Hermes V1?

Ans. Hermes V2 improves upon the initial version by compartmentalizing the system according to distinct business units (charters) within Swiggy. It incorporates a Knowledge Base and RAG-based approach to generate more accurate and contextually relevant SQL queries. This version also features a more refined model pipeline that breaks down user prompts into specific stages, such as metrics retrieval and query validation, to ensure clean and relevant data for query generation.

Q3. What is the role of the Knowledge Base in Hermes?

Ans. The Knowledge Base in Hermes stores critical metadata for each business unit, including metrics, tables, columns, and reference SQL queries. This metadata provides essential context to the AI model, helping it accurately translate natural language queries into SQL commands. It also assists in disambiguating terms, improving accuracy, and ensuring the system can scale across different data sources.

Q4. Why is metadata so crucial in the Hermes AI model?

Ans. Metadata is crucial because it provides the AI model with the context to accurately map natural language queries to database structures. It helps disambiguate terms, improves the precision of SQL query generation, and supports the model’s scalability across different datasets. Detailed metadata reduces errors and enhances the overall performance of the system.

Q5. How has Hermes been adopted within Swiggy?

Ans. Hermes has seen widespread adoption across Swiggy, with hundreds of users leveraging it to answer thousands of data-related queries. The system is precious for product managers, business teams, data scientists, and analysts, helping them perform tasks such as sizing numbers for initiatives, post-release validations, trend tracking, and in-depth data investigations, all with an average turnaround time of under 2 minutes.

AI toolsGenerative AISwiggySwiggy hermes
JanviKumari01
JanviKumari01 22 Aug, 2024

Hi I am Janvi Kumari currently a Data Science Intern at Analytics Vidhya, passionate about leveraging data for insights and innovation. Curious, driven, and eager to learn. If you'd like to connect, feel free to reach out to me on LinkedIn

Free Courses

image.name 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
image.name 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
image.name 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
image.name 4.8

Building Your first RAG System using LlamaIndex

Build your first RAG model with LlamaIndex in this free course. Dive into Retrieval-Augmented Generation now!
image.name 4.7

Building Production Ready RAG systems using LlamaIndex

Learn Retrieval-Augmented Generation (RAG): learn how it works, the RAG framework, and use LlamaIndex for advanced systems.

Recommended Articles

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag
Rahul Shah img

Rahul Shah

27
Sion Chakrabarti img

Sion Chakrabarti

16
CHIRAG GOYAL img

CHIRAG GOYAL

87
kaustubh gupta img

kaustubh gupta

24
Barney Darlington img

Barney Darlington

5
Suvojit Hore img

Suvojit Hore

9
Arnab Mondal img

Arnab Mondal

15