Mastering Zero-Shot and Few-Shot Text Classification with SCIKIT-LLM

Analyzing customer sentiment and key themes from textual data has always been a time-intensive task, requiring data collection, manual labeling, and fine-tuning specialized models. But what if you could skip the hassle of training a model and still achieve accurate results? Enter zero-shot text classification, a groundbreaking approach powered by Large Language Models (LLMs). In this article, we’ll explore how zero-shot classification simplifies sentiment analysis using the SKLLM library (a blend of scikit-learn and LLMs). In this tutorial, you’ll see how to use the SKLLM library (scikit-learn + LLM) to classify the Women’s E-Commerce Clothing Reviews dataset from Kaggle.

Learning Objectives

• Understand the traditional process of sentiment analysis and its challenges.
• Learn the concept and advantages of zero-shot text classification with LLMs.
• Explore the SKLLM library and its integration with scikit-learn.
• Classify sentiments in the Women’s E-Commerce Clothing Reviews dataset without custom training.
• Gain hands-on experience with zero-shot classification for practical use cases.

This article was published as a part of the Data Science Blogathon.

What is Zero-Shot Text Classification?

Online retailers often receive large volumes of text reviews from customers, making it challenging to quickly analyze the sentiments or key themes. Traditionally, companies would:

• Collect and clean review data.
• Manually label thousands of samples (e.g., “positive,” “negative,” “neutral”).
• Fine-tune a dedicated classification model on this labeled data.

While effective, fine-tuning requires considerable time, expertise, and computational resources. Enter zero-shot text classification: using Large Language Models (LLMs) directly to classify text with minimal effort. You can simply provide a set of descriptive labels (e.g., “positive,” “negative,” “neutral”) and let the model infer the correct class—no custom training required!

Why Zero-Shot is So Efficient?

Below we will discuss the points to understand that why zero-shot is so efficient:

• No Fine-Tuning Required: Finetuning LLMs like GPT-4o can be a costly affair. You don’t need to spend hours (or even days) training a sentiment classifier on your dataset. Instead, you leverage pre-trained LLMs like GPT-4o, giving you a high-quality classifier immediately.
• Easy Adaptation to New Labels: If your label set changes (e.g., from “positive, negative, neutral” to more specific sentiments like “happy, frustrated, curious, annoyed”), you simply update your label list. There is no need to retrain a model.
• Fewer Data Requirements: In typical supervised learning, you need labeled data for each class. Zero-shot classification only requires you to describe your classes (labels) in natural language. This is particularly helpful if you have limited or unlabeled data.
• Speed to Deployment: By skipping data annotation and model training steps, you can deploy your classification solution much faster.

Dataset Overview

We’ll use the Women’s E-Commerce Clothing Reviews dataset from Kaggle.

Click here to access the dataset.

Key points about the dataset:

• It contains thousands of customer reviews of women’s clothing items.
• The main text is in the “Review Text” column.
• Other metadata like “Title,” “Rating,” “Recommended IND,” and more are available but not always necessary for zero-shot classification.

Step-by-Step Guide

Below we will learn how to streamline sentiment analysis and theme detection with zero-shot text classification using Large Language Models (LLMs). In this tutorial, we’ll walk you through leveraging the SKLLM library to classify real-world data effortlessly—no custom training required!

Step1: Installation and Setup

Make sure you have Python 3.7+ and install SKLLM:

pip install scikit-llm

 Additionally, ensure you have a valid API key for an LLM provider (e.g., OpenAI’s API). Set it in your environment:  

from skllm.config import SKLLMConfig

# Replace with your actual OpenAI API key
SKLLMConfig.set_openai_key("your_openai_api_key")

(You can also store it in a .env file or handle it within your code, but environment variables are often cleaner.)

Step2: Import Libraries and Load the Dataset

import pandas as pd
from skllm.models.gpt.classification.zero_shot import ZeroShotGPTClassifier

# Load dataset
df = pd.read_csv("Womens Clothing E-Commerce Reviews.csv")

# Inspect the first few rows
print(df.head())

We’ll focus on the “Review Text” column. Some rows may have missing values for reviews, so let’s drop any NaNs:

# Filter out rows without review text
df = df.dropna(subset=["Review Text"]).reset_index(drop=True)

# Extract the review texts into X
X = df["Review Text"].tolist()

Step3: Define Your Labels

We’ll do a sentiment classification: [“positive”, “negative”, “neutral”].

Why these three? They’re common sentiment tags. However, you’re free to change or expand them: for example, [“positive”, “negative”, “neutral”, “mixed”].

Step4: Zero-Shot Classification

Instantiate the ZeroShotGPTClassifier. We’ll choose gpt-4o as the model, but you can select a different model if you want.

# Create a zero-shot classifier
clf = ZeroShotGPTClassifier(model="gpt-4o")

# Fit the classifier - here we pass `None` for X because we don't need training data
clf.fit(None, ["positive", "negative", "neutral"])

Why fit(None, labels)? In a pure zero-shot scenario, no actual training occurs. The call to fit() is effectively telling the classifier which labels are possible. The model can then choose among them for each review.

Step5: Classify the Reviews

# Predict labels for the entire dataset
predictions = clf.predict(X)

# Let’s see the first few results
for review_text, sentiment in zip(X[:5], predictions[:5]):
    print(f"Review: {review_text}")
    print(f"Predicted Sentiment: {sentiment}")
    print("-" * 50)

This loop will print out each review along with the zero-shot classifier’s predicted sentiment.

Results Discussion

With a traditional ML approach, you’d need:

• Labeling: A large subset of these reviews labeled as positive, negative, neutral.
• Model Training: Fine-tuning or training a classifier from scratch (e.g., an SVM, a BERT-based model).
• Model Validation: Manually verifying performance on a validation set.
• Continuous Updates: If new sentiments or categories emerge, you’d need more labeled data and additional training.

Zero-shot eliminates most of that overhead:

• Immediate Start: You only provide a label list and a well-crafted prompt behind the scenes.
• No Labeled Data Required: The LLM has learned enough semantics about language to infer meaning from descriptive labels.
• Easy to Refine: Need new categories like “slightly positive” or “ambivalent”? Just add them to the list of candidate labels.

Potential Limitations to Note

• Accuracy Variation: The quality of zero-shot classification can vary. For simple sentiment analysis, it often works surprisingly well. For highly specialized or technical domains, the model might misinterpret certain text or domain jargon.
• Cost: Using a large model like GPT-4o involves API costs if you are calling an external service.
• Data Privacy: You must ensure sending data to an API is allowed (especially if the text is sensitive).

Few-Shot Text Classification

Few-shot text classification is a task of classifying a text into one of the pre-defined classes based on a few examples of each class. For example, given a few examples of the classes positive, negative, and neutral, the model should be able to classify new text into one of these categories.

Note: The estimators provided by Scikit-LLM do not automatically select a subset of the training data; they use the entire training set to build the few-shot examples. If your training set is large, consider splitting it into training and validation sets while keeping the training set small (ideally no more than 10 examples per class). Also, be sure to permute the order of these samples to avoid any recency bias in the LLM’s attention.

from skllm.models.gpt.classification.few_shot import (
    FewShotGPTClassifier,
    MultiLabelFewShotGPTClassifier,
)
from skllm.datasets import (
    get_classification_dataset,
    get_multilabel_classification_dataset,
)

# Single-label classification
X, y = get_classification_dataset()
clf = FewShotGPTClassifier(model="gpt-4o")
clf.fit(X, y)
labels = clf.predict(X)

# Multi-label classification
X, y = get_multilabel_classification_dataset()
clf = MultiLabelFewShotGPTClassifier(max_labels=2, model="gpt-4o")
clf.fit(X, y)
labels = clf.predict(X)

Chain-of-Thought Text Classification

Chain-of-thought text classification is similar to zero-shot classification in that it does not require labeled data beforehand. The main difference is that the model generates intermediate reasoning steps along with the label. This added “chain of thought” can improve performance but increases token usage (and thus potential cost).

from skllm.models.gpt.classification.zero_shot import CoTGPTClassifier
from skllm.datasets import get_classification_dataset

# Demo sentiment analysis dataset
# Labels: positive, negative, neutral
X, y = get_classification_dataset()

clf = CoTGPTClassifier(model="gpt-4o")
clf.fit(X, y)
predictions = clf.predict(X)

# Each prediction has [label, reasoning]
labels, reasoning = predictions[:, 0], predictions[:, 1]

By testing a few-shot approach or a chain-of-thought approach, you may see an improvement over the baseline zero-shot classification results

Conclusion

Scikit-LLM’s library is a fast, flexible, and easy alternative to building a custom sentiment analysis pipeline. Without the need to label data or fine-tune a model, you can immediately classify customer feedback into descriptive categories.

In the case of the Women’s E-Commerce Clothing Reviews dataset, you can quickly unlock insights—such as customer sentiment—without the usual overhead of dataset preparation, labeling, and model retraining. This advantage is especially powerful if you need to iterate on or expand your classification labels over time.

As the AI ecosystem evolves, zero-shot and few-shot techniques will continue to grow in importance. They enable rapid prototyping and accelerate business workflows by leveraging the massive knowledge already embedded in large language models.

Key Takeaways

• Zero-shot classification simplifies sentiment analysis without the need for manual labeling or model training.
• The SKLLM library integrates scikit-learn with LLMs for efficient text classification.
• SCIKIT-LLM enables efficient Zero-Shot and Few-Shot text classification, eliminating the need for manual labeling and model training.
• Large Language Models (LLMs) like GPT-4 enable immediate, high-quality classification results.
• With SCIKIT-LLM, you can quickly deploy text classification solutions using pre-trained Large Language Models, saving time and resources.
• The Women’s E-Commerce Clothing Reviews dataset provides a practical example of zero-shot classification in action.
• Zero-shot text classification is fast, adaptable, and requires minimal data, making it ideal for quick deployment.

Frequently Asked Questions

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Adarsh Balan

Hi! I'm Adarsh, a Business Analytics graduate from ISB, currently deep into research and exploring new frontiers. I'm super passionate about data science, AI, and all the innovative ways they can transform industries. Whether it's building models, working on data pipelines, or diving into machine learning, I love experimenting with the latest tech. AI isn't just my interest, it's where I see the future heading, and I'm always excited to be a part of that journey!