Try TeapotLLM for Reliable Q&A, RAG, and Info Extraction

Text generation models are exceptional tools for both research purposes and applications. One of their strengths is their capabilities, which come from their architecture, training, and large datasets. These features shape how these models work. 

TeapotAI’s open-source model is a good example of a model that stands out with its work in TeapotLLM. This is a small language model built on 800M parameters. It is also fine-tuned on synthetic data, allowing for efficiency in low-resource environments, including smartphones and CPUs. It is a great tool for various tasks. This model can perform only Q&A, RAG, and Information extraction within a given context.

Learning Objectives 

• Understand the capabilities and unique features of TeapotLLM.
• Explore the model architecture and training process of TeapotLLM.
• Learn about retrieval-augmented generation (RAG) and hallucination resistance in TeapotLLM.
• Discover real-world applications of TeapotLLM in AI-driven tasks.
• Gain hands-on experience in running TeapotLLM for Q&A, RAG, and structured data extraction.

This article was published as a part of the Data Science Blogathon.

What is TeapotLLM?

TeapotLLM is a cutting-edge 800M parameter model with high accuracy. This small language model was built to generate hallucination-free information. It comes with its comprehensive Python package, TeapotAI, that helps work with the model. 

This model builds on a transformer architecture and performs various natural language processing tasks. Developers fine-tuned it from flan-t5-base using a synthetic dataset of LLM tasks generated with Deepseek-V3.

Features of TeapotAI LLM

There are several features of this model, including the following-

Retrieval Augmented Generation 

This model can be fine-tuned to perform retrieval augmented generation using the custom embedding model. The model can then learn to extract information from documents to answer questions. 

Hallucination Resistance

Teapot AI is trained to generate text within a provided context. This helps it avoid answering questions without sufficient data. 

Pydantic Extraction Function 

This feature means that TeapotAI has a package that provides a pydantic-based data extraction function for the model. This permits you to get data from text efficiently and accurately. 

Model Architecture of Teapot LLM

This model was built from fine-tuning Flan-T5-base and synthetic data. Its principles are based on a transformer model; Teapot AI is also built on the encoder-decoder architecture. 

Teapot LLM is a specialized language model fine-tuned from Flan-T5-Large, a well-known instruction-tuned variant of T5 (Text-To-Text Transfer Transformer). The base model, Flan-T5-Large, is a transformer-based architecture that excels at various natural language processing tasks by treating every problem as a text-to-text problem. Teapot LLM builds on this foundation and undergoes further refinement with a synthetic dataset of large language model (LLM) tasks generated by DeepSeek-V3, an advanced generative model known for producing high-quality synthetic text.

The model’s architecture uses the encoder-decoder structure popular with many transformer models to perform text generation. These two components all have their respective roles. The encoder processes input sequences, while the decoder does the same for output sequences. 

During processing, the encoder transforms the input text into a latent representation. The decoder, on the other hand, takes these representations and converts them into task-specific responses. 

This model’s performance comes with a high contextual understanding. And it is easy to prove this from its architecture with certain standard transformer principles like the Transformer attention mechanism, incorporating multi-head self-attention layers, feed-forward networks, and layer normalization. 

How to Run Teapot LLM

This model can be used for various applications, such as answering questions, chatting with RAG, and extracting information. We’ll explore the steps for running this model to perform these tasks. 

Preparing the Environment

! pip install teapotai

Firstly, you install the Python package needed for executing this task. This command installs TeapotAI with the functionalities required to carry out hallucination-resistant tasks.  

Importing Essential Library

This step requires you to import the TeapotAI class from the TeapotAI library. Importing this helps the model perform tasks like hallucination-resistant Q&A, Retrieval-Augmented Generation (RAG), and JSON extraction.

from teapotai import TeapotAI

Context

Providing context is another important step in running this model. This helps the model access information to perform the specified task.

context = """
The Eiffel Tower is a wrought iron lattice tower in Paris, France. It was designed
 by Gustave Eiffel and completed in 1889.
It stands at a height of 330 meters and is one of the most recognizable structures
 in the world.
"""

This context usually comes in a multi-line string as shown above, with the information wrapped in the triple quotes. 

Model Initialization and Query

 teapot_ai = TeapotAI()

answer = teapot_ai.query(
   query="What is the height of the Eiffel Tower?",
   context=context
)

The code initializes Teapotai and uses it to request information based on the provided context mentioned earlier. To get the answer, we print (result) as shown below;

print (answer)

Here is a shot of the answer based on the context. 

Using this model as a chat when answering a question with many documents. Let’s look at how we can run Teapot with this feature.

 from teapotai import TeapotAI

This code imports the necessary library just as with the first task. 

Context

Here, can provide context that the RAG application will answer questions bases on; this could be long articles or a document. Below is a sample of this below;

documents = [
   "The Eiffel Tower is located in Paris, France. It was built in 1889 and stands
 330 meters tall.",
   "The Great Wall of China is a historic fortification that stretches over 13,000
 miles.",
   "The Amazon Rainforest is the largest tropical rainforest in the world, covering
 over 5.5 million square kilometers.",
   "The Grand Canyon is a natural landmark located in Arizona, USA, carved by the
 Colorado River.",
   "Mount Everest is the tallest mountain on Earth, located in the Himalayas along
 the border between Nepal and China.",
   "The Colosseum in Rome, Italy, is an ancient amphitheater known for its gladiator
 battles.",
   "The Sahara Desert is the largest hot desert in the world, located in North
 Africa.",
   "The Nile River is the longest river in the world, flowing through northeastern
 Africa.",
   "The Empire State Building is an iconic skyscraper in New York City that was
 completed in 1931 and stands at 1454 feet tall."
]

This code defines a list named ‘documents’, where each element is a string containing factual information.

Initializing Teapot With Documents for RAG

This initialization ensures that TeapotAI can use these documents for retrieval-augmented generation (RAG), answering questions based on the given information rather than generating responses from general knowledge.

teapot_ai = TeapotAI(documents=documents)

Getting the Answer Using RAG

 answer = teapot_ai.chat([
   {
       "role":"system",
       "content": "You are an agent designed to answer facts about famous landmarks."
   },
   {
       "role":"user",
       "content": "What landmark was constructed in the 1800s?"
   }
])

This code uses TeapotAI’s ‘chat’ method to generate a structured conversation and response. The input would be the message indicated in the “role”: “system” and the “role”: “user” fields. So, the answer will be based only on the given context of the list named ‘documents’ above.   

print(answer)

Here is the answer based on the documents. 

Information Extraction with Teapot

This model can extract information from context using JSON structures. The extract method uses a Pydantic model to guarantee that Teapot retrieves data in the correct format. It can infer fields based on their names and utilize descriptions when provided. This method seamlessly integrates with RAG and query functionalities for enhanced data extraction.

Importing Necessary Libraries

from teapotai import TeapotAI
from pydantic import BaseModel, Field

These libraries help validate data structures, such as the pydantic model. The BaseModel and Field are crucial to enforcing correct data formats. Together, they ensure accurate and structured information extraction from text.

Context

Here, we provide the description from which we want to extract information: the details of an apartment.

 apartment_description = """
This spacious 2-bedroom apartment is available for rent in downtown New York. The
 monthly rent is $2500.
It includes 1 bathrooms and a fully equipped kitchen with modern appliances. There 
is also a swimming pool at the backyard and beside the building.

Pets are welcome!

Please reach out to us at 555-123-4567 or john@realty.com
"""

Data Extraction Using a Pydantic Model

class ApartmentInfo(BaseModel):
   rent: float = Field(..., description="the monthly rent in dollars")
   bedrooms: int = Field(..., description="the number of bedrooms")
   bathrooms: int = Field(..., description="the number of bathrooms")
   phone_number: str

This code defines the ‘ApartmentInfo’ model using Pydantic to ensure structured data extraction. The respective fields clarify each description so the model can validate and organise extracted information. 

Initialize Teapot

This initializes the TeapotAI model and allows access to the structured data extraction features.

teapot_ai = TeapotAI()

Extract the Apartment Details

extracted_info = teapot_ai.extract(
   ApartmentInfo,
   context=apartment_description
)
print(extracted_info)

Here, we use the Teapot AI model to extract structure data from the ‘ApartmentInfor’, identifying key details like rent, phone number, and number of rooms. 

Here is the result:

Hallucination Resistance of TeapotLLLM

One essential technique this model employs to ensure accurate performance is the hallucination resistance. This permits the model to provide an answer only within the context of the document or information provided. 

Let’s illustrate a good example of this with the output. 

from teapotai import teapotAI
context = """
The Great Pyramid of Giza, built around 2560 BCE, is the oldest of the Seven Wonders of the Ancient World and the only one still standing.
"""

Real-Life Application of TeapotLLM 

Let us highlight some common use cases of this model in modern day. 

• AI-powered chatbots and virtual assistants are great examples of how to apply this model’s features. You can generate answers based on a specific context so users get more accurate and correct information. 
• This model can also generate content for blogs, reports, and marketing data by summarizing lengthy documents and retrieving key details. 
• Many industries thrive on data-driven systems. TeapotLLM can help extract details from real estate documents, finance, and legal systems. You can access contracts, legal documents, or raw data. 

Conclusion

This powerful open-source model is designed for reliable Q&A, retrieval-augmented generation (RAG), and structured information extraction. Its  800M parameter transformer architecture optimizes it for efficiency in low-resource environments while maintaining high accuracy. 

TeapotLLM’s ability to resist hallucinations and provide structured outputs makes it a valuable tool in AI-driven applications, from chatbots to document analysis.

Key Takeaway

• Its 800 million parameters and architecture make it lightweight and suitable for low-resource environments, such as CPUs and smartphones.
• The hallucination-resistant capability of this model makes it more context-aware and reduces the margin for inaccurate answers. 
• The model uses Pydantic to extract information in predefined formats, making it ideal for applications like real estate listings, financial documents, and legal text processing.

Resource

• Hugging Face
• Architecture
• TeapotLLM
• Github

Frequently Asked Questions

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Maigari David

Hey there! I'm David Maigari, a dynamic professional with a passion for technical writing, Web Development, and the AI world. David is also an enthusiast of ML/AI innovations. Reach out to me on X (Twitter) at @maigari_david