Going Beyond the Basics of RAG Pipelines: How Txtai Can Help?

Introduction

Effective retrieval methods are paramount in an era where data is the new gold. This article introduces an innovative data extraction and processing approach. Dive into the world of txtai and Retrieval Augmented Generation (RAG), where complex data becomes easily navigable and insightful. By the end of this article, you will know how the fusion of txtai with RAG pipelines revolutionizes our interaction with large data sets, making data retrieval faster and smarter.

Also Read: What is Retrieval-Augmented Generation (RAG) in AI?

Learning Objectives

• Understand the fundamental concepts of RAG and its integration with txtai.
• Learn how to construct a basic RAG pipeline using txtai in a practical setting.
• Explore the scalability and adaptability of txtai’s RAG pipelines for large data sets.
• Gain insights into troubleshooting and optimizing RAG pipelines for enhanced performance and accuracy.

This article was published as a part of the Data Science Blogathon.

What is Txtai?

“Txtai” is an open-source Python package that uses Natural Language Processing (NLP) and Machine Learning to search, summarize, and analyze text data. It lets users quickly and effortlessly create powerful text-based applications without requiring extensive machine learning or data science knowledge.

With txtai, users can perform tasks such as document retrieval, keyword extraction, and text classification, making it a versatile tool for various text analysis needs.”

GitHub: https://github.com/neuml/txtai

Official Documentation of txtai: https://neuml.github.io/txtai/

txtai is an open-source library on GitHub with ~6K stars. Please see this guide for those who would like to contribute to txtai.

Take your AI innovations to the next level with GenAI Pinnacle. Fine-tune models like Gemini and unlock endless possibilities in NLP, image generation, and more. Dive in today! Explore Now

Cool Features of Txtai

Some really exciting features of the open-source library txtai are:

• Txtai is an AI-powered engine for data retrieval and processing
• It enables semantic search, language model workflows, and large language model orchestration
• It integrates vector indexes, graph networks, and relational databases
• Advanced functionalities include vector search with SQL, topic modeling, and retrieval augmented generation
• Txtai is designed with Python and uses Hugging Face Transformers
• It is an open-source solution under the Apache 2.0 license

Use-Cases of Txtai

• Semantic Search: Enhancing search capabilities by understanding the meaning and context of queries.
• Data Organization: Automating the categorization and tagging of large datasets.
• Question Answering Systems: Building systems that understand and respond to natural language queries.
• Content Summarization: Generating concise summaries from extensive text data.
• Language Translation: Facilitating cross-lingual communication and content translation.

Exploring RAG with Txtai

Retrieval Augmented Generation (RAG) combines the strengths of large language models with information retrieval systems to enhance the accuracy and contextuality of generated responses.

RAG pipelines in txtai enable dynamic fetching of relevant data during the response generation process, ensuring that outputs are based on pre-trained knowledge and the most current and relevant information available.

Txtai’s architecture can seamlessly integrate with various data sources and models. This makes it a powerful tool for providing contextually rich responses, including RAG implementation. It uses language that is easy to understand, keeping the sentences short and straightforward.

The text is arranged logically, with the most significant details presented first. The vocabulary used is everyday language that is familiar to the reader. Additionally, the text uses active voice to increase clarity.

Building Your First RAG Pipeline with Txtai

LLMs are popular in AI and machine learning, but they have a problem with hallucinations, which occur when the LLM generates factually incorrect output that seems plausible. RAG reduces this risk by limiting context with a vector search query.

It’s a practical and production-ready use case for Generative AI, and some companies are building their businesses around it. Txtai has question-answering pipelines that retrieve relevant context, and LLMs are used to analyze the context. RAG pipelines are a primary feature of txtai, and they are also a vector database. This feature is called the “all-in-one embedding database.” Open a Jupyter Notebook and follow the steps below. You can use Google Colab, which is free to use. The notebook shows how to build RAG pipelines with txtai.

Step 1: Install Dependencies

Install txtai and its dependencies. As we use optional pipelines, we must install some extra pipeline packages.

%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline] autoawq==0.1.5

# Download data sample for this tutorial
!wget -N https://github.com/neuml/txtai/releases/download/v6.2.0/tests.tar.gz
!tar -xvzf tests.tar.gz

# Install NLTK library
import nltk
nltk.download('punkt')

Step 2: Start with the Basics

The LLM pipeline can load local LLM models from the Hugging Face Hub. If you’re using LLM API services like OpenAI or Cohere, you can replace this call with an API call.

#import LLM pipeline from txtai
from txtai.pipeline import LLM

# Create LLM pipeline
llm = LLM("TheBloke/Mistral-7B-OpenOrca-AWQ")

We’ll now load a document to query. Textractor can extract text from commonly used document formats like doc, pdf, and xlsx.

# import the Textractor instance
from txtai.pipeline import Textractor

# Create Textractor pipeline that extracts 
# and splits text from documents
# Change the name of document as per your file name
textractor = Textractor()
texttr = textractor("txtai/document.docx")
print(texttr)

We’ll create a basic LLM pipeline by inputting a question and context (the entire file), generating a prompt, and running it through the LLM.

def execute(question, texttr):
  prompt = f"""<|im_start|>system
  You are a friendly assistant. You answer questions from users.<|im_end|>
  <|im_start|>user
  Answer the following question using only the context below. Only include 
  information specifically discussed.

  question: {question}
  context: {texttr} <|im_end|>
  <|im_start|>assistant
  """

  return llm(prompt, maxlength=4096, pad_token_id=32000)

execute("Tell me about txtai in one sentence", texttr)

execute("What model does txtai recommend for transcription?", text)

execute("I don't know anything about txtai, what would be the best thing /
to read?", text)

Generative AI is impressive. Even for those familiar with it, the language model’s understanding and quality of answers are astounding. Let’s explore scaling it to a larger set of documents.

Step 3: Build a RAG Pipeline with Vector Search

When dealing with many documents, such as hundreds or thousands, putting them all into a single prompt can quickly exhaust GPU memory.

Retrieval augmented generation helps by using a query step to find the best candidates to add to the prompt.

This query usually employs vector search, but any search method that returns results can be used. Many production systems have tailored retrieval pipelines that supply context to LLM prompts.

Build A Knowledge Store

This involves setting up a vector database of file content, where each paragraph is stored as a separate row.

import os

# import the embeddings package
from txtai import Embeddings

# create a pipeline stream
def stream(path):
  for f in sorted(os.listdir(path)):
    fpath = os.path.join(path, f)

    # List of only accepted documents
    if f.endswith(("docx", "xlsx", "pdf")):
      print(f"Indexing {fpath}")
      for paragraph in textractor(fpath):
        yield paragraph

# Document text extraction and split into paragraphs
textractor = Textractor(paragraphs=True)

# Vector Database to index articles
embeddings = Embeddings(content=True)
embeddings.index(stream("txtai"))

Defining the RAG Pipeline

This pipeline takes the input question, runs a vector search, and builds a context using the search results. You can then insert the context into a prompt template and run with the LLM models.

# write custom question to extract data
def context(question):
  context =  "\n".join(x["text"] for x in embeddings.search(question))
  return context

def rag(question):
  return execute(question, context(question))

rag("What model does txtai recommend for image captioning?")

output = rag("When was the BLIP model added for image captioning?")
print(output)

With vector search, we used a relevant portion of the documents to generate the answer, resulting in a similar output to the previous method.

When working with large volumes of data, it’s important only to include the most relevant context in the LLM prompt. Otherwise, the LLM may need to generate high-quality answers.

Troubleshooting and Optimization

Implementing RAG with txtai can sometimes present challenges, such as integration complexities or performance issues. Common issues include difficulties in configuring the RAG pipeline with specific data sources and optimizing query response times.

To achieve this, it is necessary to fine-tune the model parameters, keep the data sources up-to-date, and experiment with different configurations until you find the optimal balance for your specific use case.

To effectively address any issues related to txtai, it is important to deeply understand its Documentation. This will provide valuable insights and examples that can help you optimize the performance and accuracy of txtai.

Alternatively, you can check any errors related to txtai from the list of Issues on their GitHub.

Conclusion

The development of RAG and txtai appears promising, with continuous improvements to enhance their capabilities. One can expect to integrate more advanced AI models and expand Txtai’s functionalities, opening new semantic search and data processing frontiers. Txtai is an open-source library that welcomes Contributions and offers an amazing learning opportunity.

Key Takeaways

Here is a quick summary of what you learned in today’s article:

• Retrieval Augmented Generation (RAG) combined with txtai is innovative and transformative.
• Txtai’s advanced functionalities can enhance the quality and relevance of retrieved information.
• RAG pipelines are efficient and contextually aware.
• The combination of RAG and txtai has much potential in data retrieval and processing.

Dive into the future of AI with GenAI Pinnacle. From training bespoke models to tackling real-world challenges like PII masking, empower your projects with cutting-edge capabilities. Start Exploring.

Frequently Asked Questions

Do you have any questions?

You can ask questions in the comments below or connect with me. My social media accounts are below,  and I promise I’ll do my best to answer them.

LinkedIn | GitHub

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Mrinal Singh

Data Scientist and a Technical Writer! I will give you the best of Open-Source and AI.

Talks about #chatgpt, #opensource, #contentcreation, #communitybuilding, and #artificialintelligence

Technical Writer | Data Science, ML, AI, Open-Source | Do More with Data - Litmus