How to Build a RAG System Using DeepSeek R1?

I have been reading a lot about RAG and AI Agents, but with the release of new models like DeepSeek V3 and DeepSeek R1, it seems that the possibility of building efficient RAG systems has significantly improved, offering better retrieval accuracy, enhanced reasoning capabilities, and more scalable architectures for real-world applications. The integration of more sophisticated retrieval mechanisms, enhanced fine-tuning options, and multi-modal capabilities are changing how AI agents interact with data. It raises questions about whether traditional RAG approaches are still the best way forward or if newer architectures can provide more efficient and contextually aware solutions.

Retrieval-augmented generation (RAG) systems have revolutionized the way AI models interact with data by combining retrieval-based and generative approaches to produce more accurate and context-aware responses. With the advent of DeepSeek R1, an open-source model known for its efficiency and cost-effectiveness, building an effective RAG system has become more accessible and practical. In this article, we are building an RAG system using DeepSeek R1.

What is DeepSeek R1?

DeepSeek R1 is an open-source AI model developed with the goal of providing high-quality reasoning and retrieval capabilities at a fraction of the cost of proprietary models like OpenAI’s offerings. It features an MIT license, making it commercially viable and suitable for a wide range of applications. Also, this powerful model, lets you see the CoT but the OpenAI o1 and o1-mini don’t show any reasoning token. 

To know how DeepSeek R1 is challenging the OpenAI o1 model: DeepSeek R1 vs OpenAI o1: Which One is Faster, Cheaper and Smarter? 

Benefits of Using DeepSeek R1 for RAG System

Building a Retrieval-Augmented Generation (RAG) system using DeepSeek-R1 offers several notable advantages:

1. Advanced Reasoning Capabilities: DeepSeek-R1 is designed to emulate human-like reasoning by analyzing and processing information step-by-step before reaching conclusions. This approach enhances the system’s ability to handle complex queries, particularly in areas requiring logical inference, mathematical reasoning, and coding tasks.

2. Open-Source Accessibility: Released under the MIT license, DeepSeek-R1 is fully open-source, allowing developers unrestricted access to its model. This openness facilitates customization, fine-tuning, and integration into various applications without the constraints often associated with proprietary models.

3. Competitive Performance: Benchmark tests indicate that DeepSeek-R1 performs on par with, or even surpasses, leading models like OpenAI’s o1 in tasks involving reasoning, mathematics, and coding. This level of performance ensures that an RAG system built with DeepSeek-R1 can deliver high-quality, accurate responses across diverse and challenging queries.

4. Transparency in Thought Process: DeepSeek-R1 employs a “chain-of-thought” methodology, making its reasoning steps visible during inference. This transparency not only aids in debugging and refining the system but also builds user trust by providing clear insights into how conclusions are reached.

5. Cost-Effectiveness: The open-source nature of DeepSeek-R1 eliminates licensing fees, and its efficient architecture reduces computational resource requirements. These factors contribute to a more cost-effective solution for organizations looking to implement sophisticated RAG systems without incurring significant expenses.

Integrating DeepSeek-R1 into an RAG system provides a potent combination of advanced reasoning abilities, transparency, performance, and cost efficiency, making it a compelling choice for developers and organizations aiming to enhance their AI capabilities.

Steps to Build a RAG System Using DeepSeek R1

The script is a Retrieval-Augmented Generation (RAG) pipeline that:

• Loads and processes a PDF document by splitting it into pages and extracting text.
• Stores vectorized representations of the text in a database (ChromaDB).
• Retrieves relevant content using similarity search when a query is asked.
• Uses an LLM (DeepSeek model) to generate responses based on the retrieved text.

Install Prerequisites

• Download Ollama: Click here to download
• For Linux users: Run the following command in your terminal:

curl -fsSL https://ollama.com/install.sh | sh

after this pull the DeepSeek R1:1.5b using:

ollama pull deepseek-r1:1.5b

This will take a moment to download:

ollama pull deepseek-r1:1.5b

pulling manifest
pulling aabd4debf0c8... 100% ▕████████████████▏ 1.1 GB                         
pulling 369ca498f347... 100% ▕████████████████▏  387 B                         
pulling 6e4c38e1172f... 100% ▕████████████████▏ 1.1 KB                         
pulling f4d24e9138dd... 100% ▕████████████████▏  148 B                         
pulling a85fe2a2e58e... 100% ▕████████████████▏  487 B                         
verifying sha256 digest 
writing manifest 
success 

After doing this, open your Jupyter Notebook and start with the coding part:

1. Install Dependencies

Before running, the script installs the required Python libraries:

• langchain → A framework for building applications using Large Language Models (LLMs).
• langchain-openai → Provides integration with OpenAI services.
• langchain-community → Adds support for various document loaders and utilities.
• langchain-chroma → Enables integration with ChromaDB, a vector database.

2. Enter OpenAI API Key

To access OpenAI’s embedding model, the script prompts the user to securely enter their API key using getpass(). This prevents exposing credentials in plain text.

3. Set Up Environment Variables

The script stores the API key as an environment variable. This allows other parts of the code to access OpenAI services without hardcoding credentials, which improves security.

4. Initialize OpenAI Embeddings

The script initializes an OpenAI embedding model called "text-embedding-3-small". This model converts text into vector embeddings, which are high-dimensional numerical representations of the text’s meaning. These embeddings are later used to compare and retrieve similar content.

5. Load and Split a PDF Document

A PDF file (AgenticAI.pdf) is loaded and split into pages. Each page’s text is extracted, which allows for smaller and more manageable text chunks instead of processing the entire document as a single unit.

6. Create and Store a Vector Database

• The extracted text from the PDF is converted into vector embeddings.
• These embeddings are stored in ChromaDB, a high-performance vector database.
• The database is configured to use cosine similarity, which ensures that text with a high degree of semantic similarity is retrieved efficiently.

7. Retrieve Similar Texts Using a Similarity Threshold

A retriever is created using ChromaDB, which:

• Searches for the top 3 most similar documents based on a given query.
• Filters results with a similarity threshold of 0.3 (i.e., documents must have at least 30% similarity to be considered relevant).

8. Query for Similar Documents

Two test queries are used:

1. "What is the old capital of India?"
   • No results were found, which indicates that the stored documents do not contain relevant information.
2. "What is Agentic AI?"
   • Successfully retrieves relevant text, demonstrating that the system can fetch meaningful context.

9. Build a RAG (Retrieval-Augmented Generation) Chain

The script sets up a RAG pipeline, which ensures that:

• Text retrieval happens before generating an answer.
• The model’s response is based strictly on retrieved content, preventing hallucinations.
• A prompt template is used to instruct the model to generate structured responses.

10. Load a Connection to an LLM (DeepSeek Model)

Instead of OpenAI’s GPT, the script loads DeepSeek-R1 (1.5B parameters), a powerful LLM optimized for retrieval-based tasks.

11. Create a RAG-Based Chain

LangChain’s Retrieval module is used to:

• Fetch relevant content from the vector database.
• Format a structured response using a prompt template.
• Generate a concise answer with the DeepSeek model.

12. Test the RAG Chain

The script runs a test query:
"Tell the Leaders’ Perspectives on Agentic AI"

The LLM generates a fact-based response strictly using the retrieved context.

The system retrieves relevant information from the database.

Code to Build a RAG System Using DeepSeek R1

Here’s the code:

Install OpenAI and LangChain dependencies

!pip install langchain==0.3.11
!pip install langchain-openai==0.2.12
!pip install langchain-community==0.3.11
!pip install langchain-chroma==0.1.4

Enter Open AI API Key

from getpass import getpass
OPENAI_KEY = getpass('Enter Open AI API Key: ')

Setup Environment Variables

import os
os.environ['OPENAI_API_KEY'] = OPENAI_KEY

Open AI Embedding Models

from langchain_openai import OpenAIEmbeddings
openai_embed_model = OpenAIEmbeddings(model='text-embedding-3-small')

Create a Vector DB and persist on the disk

from langchain_community.document_loaders import PyPDFLoader
loader = PyPDFLoader('AgenticAI.pdf')
pages = loader.load_and_split()
texts = [doc.page_content for doc in pages]

from langchain_chroma import Chroma
chroma_db = Chroma.from_texts(
    texts=texts,
    collection_name='db_docs',
    collection_metadata={"hnsw:space": "cosine"},  # Set distance function to cosine
embedding=openai_embed_model
)

Similarity with Threshold Retrieval

similarity_threshold_retriever = chroma_db.as_retriever(search_type="similarity_score_threshold",search_kwargs={"k": 3,"score_threshold": 0.3})

query = "what is the old capital of India?"
top3_docs = similarity_threshold_retriever.invoke(query)
top3_docs

[]

query = "What is Agentic AI?"
top3_docs = similarity_threshold_retriever.invoke(query)
top3_docs

Build a RAG Chain

from langchain_core.prompts import ChatPromptTemplate
prompt = """You are an assistant for question-answering tasks.
            Use the following pieces of retrieved context to answer the question.
            If no context is present or if you don't know the answer, just say that you don't know.
            Do not make up the answer unless it is there in the provided context.
            Keep the answer concise and to the point with regard to the question.
            Question:
            {question}
            Context:
            {context}
            Answer:
         """
prompt_template = ChatPromptTemplate.from_template(prompt)

Load Connection to LLM

from langchain_community.llms import Ollama
deepseek = Ollama(model="deepseek-r1:1.5b")

LangChain Syntax for RAG Chain

from langchain.chains import Retrieval
rag_chain = Retrieval.from_chain_type(llm=deepseek,
                                           chain_type="stuff",
                                           retriever=similarity_threshold_retriever,
                                           chain_type_kwargs={"prompt": prompt_template})
query = "Tell the Leaders’ Perspectives on Agentic AI"
rag_chain.invoke(query)
{'query': 'Tell the Leaders’ Perspectives on Agentic AI',

Checkout our detailed articles on DeepSeek working and comparison with similar models:

• DeepSeek R1- OpenAI’s o1 Biggest Competitor is HERE!
• Building AI Application with DeepSeek-V3
• DeepSeek-V3 vs GPT-4o vs Llama 3.3 70B
• DeepSeek V3 vs GPT-4o: Which is Better?
• DeepSeek R1 vs OpenAI o1: Which One is Better?
• How to Access DeepSeek Janus Pro 7B?

Conclusion

Building a RAG system using DeepSeek R1 provides a cost-effective and powerful way to enhance document retrieval and response generation. With its open-source nature and strong reasoning capabilities, it is a great alternative to proprietary solutions. Businesses and developers can leverage its flexibility to create AI-driven applications tailored to their needs.

Want to build applications using DeepSeek? Checkout our Free DeepSeek Course today!

Pankaj Singh

Hi, I am Pankaj Singh Negi - Senior Content Editor | Passionate about storytelling and crafting compelling narratives that transform ideas into impactful content. I love reading about technology revolutionizing our lifestyle.