RAG is an abbreviation of Retrieval Augmented Generation. Let’s breakdown this term to get a clear overview of what RAG is:
R -> Retrieval
A -> Augmented
G -> Generation
So basically, the LLM that we use today is not up to the date. If I ask a question to a LLM let’s say ChatGPT, it may be hallucinated and give us the incorrect answer. To overcome this situation, we train our LLM with some more data(data which is only accessible to limited people, not globally). Then we ask some questions to the LLM trained on that data. Surely, it will give us the relevant information. Here are the some situation that may occur if we don’t use RAG:
- Increasing possibility of hallucination
- LLM is outdated
- Reduced Accuracy and Factual information
You can have a look at the diagram mentioned below:
RAG is a hybrid system which combines the strength of a retrieval based system with LLMs to generate more accurate, relevant and informed decisions. This method leverages external knowledge sources during the generation process, enhancing the model’s ability to provide up-to-date and contextually appropriate information. In the above diagram:
- In the first step, the user asks the query to the LLM.
- The query is then sent to the
- The
- The retrieved documents, along with the original query, are sent to the language model (LLM).
- The generator processes both the query and the relevant documents to generate a response, which is then sent back to the user.
Now I know you are fully interested in learning RAG from basic to advanced. Now let me tell you the perfect roadmap to learn RAG in just 5 days. Yes, you heard it right, in just 5 days you can learn the RAG system. Let’s dive straight into the roadmap:
Day 1: Build a Foundation for RAG
The core objective of day 1 is understanding the RAG at a high level and exploring what are the key components of RAG. Below are the breakdown of the topics for day 1
Overview of RAG:
- Recognize RAG’s functions, significance, and place in contemporary NLP.
- The main idea is that retrieval-augmented generation improves generative models by incorporating outside information.
Key Components:
- Learn about retrieval and generation separately.
- Look into the architectures for both retrieval (e.g., dense passage retrieval (DPR), BM25) and generation (e.g., GPT, BART, T5).
Day 2: Building your own Retrieval System
The core objective of day 2 is to Successfully implement a retrieval system (even a basic one).Below are the breakdown of the topics for day 2
Deep Dive into Retrieval Models:
- Learn about Dense Retrieval vs. Sparse Retrieval:
- Dense: DPR, ColBERT.
- Sparse: BM25, TF-IDF.
- Discover the advantages and disadvantages of each method.
Implementation of Retrieval:
- Use libraries such as elasticsearch for sparse retrieval or faiss for dense retrieval to carry out basic retrieval tasks.
- Work through Hugging Face’s DPR tutorial to understand how to retrieve relevant documents from a knowledge base.
Knowledge Databases:
- Understand how knowledge bases are structured.
- Learn how to prepare data for retrieval tasks, such as pre-processing a corpus and indexing documents.
Day 3: Fine-tune a generative model and observe the results
The goal of day 3 is to Fine-tune a generative model and observe the results. Understand the role of retrieval in augmenting generation. Below are the breakdown of the topics for day 3
Deep Dive into Generative Models:
- Examine trained models such as T5, GPT-2, and BART.
- Learn the fine-tuning process for generation tasks such as question-answering or summarization.
Hands-on with Generative Models:
- Apply the transformers provided by Hugging Face to refine a model on a short dataset.
- Test generating answers to questions using the generative model.
Exploring the Interaction Between Retrieval and Generation:
- Examine the generative model’s input methods for retrieved data.
- Recognize how retrieval enhances the precision and caliber of responses that are generated.
Day 4: Implement a working RAG system
Now, we are getting closer to the goal. The main objective of this day is to Implement a working RAG system on a simple dataset and Gain familiarity with tweaking parameters.Below are the breakdown of the topics for day 4
Combining Retrieval and Generation:
- Combine the components for generation and retrieval into a single system.
- Implement the interaction between retrieval outputs and the generative model.
Using Llamaindex’s RAG Pipeline:
- Go through the official documentation or a tutorial to learn how the RAG pipeline functions.
- Utilizing LlamaIndex’s RAG model, set up and execute an example.
Hands-on Experimentation:
- Start experimenting with different parameters like the number of documents retrieved, beam search strategies for generation, and temperature scaling.
- Try running the model on simple knowledge-intensive tasks
Day 5: Build and Fine-tune a More Robust RAG System
The goal of this last day to create a more robust RAG model by Finetuning it and get knowledge about the different types of RAG models that you can explore. Below are the breakdown of the topics for day 5
- Advanced Fine-Tuning: Examine how to optimize the generation and retrieval components for tasks that are specific to a given domain.
- Scaling Up: Use bigger datasets and more intricate knowledge bases to increase the size of your RAG system.
- Performance Optimization: Learn how to maximize memory consumption and retrieval speed (for example, by utilizing faiss with GPU).
- Evaluation: Acquire the skillset to assess RAG models in knowledge-intensive jobs. utilizing various metrics BLEU, ROUGE, and more measures for addressing questions.
End Note
By following this roadmap, you can learn the RAG system within 5 days depending upon your learning capabilities. I hope you like this roadmap. I usually share Generative AI stuff in the form of a carousel or you can say a bit sized informative post. You can check more carousels on my Linkedin Profile.
If you are looking want to build your RAG from scratch, tune into our FREE course on building RAG system using LlamaIndex!
’m a Generative AI enthusiast, exploring the limitless possibilities of Generative AI, where creativity meets technology. With a passion for the evolving landscape of artificial intelligence, I dive deep into the innovations shaping our future, from text generation to creative visualizations. Continuously fascinated by the intersection of machine learning and human ingenuity, I’m driven by a curiosity to understand and contribute to this ever-growing field.
