Large language models (LLMs) have revolutionized natural language processing (NLP), enabling various applications, from conversational assistants to content generation and analysis. However, working with LLMs can be challenging, requiring developers to navigate complex prompting, data integration, and memory management tasks. This is where LangChain comes into play, a powerful open-source Python framework designed to simplify the development of LLM-powered applications.
LangChain addresses the difficulties of building sophisticated LLM applications by providing modular, easy-to-use components for connecting language models with external data sources and services. It abstracts away the complexities of LLM integration, enabling developers to focus on building impactful applications that leverage the full potential of these advanced language models.
As the importance of LLMs continues to grow in various domains, LangChain plays a crucial role in democratizing their use and empowering developers to create innovative solutions that can transform industries.
Here is the comprehensive LangChain Guide for you!
LangChain is an open-source orchestration framework for building applications using large language models (LLMs). Available in both Python and JavaScript-based libraries, LangChain provides a centralized development environment and set of tools to simplify the process of creating LLM-driven applications like chatbots and virtual agents.
Serving as a generic interface for integrating with various LLMs, LangChain’s modular design allows developers and data scientists to dynamically compare different prompts and even different foundation models with minimal need to rewrite code. This flexibility also enables building programs that utilize multiple LLMs together, such as one model for interpreting user queries and another for generating responses.
The way I see it, LangChain is filling an important gap – providing a common framework to build upon, so that innovators and creators don’t have to reinvent the wheel every time they want to leverage the power of LLM.
The core idea behind LangChain is to provide a modular, flexible framework for building applications that utilize large language models (LLMs). At the heart of LangChain are a few key concepts:
At the core of LangChain is the ability to seamlessly integrate with a variety of large language models (LLMs) from different providers, such as OpenAI, Anthropic, and Google. LangChain provides a standardized interface to interact with these powerful AI models, abstracting away the complexities of working with each vendor’s unique APIs and input/output formats.
LangChain’s Chains are the basic building blocks for creating complex workflows and processing pipelines. A Chain is a sequence of operations that can be performed on the outputs of an LLM. For example, you might have a Chain that first uses an LLM to extract key information from user input, then passes that to another LLM to generate a relevant response. Chaining multiple LLM-powered steps together enables developers to tackle increasingly sophisticated natural language tasks.
Also Read: A Comprehensive Guide to Using Chains in Langchain
Building on the Chains concept, LangChain introduces higher-level abstractions called Agents. Agents are self-contained units that can leverage Chains and other LangChain components to autonomously solve complex, goal-driven tasks. Agents encapsulate the logic for interacting with LLMs, managing state and memory, and coordinating multi-step workflows. This allows developers to create intelligent, LLM-powered “actors” that can engage in more natural, contextual conversations and complete intricate assignments.
A crucial capability provided by LangChain is its memory management system. This allows LLMs to store and retrieve relevant information during the course of a multi-step workflow, enabling context preservation and statefulness across executions. The memory component is essential for building conversational applications and other LLM-powered experiences that require an understanding of previous interactions and intermediate results.
By combining these core elements – LLM integrations, Chains, Agents, and Memory – LangChain gives developers a comprehensive toolkit for building sophisticated applications driven by large language models. The framework’s modular, flexible design empowers creators to experiment, iterate, and scale their LLM-powered solutions more efficiently. Rather than getting bogged down in the underlying technical complexities, they can focus on the specific use case and workflow, seamlessly swapping out components as needed to find the optimal configuration.
To install LangChain, you can use pip, the package installer for Python. Run the following command:
!pip install langchain
Setting up an LLM provider (e.g., OpenAI, Anthropic, Cohere):
LangChain supports integration with various large language model providers. In this example, we’ll set up the OpenAI provider. First, install the necessary dependency:
!pip install qU langchain-openai
Next, import the required modules and set your OpenAI API key as an environment variable:
import getpass
import os
os.environ["OPENAI_API_KEY"] = getpass.getpass()
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-3.5-turbo")
With the LLM provider set up, we can now interact with the language model. Here’s a basic example of using the model for translation:
from langchain_core.messages import HumanMessage, SystemMessage
messages = [
SystemMessage(content="Translate the following from English into Italian"),
HumanMessage(content="hi!"),
]
model.invoke(messages)
This will return an `AIMessage` object containing the model’s response and metadata about the response.
To extract just the string response, we can use an output parser:
from langchain_core.output_parsers import StrOutputParser
parser = StrOutputParser()
result = model.invoke(messages)
parser.invoke(result)
In this example, we first create a list of messages representing the conversation context and the input to translate. Using the ‘ invoke ‘ method, we then invoke the language model with these messages. The model returns an `AIMessage` object containing the translation in Italian (`’Ciao!’`) along with additional metadata.
Using LangChain’s modular components, you can easily set up and interact with various large language models, enabling you to build sophisticated NLP applications with relative ease.
To read and split a PDF document, you can use the `PyPDFLoader` class from `langchain_community.document_loaders`:
Installing dependencies:
!pip install pypdf
from langchain_community.document_loaders import PyPDFLoader
loader = PyPDFLoader("2310.06625v4.pdf")
pages = loader.load_and_split()
print(pages[1].page_content)
Effective text splitting and chunking are essential for handling large documents. The `CharacterTextSplitter` class can split documents into smaller chunks, which are easier to process and manage.
This is the simplest method. This splits based on characters (by default “\n\n”) and measures
chunk length by number of characters.
from langchain.text_splitter import CharacterTextSplitter
# Assuming you have a list of pages loaded
page = pages[0] # Get the first page
# Get the text content of the first page
page_content = page.page_content
# Create a CharacterTextSplitter instance
text_splitter = CharacterTextSplitter(
chunk_size=100, # Adjust the chunk size as needed
chunk_overlap=20, # Adjust the chunk overlap as needed
separator="\n" # Use newline character as the separator
)
# Split the page content into chunks
chunks = text_splitter.split_text(page_content)
chunks
Output
Vector stores are critical for storing and retrieving document embeddings. This walkthrough showcases basic functionality related to vector stores. A key part of working with vector stores is creating the vector to put in them, usually created via embeddings. Therefore, it is recommended that you familiarize yourself with the text-embedding model interfaces before diving into this. There are many great vector store options; a few are free, open-source, and run entirely on your local machine. Review all integrations for many great hosted offerings.
Here’s an example using the Chroma vector store:
## this code is if you have latest version of the langchain installed
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')
from langchain.document_loaders import TextLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma
# Load your documents (assuming 'pages' is already loaded)
text_splitter = CharacterTextSplitter(chunk_size=1000,
chunk_overlap=0)
documents = text_splitter.split_documents(pages)
# Create the embeddings
embeddings = OpenAIEmbeddings()
# Create the Chroma vector store
db = Chroma.from_documents(documents, embeddings)
query = "What is i transformer"
docs = db.similarity_search(query)
print(docs[0].page_content)
This code creates embeddings for the documents and stores them in a Chroma vector store, enabling efficient similarity search queries.
Chains refer to sequences of operations, including calls to LLMs, tools, or data preprocessing steps. They are essential for creating complex workflows by linking multiple components together.
LCEL is great for constructing chains, but using chains already on the shelf is also nice.
Chains built with LCEL: LangChain offers a higher-level constructor method in this case. However, all that is being done under the hood is constructing a chain with LCEL. Chains are constructed by subclassing from a legacy Chain class. These chains do not use LCEL under the hood but are the standalone classes. We are working on creating methods that create LCEL versions of all chains. We are doing this for a few reasons.
Here, we are going to explore only about the LCEL Chains:
LLM Chain: Chain to run queries against LLMs
from langchain_core.prompts import PromptTemplate
from langchain_openai import OpenAI
prompt_template = "Tell me a {adjective} joke"
prompt = PromptTemplate(
input_variables=["adjective"], template=prompt_template
)
llm = OpenAI()
chain = prompt | llm
result=chain.invoke("your adjective here")
print(result)
Combining and Customizing Chains for Complex Tasks
Chains can be combined and customized to handle more complex tasks. By linking multiple chains, you can create sophisticated workflows that leverage various capabilities of LLMs and tools.
In LangChain, agents are built to expand the functionality of LLMs by enabling them to interact with diverse tools and data sources. These agents can dynamically make decisions, execute actions, and retrieve information.
There are several types of agents, including ZeroShotAgent and ConversationalAgent. Each type is suited for different tasks:
Next, let’s define some tools to use. Let’s write a really simple Python function to calculate the length of a word that is passed in.
## Loading the model first
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
from langchain.agents import tool
@tool
def get_word_length(word: str) -> int:
"""Returns the length of a word."""
return len(word)
get_word_length.invoke("abc")
#output = 3
tools = [get_word_length]
Now, let us create the prompt. Because OpenAI Function Calling is finetuned for tool usage, we hardly need any instructions on how to reason or how to output format. We will just have two input variables: input and agent_scratchpad.
Input should be a string containing the user objective. agent_scratchpad should be a message sequence containing the previous agent tool invocations and the corresponding tool outputs.
from langchain_core.prompts import ChatPromptTemplate,
MessagesPlaceholder
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are very powerful assistant, but don't know current
events",
),
("user", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
]
)
How does the agent know what tools it can use? In this case, we rely on an OpenAI tool called LLMs, which takes tools as a separate argument. We have been specifically trained to know when to invoke those tools. To pass our tools to the agent, we just need to format them in the OpenAI tool format and pass them to our model. (By binding the functions, we ensure they’re passed each time the model is invoked.)
llm_with_tools = llm.bind_tools(tools)
After putting those pieces together, we can now create the agent. We will import two last utility functions: a component for formatting intermediate steps (agent action, tool output pairs) to input messages that can be sent to the model and a component for converting the output message into an agent action/agent finish.
from langchain.agents.format_scratchpad.openai_tools import (
format_to_openai_tool_messages,
)
from langchain.agents.output_parsers.openai_tools import
OpenAIToolsAgentOutputParser
agent = (
{
"input": lambda x: x["input"],
"agent_scratchpad": lambda x: format_to_openai_tool_messages(
x["intermediate_steps"]
),
}
| prompt
| llm_with_tools
| OpenAIToolsAgentOutputParser()
)
from langchain.agents import AgentExecutor
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
list(agent_executor.stream({"input": "How many letters in the word
eudca"}))
This is great – we have an agent! However, this agent is stateless – it doesn’t remember anything about previous interactions. This means you can’t ask follow-up questions easily. Let’s fix that by adding in memory. To do this, we need to do two things:
Add a place for memory variables in the prompt. Keep track of the chat history. First, let’s add a place for memory in the prompt. We do this by adding a message placeholder with the key “chat_history.” Notice that we put this above the new user input (to follow the conversation flow).
Code:
from langchain_core.prompts import MessagesPlaceholder
MEMORY_KEY = "chat_history"
prompt = ChatPromptTemplate.from_messages([
(
"system",
"You are very powerful assistant, but bad at calculating lengths of words.",
),
MessagesPlaceholder(variable_name=MEMORY_KEY),
("user", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
])
from langchain_core.messages import AIMessage, HumanMessage
chat_history = []
agent = (
{
"input": lambda x: x["input"],
"agent_scratchpad": lambda x: format_to_openai_tool_messages(x["intermediate_steps"]),
"chat_history": lambda x: x["chat_history"],
}
| prompt
| llm_with_tools
| OpenAIToolsAgentOutputParser()
)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
input1 = "how many letters in the word educa?"
result = agent_executor.invoke({"input": input1, "chat_history": chat_history})
chat_history.extend([HumanMessage(content=input1), AIMessage(content=result["output"])])
agent_executor.invoke({"input": "is that a real word?", "chat_history": chat_history})
Memory management is crucial in LangChain applications, especially in multistep workflows, where maintaining context is essential for coherent and accurate interactions. This section delves into the importance of memory and the types of memory used, and it provides examples and use cases to illustrate its application.
Memory ensures that the application can retain information across multiple interactions in multistep workflows. This capability is vital for creating conversational agents that remember previous exchanges and provide relevant, context-aware responses. Each interaction would be independent without memory, leading to disjointed and less useful dialogues.
LangChain supports different types of memory to suit various needs:
ConversationBufferMemory stores the conversation history in a buffer. This type of memory is suitable for scenarios where maintaining a sequential record of interactions is important. It helps the model remember previous interactions and use that context to generate more coherent and contextually relevant responses.
Code
from langchain_openai import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain_core.prompts import ChatPromptTemplate,MessagesPlaceholder
from langchain_core.runnables import RunnableLambda,RunnablePassthrough
model=ChatOpenAI(model_name='gpt-3.5-turbo', temperature=0.0)
prompt= ChatPromptTemplate.from_messages(
[
("system", "Act as a helpful Al Assistant"),
MessagesPlaceholder(variable_name="history"),
("human", "(input)"),
]
)
memory=ConversationBufferMemory(return_messages=True)
from operator import itemgetter
chain=(
RunnablePassthrough.assign
(
history=RunnableLambda(memory.load_memory_variables)
|
itemgetter("history")
)
|
prompt
|
model
)
#use the conversation chain
user_input={'input':'what are the first four colors of a rainbow'}
response=chain.invoke(user_input)
#saving the context
memory.save_context(user_input,{'output':response.content})
print('First Response: ',response.content)
user_input={'input':'And the last 3 ?'}
response=chain.invoke(user_input)
#also using memory present in the chain
memory.save_context(user_input,{'output':response.content})
print('Second Response: ',response.content)
user_input = {'input': 'Which color is in the center of a rainbow?'}
response = chain.invoke(user_input)
memory.save_context(user_input, {'output': response.content})
print('Third Response: ', response.content)
Also Read: How to Build a LangChain Chatbot with Memory?
LangChain has found numerous applications across various industries due to its powerful capabilities in handling large language models (LLMs) and maintaining conversational memory. Some practical applications include:
LangChain’s roadmap includes several exciting features aimed at enhancing its capabilities:
The integration of LLMs into different sectors is poised to revolutionize how businesses operate and interact with their customers:
As LLMs become more prevalent, it is crucial to address ethical concerns and promote responsible AI practices:
LangChain offers a robust framework for building applications with large language models. It provides features like conversational memory that enhance user experience and interaction quality. Its practical applications across various industries demonstrate its potential to revolutionize customer support, healthcare, education, and more.
By democratizing LLM development, LangChain empowers developers and businesses to harness the power of advanced language models. As LangChain continues to evolve, it will play a crucial role in shaping the future of AI-driven applications.
If you’re looking to master LangChain and other Generative AI concepts, don’t miss out on our GenAI Pinnacle Program.
A. LangChain is used to build applications powered by language models, enabling them to interact with data sources, perform actions, and execute complex workflows.
A. LangChain is popular for its ability to extend the functionality of language models, offering tools to build versatile, dynamic applications.
A. Install LangChain via pip, set up an LLM provider like OpenAI, and interact with the model using simple code snippets provided in the LangChain documentation.
A. LangChain supports Conversational Memory, Buffer Memory, and Entity Memory, which are crucial for maintaining context and coherence in multistep workflows.
A. LangChain is used in customer support, healthcare, education, and content creation to develop intelligent, context-aware applications and improve user interactions.
A. Yes, LangChain is a Python library designed to simplify the development of applications utilizing language models.
A. LangChain can integrate with ChatGPT and other large language models to enhance application functionality and decision-making capabilities.