Retrieval-Augmented Generation (RAG): From Theory to LangChain Implementation | by Leonie Monigatti | Nov, 2023

Step one is to gather and cargo your knowledge — For this instance, you’ll use President Biden’s State of the Union Tackle from 2022 as extra context. The uncooked textual content doc is obtainable in LangChain’s GitHub repository. To load the info, You should utilize one among LangChain’s many built-in DocumentLoaders. A Doc is a dictionary with textual content and metadata. To load textual content, you’ll use LangChain’s TextLoader.

import requestsfrom langchain.document_loaders import TextLoader

url = “https://uncooked.githubusercontent.com/langchain-ai/langchain/grasp/docs/docs/modules/state_of_the_union.txt”res = requests.get(url)with open(“state_of_the_union.txt”, “w”) as f:f.write(res.textual content)

loader = TextLoader(‘./state_of_the_union.txt’)paperwork = loader.load()

Subsequent, chunk your paperwork — As a result of the Doc, in its unique state, is simply too lengthy to suit into the LLM’s context window, you must chunk it into smaller items. LangChain comes with many built-in textual content splitters for this goal. For this easy instance, you should utilize the CharacterTextSplitter with a chunk_size of about 500 and a chunk_overlap of fifty to protect textual content continuity between the chunks.

from langchain.text_splitter import CharacterTextSplittertext_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)chunks = text_splitter.split_documents(paperwork)

Lastly, embed and retailer the chunks — To allow semantic search throughout the textual content chunks, you must generate the vector embeddings for every chunk after which retailer them along with their embeddings. To generate the vector embeddings, you should utilize the OpenAI embedding mannequin, and to retailer them, you should utilize the Weaviate vector database. By calling .from_documents() the vector database is robotically populated with the chunks.

from langchain.embeddings import OpenAIEmbeddingsfrom langchain.vectorstores import Weaviateimport weaviatefrom weaviate.embedded import EmbeddedOptions

consumer = weaviate.Shopper(embedded_options = EmbeddedOptions())

vectorstore = Weaviate.from_documents(consumer = consumer, paperwork = chunks,embedding = OpenAIEmbeddings(),by_text = False)

Step 1: Retrieve

As soon as the vector database is populated, you may outline it because the retriever element, which fetches the extra context based mostly on the semantic similarity between the person question and the embedded chunks.

retriever = vectorstore.as_retriever()

Step 2: Increase

Subsequent, to reinforce the immediate with the extra context, you must put together a immediate template. The immediate might be simply custom-made from a immediate template, as proven under.

from langchain.prompts import ChatPromptTemplate

template = “””You might be an assistant for question-answering duties. Use the next items of retrieved context to reply the query. If you do not know the reply, simply say that you do not know. Use three sentences most and preserve the reply concise.Query: {query} Context: {context} Reply:”””immediate = ChatPromptTemplate.from_template(template)

print(immediate)

Step 3: Generate

Lastly, you may construct a sequence for the RAG pipeline, chaining collectively the retriever, the immediate template and the LLM. As soon as the RAG chain is outlined, you may invoke it.

from langchain.chat_models import ChatOpenAIfrom langchain.schema.runnable import RunnablePassthroughfrom langchain.schema.output_parser import StrOutputParser

llm = ChatOpenAI(model_name=”gpt-3.5-turbo”, temperature=0)

rag_chain = ({“context”: retriever, “query”: RunnablePassthrough()} | immediate | llm| StrOutputParser() )

question = “What did the president say about Justice Breyer”rag_chain.invoke(question)

“The president thanked Justice Breyer for his service and acknowledged his dedication to serving the nation. The president additionally talked about that he nominated Decide Ketanji Brown Jackson as a successor to proceed Justice Breyer’s legacy of excellence.”

You possibly can see the ensuing RAG pipeline for this particular instance illustrated under:

This text coated the idea of RAG, which was introduced within the paper Retrieval-Augmented Era for Data-Intensive NLP Duties [1] from 2020. After masking some concept behind the idea, together with motivation and downside answer, this text transformed its implementation in Python. This text applied a RAG pipeline utilizing an OpenAI LLM together with a Weaviate vector database and an OpenAI embedding mannequin. LangChain was used for orchestration.