RAG Tutorial 2026.
Production Grade. LangChain + ChromaDB.

// 01What RAG actually is — and what it isn't

Let's be direct: most RAG tutorials online show you how to ask a question about a single PDF. That's not RAG in production. That's a weekend demo.

Retrieval-Augmented Generation is the pattern of giving an LLM access to an external knowledge base at query time. Instead of relying purely on training-time knowledge, you fetch relevant context dynamically and include it in the prompt. The model answers based on what you gave it — not just what it memorized.

Why does this actually matter in production?

• Your data changes. LLMs have a cutoff date. Your internal docs, support tickets, and product FAQs don't freeze in time.
• You can cite sources. Every answer has a paper trail — you know exactly which documents produced it.
• It's cheaper than fine-tuning. Updating a vector store costs almost nothing compared to retraining.
• You stay in control. You decide what the model can and cannot see.

// 02Architecture overview

Before touching any code — here's the complete pipeline in two phases. The key insight: ingestion and querying are separate concerns. Keep them that way in your codebase.

// 03Project setup & dependencies

# Create and activate env
python -m venv .venv
source .venv/bin/activate

# Install everything we need
pip install langchain langchain-openai langchain-community \
            chromadb openai tiktoken \
            pypdf sentence-transformers \
            python-dotenv fastapi uvicorn

OPENAI_API_KEY=sk-...
CHROMA_PERSIST_DIR=./chroma_db
EMBED_MODEL=text-embedding-3-small
LLM_MODEL=gpt-4o-mini

// 04Document ingestion pipeline

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader, DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

load_dotenv()

def build_vector_store(docs_dir="./data/docs"):
    loader = DirectoryLoader(docs_dir, glob="**/*.pdf", loader_cls=PyPDFLoader)
    docs = loader.load()
    
    splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=120)
    chunks = splitter.split_documents(docs)
    
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    vectordb = Chroma.from_documents(chunks, embeddings, persist_directory="./chroma_db")
    return vectordb

// 05Chunking strategies that actually matter

This is where most tutorials cut corners — and where most production RAG pipelines silently fail. Chunking isn't just "split every N characters." How you split determines whether retrieval returns useful context or incoherent garbage.

For most production use cases, recursive character splitting with 800 chars and ~15% overlap is where to start.

// 06Embeddings and ChromaDB

An embedding is a high-dimensional vector representing the semantic meaning of text. Two chunks about the same concept should have vectors pointing in roughly the same direction — even if they share no words in common.

ChromaDB is our vector store. It runs in-process, persists to disk, and has a clean Python API. Right call for most teams building their first production RAG. When you scale past ~500k chunks, look at Pinecone, Weaviate, or pgvector.

// 07Retrieval chain + FastAPI server

from langchain_openai import ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

def build_rag_chain():
    retriever = vectordb.as_retriever(
        search_type="mmr",
        search_kwargs={"k": 5, "fetch_k": 20}
    )
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.1)
    
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Answer using ONLY the context below. {context}"),
        ("human", "{input}"),
    ])
    
    qa_chain = create_stuff_documents_chain(llm, prompt)
    return create_retrieval_chain(retriever, qa_chain)

// 08Evaluating your RAG pipeline

Two metrics that matter most in practice:

• Context Precision — What fraction of retrieved chunks were actually relevant?
• Faithfulness — Does the answer stick to the retrieved context?

For a rigorous setup, use RAGAS — a Python library for evaluating faithfulness, answer relevancy, and context recall using LLM-based judges.

// 09Production checklist

• Cache embeddings — Redis cache cuts OpenAI bills significantly
• Handle stale documents — Track document hashes, re-ingest only changed files
• Add a reranker — Cohere Rerank or BAAI/bge-reranker delivers the biggest quality improvement
• Log everything — Question, chunks, answer, latency, feedback
• Access control from day one — Use ChromaDB's where filters

// 10Where to go from here

• Conversational memory — Add ConversationBufferWindowMemory for multi-turn chat
• Hybrid search — Combine vector similarity with BM25 via EnsembleRetriever
• HyDE — Generate a fake ideal answer, embed that for retrieval
• Parent document retrieval — Index small chunks, retrieve parent documents
• Self-RAG — Model decides whether retrieval is needed