03. RAG Architecture, Vector DBs & Evaluation

22. RAG — Retrieval-Augmented Generation

🇬🇧 English Definition

Retrieval-Augmented Generation (RAG) retrieves relevant external facts from a knowledge index (vector store/database) and injects them into the prompt context so the LLM grounds its answer in factual domain data.

🇮🇳 Hinglish Definition

RAG architecture enterprise documents (PDFs, policies, DBs) ko embeddings banana aur user query ke time exact matching chunks retrieve karke prompt context mein bhejta hai taaki model accurately facts bol sake.

// Basic RAG Workflow Simulation in Python
def simple_rag(query, vector_db, llm_client):
    // 1. Convert user query into dense vector
    query_vector = embed(query)
    // 2. Retrieve top-k relevant document chunks from Vector DB
    retrieved_chunks = vector_db.search(query_vector, top_k=3)
    // 3. Construct grounded prompt context
    context_str = "\n".join(retrieved_chunks)
    prompt = f"Context:\n{context_str}\n\nQuestion: {query}\nAnswer based ONLY on context:"
    // 4. Generate grounded completion
    return llm_client.generate(prompt)

print("RAG Pipeline Ready.")
            

🔍 Line-by-Line Code Breakdown

vector_db.search(query_vector, top_k=3): Performs semantic search returning top matching text chunks.
prompt construction: Bounds model answer explicitly within retrieved text context.

🎯 Interview Focus & Questions

Q: Why choose RAG over Fine-Tuning for real-time document Q&A?
A: RAG allows instant knowledge updates without expensive GPU retraining, provides source attribution (citations), and guarantees strict permission access control.

23. Document Ingestion and Chunking

🇬🇧 English Definition

Ingestion extracts clean text and metadata from files (PDF, HTML, DOCX). Chunking splits long documents into optimal retrieval units using strategies like fixed-size overlap or semantic splitting.

🇮🇳 Hinglish Definition

Ingestion PDF/DOCX se text extract aur clean karta hai. Chunking content ko smaller chunks (e.g., 512 tokens with 50 token overlap) mein break karta hai taaki embedding quality aur retrieval precision intact rahe.

// Fixed-Size Overlapping Chunking Function in Python
def chunk_text(text, chunk_size=200, overlap=40):
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        end = start + chunk_size
        chunk = " ".join(words[start:end])
        chunks.append(chunk)
        start += (chunk_size - overlap)
    return chunks

sample_doc = "Generative AI systems rely heavily on high quality retrieval pipelines. " * 50
chunks = chunk_text(sample_doc)
print(f"Total Chunks Created: {len(chunks)}, First Chunk Length: {len(chunks[0].split())} words")
            

🔍 Line-by-Line Code Breakdown

chunk_size & overlap: Slide window across text array ensuring context across chunk boundaries is preserved.

🎯 Interview Focus & Questions

Q: What is the risk of using too small or too large chunk sizes in RAG?
A: Chunks that are too small lose surrounding semantic context; chunks that are too large dilute vector embeddings and fill the model's context window with irrelevant noise.

24. Vector Databases

🇬🇧 English Definition

Vector databases (FAISS, Chroma, Qdrant, Milvus, Pinecone) index dense embeddings and support fast Approximate Nearest Neighbor (ANN) vector similarity searches using algorithms like HNSW or IVF.

🇮🇳 Hinglish Definition

Vector DB semantic search engine hota hai jo embeddings ko HNSW / IVF indexes mein store karta hai aur query vector ke nearest neighbor chunks milliseconds mein locate karta hai.

// In-Memory Vector Search Simulation with Cosine Distance
import numpy as np

class InMemoryVectorStore:
    def __init__(self):
        self.vectors = []
        self.texts = []

    def add(self, text, vector):
        self.texts.append(text)
        self.vectors.append(vector / np.linalg.norm(vector))

    def search(self, query_vec, k=2):
        query_norm = query_vec / np.linalg.norm(query_vec)
        scores = [np.dot(query_norm, v) for v in self.vectors]
        top_indices = np.argsort(scores)[-k:][::-1]
        return [(self.texts[i], scores[i]) for i in top_indices]

db = InMemoryVectorStore()
db.add("Python is great for AI", np.array([0.1, 0.9, 0.2]))
db.add("FastAPI is an async framework", np.array([0.8, 0.2, 0.1]))

results = db.search(np.array([0.15, 0.85, 0.25]))
print("Top Result:", results[0])
            

🔍 Line-by-Line Code Breakdown

InMemoryVectorStore: Minimal vector index performing normalized dot product (cosine similarity) ranking across stored vectors.

🎯 Interview Focus & Questions

Q: What is HNSW in Vector Databases?
A: HNSW (Hierarchical Navigable Small World) is a graph-based Approximate Nearest Neighbor algorithm that enables logarithmic search scaling over millions of high-dimensional vectors.

25. RAG Retrieval Strategies

🇬🇧 English Definition

Advanced retrieval strategies improve ground truth recall by combining dense vector similarity with sparse lexical BM25 search (Hybrid Search), metadata filtering, query expansion, and cross-encoder reranking.

🇮🇳 Hinglish Definition

Basic vector search exact keyword match miss kar sakta hai. Advanced RAG Hybrid Search (Dense Embeddings + BM25 Lexical) and Reranking (Cross-Encoder) combine karta hai best accuracy ke liye.

// Hybrid Search Score Fusion (Reciprocal Rank Fusion Concept)
def reciprocal_rank_fusion(dense_ranks, sparse_ranks, k=60):
    rrf_scores = {}
    for item in set(dense_ranks.keys()).union(sparse_ranks.keys()):
        r1 = dense_ranks.get(item, 1000)
        r2 = sparse_ranks.get(item, 1000)
        score = (1.0 / (k + r1)) + (1.0 / (k + r2))
        rrf_scores[item] = score
    return sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)

dense = {"doc1": 1, "doc2": 2, "doc3": 3}
sparse = {"doc2": 1, "doc1": 3, "doc4": 2}
print("RRF Combined Ranks:", reciprocal_rank_fusion(dense, sparse))
            

🔍 Line-by-Line Code Breakdown

reciprocal_rank_fusion(rrf): Standard algorithm combining rankings from dense embedding search and sparse keyword search without requiring score normalization.

🎯 Interview Focus & Questions

Q: What is the role of a Cross-Encoder Reranker?
A: A reranker takes the top 20 candidate chunks retrieved by bi-encoder vector search and processes the full (Query, Chunk) pair together in a deep model to compute highly precise relevance scores.

26. RAG Evaluation & Common Failures

🇬🇧 English Definition

RAG evaluation evaluates retrieval quality (Context Precision, Recall) and generation quality (Faithfulness, Answer Relevance). Frameworks like RAGAS help detect parsing errors, hallucination, and context rot.

🇮🇳 Hinglish Definition

RAG system test karne ke liye do cheezein measure hoti hain: Retrieval Quality (kya sahi doc mila?) aur Generation Quality (kya answer true aur contextual hai?). RAGAS framework is automated evaluation to measure karta hai.

// RAG Triad Evaluation Metrics Framework Concept
def evaluate_rag_triad(query, retrieved_context, generated_answer):
    // 1. Groundedness / Faithfulness (Is answer derived strictly from context?)
    grounded = True if len(retrieved_context) > 0 else False
    // 2. Answer Relevance (Does answer directly address query?)
    relevant = True if len(generated_answer) > 10 else False
    return {"Groundedness": 0.95 if grounded else 0.0, "Answer_Relevance": 0.92 if relevant else 0.0}

print("Eval Score:", evaluate_rag_triad("What is RAG?", "RAG is retrieval...", "RAG retrieves context..."))
            

🔍 Line-by-Line Code Breakdown

evaluate_rag_triad: Evaluates the three critical linkages: Query -> Context -> Answer.

🎯 Interview Focus & Questions

Q: What are the primary failure modes of a production RAG system?
A: Poor document parsing, bad chunk boundaries, irrelevant context retrieval (low recall), context overflow, and model hallucination despite having valid context.