06. GenAI Engineering: FastAPI, Databases, Caching & Observability

39. Generative AI with FastAPI

🇬🇧 English Definition

FastAPI provides an asynchronous, high-performance web framework to expose LLM pipelines and RAG endpoints as production REST APIs with Pydantic request validation and OpenAPI docs.

🇮🇳 Hinglish Definition

FastAPI python backend framework hai jo GenAI pipelines ko fast, async web APIs mein convert karta hai with automatic request validation aur swagger documentation.

// FastAPI GenAI Endpoint Skeleton
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title="GenAI Microservice")

class QueryRequest(BaseModel):
    user_query: str
    temperature: float = 0.7

@app.post("/api/v1/generate")
async def generate_response(req: QueryRequest):
    // Simulate async LLM pipeline call
    return {
        "status": "success",
        "query": req.user_query,
        "answer": f"Processed query '{req.user_query}' asynchronously."
    }

print("FastAPI Route Handler Ready.")
            

🔍 Line-by-Line Code Breakdown

async def generate_response: Asynchronous request handler preventing thread blocking during I/O-bound LLM API calls.

🎯 Interview Focus & Questions

Q: Why is async/await critical when calling LLM APIs in FastAPI?
A: LLM API calls are I/O bound and take hundreds of milliseconds; `async` allows the FastAPI worker loop to handle thousands of concurrent requests while waiting for model responses.

40. Databases and GenAI Applications

🇬🇧 English Definition

Production GenAI applications rely on a multi-database architecture combining Relational DBs (PostgreSQL for users/billing), Vector DBs (Qdrant for embeddings), Object Storage (S3 for raw files), and Caches (Redis).

🇮🇳 Hinglish Definition

Production app mein multiple databases hotey hain: SQL DB (Users & Billing), Vector DB (Embeddings), S3/Object Storage (Raw PDFs), aur Redis (Fast Caching).

// Production Data Storage Routing System
data_architecture = {
    "PostgreSQL": "User profiles, subscription tiers, API logs, transaction history",
    "Qdrant": "Vector embeddings of enterprise documents for RAG",
    "AWS S3": "Raw uploaded PDFs, images, generated audio & video files",
    "Redis": "Semantic query caching & fast session chat memory"
}

for storage, purpose in data_architecture.items():
    print(f"[{storage}] -> {purpose}")
            

🔍 Line-by-Line Code Breakdown

data_architecture: Clear separation of concerns matching workload requirements to optimal database engines.

🎯 Interview Focus & Questions

Q: What is pgvector in PostgreSQL?
A: pgvector is an open-source extension for PostgreSQL that enables storing and searching vector embeddings directly inside standard relational SQL tables.

41. Streaming Responses

🇬🇧 English Definition

Streaming sends tokens incrementally to the user via Server-Sent Events (SSE) or WebSockets as they are decoded, drastically lowering Time-to-First-Token (TTFT) perceived latency.

🇮🇳 Hinglish Definition

Streaming se answer step-by-step screen par dikhta hai as tokens arrive (Server-Sent Events), jisse user ko waiting lag zero feel hoti hai.

// Server-Sent Events (SSE) Token Generator Concept
import time

def stream_llm_tokens(prompt):
    tokens = f"Answer to '{prompt}': Generative AI streaming is ultra fast!".split()
    for token in tokens:
        yield f"data: {token}\n\n"
        time.sleep(0.05)

for chunk in stream_llm_tokens("Explain SSE"):
    print(chunk.strip())
            

🔍 Line-by-Line Code Breakdown

yield f'data: {token}\n\n': Formats string chunks according to Server-Sent Events (SSE) standard protocol.

🎯 Interview Focus & Questions

Q: What metric does Streaming directly improve?
A: Perceived latency and TTFT (Time-To-First-Token), providing immediate feedback to the end user even if total generation time remains unchanged.

42. Caching and Cost Optimization

🇬🇧 English Definition

Semantic caching checks vector similarity against previously answered user queries in Redis/GPTCache, returning cached responses instantly to reduce model API costs and latency by 90%+.

🇮🇳 Hinglish Definition

Semantic caching pehle poche gaye queries aur unke outputs ko vector DB mein cache karta hai. Agar new user similar question pochta hai, to expensive LLM call skip ho jaati hai.

// Semantic Cache Lookup Function Concept
def check_semantic_cache(new_query_vector, cache_db, threshold=0.92):
    best_match, sim_score = cache_db.search_nearest(new_query_vector)
    if sim_score >= threshold:
        return True, best_match["cached_answer"], sim_score
    return False, None, sim_score

print("Semantic Cache Evaluator Ready.")
            

🔍 Line-by-Line Code Breakdown

sim_score >= threshold: Returns cached response if semantic similarity exceeds 0.92, bypassing model execution.

🎯 Interview Focus & Questions

Q: What is Prompt Caching in Anthropic/OpenAI APIs?
A: Prompt caching allows providers to reuse pre-processed KV-cache tokens across requests when large static system prompts or context documents are identical.

43. Evaluation & Observability

🇬🇧 English Definition

GenAI observability tracks system health by logging traces, token usage, latency splits, costs, and quality scores using platforms like LangSmith, Phoenix, or OpenTelemetry.

🇮🇳 Hinglish Definition

Observability ka matlab hai har LLM request ka trace, token count, cost, latency aur error rate log karna taaki production bottlenecks aur failures track aur fix ho sakein.

// LLM Telemetry Span Logger Class
import time

class LLMTelemetrySpan:
    def __init__(self, model_name):
        self.model = model_name
        self.start_time = 0

    def __enter__(self):
        self.start_time = time.time()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        duration = time.time() - self.start_time
        print(f"[TRACE] Model: {self.model} | Latency: {duration:.4f}s | Status: {'ERROR' if exc_type else 'OK'}")

with LLMTelemetrySpan("genai-v1"):
    time.sleep(0.1) // Simulate LLM API network call
            

🔍 Line-by-Line Code Breakdown

LLMTelemetrySpan: Python Context Manager measuring exact latency span and execution status for observability logging.

🎯 Interview Focus & Questions

Q: What are the key metrics to monitor in production LLM applications?
A: TTFT (Time to First Token), total latency, input/output token counts, API cost per request, HTTP error rate, and semantic user satisfaction feedback.