Building Agentic RAG Applications
Build a document-retrieval application with ingestion, hybrid search, context budgets, bounded search/read policies, FastAPI and MCP interfaces, and local evaluation and release controls.
- Prerequisites
- Basic Python functions and JSON. Model requests and embeddings are introduced in the first module; review development and held-out splits before Module 2. Module 3 uses local UTF-8 fixtures and needs no model server.
- Duration
- about 14 hours of reading
- Cost
- Free, no sign-up
What you'll learn
- Trace RAG requests, replay saved replies and consume a real Gemini provider stream.
- Ingest and version documentation, then compare lexical, vector and hybrid retrieval.
- Choose source windows and fit them into a token-bounded request.
- Compare fixed, scripted and Gemini-selected retrieval policies, then expose search and read through HTTP and MCP.
- Apply collection permissions, caching, tracing and concurrency limits, then use task evaluations to control a local release.
Lessons
Module 1 · 2 lessonsRAG fundamentals
How RAG worksRead
Learn how an application searches documents, supplies relevant passages to a language model and links its answer to sources.
Build a RAG pipelineRead
Follow a question through document search, context assembly and the model request, then check the answer’s source links.
Module 2 · 2 lessonsDefine the task and baseline
Build a RAG eval datasetRead
Create eight source-bound developer questions, label relevant passages by documentation version, and validate answerability and frozen held-out targets.
Record a RAG baselineRead
Record a fixed RAG pipeline on eight versioned questions, separate retrieval from answer checks, and preserve missing answers without retries.
Module 3 · 3 lessonsDocument ingestion
Load developer documentationRead
Load a pinned local UTF-8 documentation snapshot into records that keep the source identity, version, URL and text together.
Track passage sourcesRead
Bind an exact passage to its source path, byte hash and line span, then reject stale or altered records before displaying the text.
Handle ingestion failuresRead
Keep accepted documents and explicit failure records together, and return an incomplete batch status when a source cannot be loaded.
Module 4 · 3 lessonsMaintain the corpus
Version documents and chunksRead
Create source and chunk revisions from pinned documentation, compare text and metadata changes, and reject inconsistent saved snapshots.
Resume document ingestionRead
Resume a pinned SQLite ingestion job after an injected interruption, checking committed sources, chunk rows and atomic completion checkpoints.
Update and delete indexed contentRead
Update or delete source and chunk rows with a SQLite FTS5 index in one transaction, then check actual search results and rollback behavior.
Module 5 · 3 lessonsRetrieval fundamentals
How BM25 ranks documentsRead
Inspect native SQLite FTS5 BM25 scores, indexed terms and document lengths, and keep matching passages separate from answer validation.
Add vector retrievalRead
Query an actual LangChain vector store with preserved Qwen embeddings, refuse unrecorded inputs, and inspect scores without new model calls.
Apply filters across retrieversRead
Apply one version policy to SQLite lexical and LangChain vector retrieval before top-k, and compare eligible candidates and post-filter underfill.
Module 6 · 3 lessonsHybrid search and reranking
Build hybrid retrievalRead
Combine native BM25 and preserved-vector retrieval with reciprocal rank fusion, inspect each contribution, and retain filtering and failure records.
Rerank retrieved passagesRead
Compare coverage, Gemini relevance and a pinned cross-encoder on frozen candidates, preserving passage identity and honest ranking limits.
Compare retrieval configurationsRead
Compare frozen lexical, vector, fused and coverage-reranked configurations on preserved development questions, with exact target mapping and retained misses.
Module 7 · 3 lessonsContext engineering
Choose chunk boundariesRead
Compare fixed-character, paragraph and heading-aware chunk boundaries on a synthetic developer document, with exact source spans and stable identity checks.
Retrieve surrounding contextRead
Expand explicit seed chunks into bounded source-local context windows, merge overlaps, and preserve section, revision and seed provenance before assembly.
Assemble context within a token budgetRead
Fit validated RAG windows into a complete templated request using the pinned local tokenizer, with whole-window admission, output reserve and failure records.
Module 8 · 2 lessonsAnswer generation
Stream RAG answersRead
Compare preserved answer replay with actual Gemini SSE events, keep partial output provisional, and validate completion and context-bound source IDs.
Handle conflicting documentationRead
Filter synthetic developer documentation by the requested version, compare explicit contract directives, and withhold answer requests when claims conflict.
Module 9 · 4 lessonsBounded retrieval policies
Add search and read toolsRead
Expose bounded documentation search and passage reads, bind handles to a pinned corpus and exact request, and inspect provenance before accepting tool output.
Refine queries with a scripted policyRead
Run a scripted retrieval controller that observes source passages, revises a query under an explicit rule, and retains its attempts and stopping reasons.
Bound scripted retrieval runsRead
Bound a scripted retrieval run with action and source-byte admission checks, retain denied and failed attempts, and separate limits from task completion.
Compare fixed, scripted and Gemini retrieval policiesRead
Compare fixed, scripted and Gemini-selected retrieval policies on frozen questions, enforce tool limits, and retain invalid proposals and incomplete outcomes.
Module 10 · 3 lessonsApplication integration
Serve the RAG application with FastAPIRead
Serve the existing retrieval core through FastAPI, validate bounded requests, preserve source metadata and response hashes, and inspect local HTTP failures.
Build a retrieval inspectorRead
Build a same-origin retrieval inspector, verify response bytes and request identity before rendering, and inspect real loading, empty, limited and error states.
Connect retrieval tools through MCPRead
Expose pinned documentation tools through an actual MCP stdio server, initialize an SDK client, preserve issued read handles and inspect real tool failures.
Module 11 · 4 lessonsRuntime controls
Authorize collection searchesRead
Resolve synthetic fixture identities, filter granted collections before top-k, and recheck issued passage handles before returning source text.
Cache retrieval resultsRead
Cache completed retrieval sources with principal, policy, corpus and normalized-request binding, bounded TTL/LRU retention and explicit failure bypass.
Trace RAG requestsRead
Trace actual local retrieval executions with isolated context IDs, fixed-field redacted spans, bounded retention and reproducible failure diagnostics.
Limit concurrent RAG requestsRead
Bound active retrieval and waiting work in one process; inspect actual FIFO promotion, queue refusals, wait deadlines and cancellation-safe lease cleanup.
Module 12 · 4 lessonsEvals and release
Run RAG evals on an updated corpusRead
Compare fixed development retrieval cases across a synthetic corpus update, refuse stale target hashes, and retain deleted targets outside paired means.
Evaluate scripted retrieval tasksRead
Evaluate deterministic retrieval policies with separate execution and source checks, exact target joins, repeated trials and an explicit fixture release gate.
Release the RAG applicationRead
Stage the existing retrieval service locally, bind its files to a retained release identity, block failed task checks, and test artifact-pointer rollback.
Build the developer knowledge assistantRead
Build a local docs assistant with a browser UI, CLI, versioned retrieval, collection permissions, cited answers and application evaluation.