Retrieval-Augmented Generation (RAG)
Build developer-document search with LangChain and pgvector, then answer from retrieved text and charts with checked source references.
- Prerequisites
- Python, JSON, basic SQL tables, embeddings and cosine similarity.
- Duration
- about 2 hours of reading
- Cost
- Free, no sign-up
What you'll learn
- Split and embed documents with LangChain while recording the model and chunk settings.
- Use PGVectorStore to replace, filter and delete indexed chunks.
- Compare exact, HNSW and IVFFlat results with the same saved questions.
- Retrieve source chunks and chart images, then check citation IDs and excerpts.
Lessons
Build an embedding pipeline with LangChainRead
Split internal developer documentation with LangChain, embed chunks through Ollama, and keep the model and chunk settings beside the vectors.
Build a PostgreSQL vector store with LangChainRead
Use PGVectorStore to insert, replace, filter and delete internal documentation chunks, then test retries and a partially written batch.
Compare exact, HNSW and IVFFlat search in pgvectorRead
Compare pgvector exact, HNSW and IVFFlat results through PGVectorStore, check the query plan, and measure overlap on saved developer questions.
Build a RAG pipeline with citationsRead
Retrieve internal developer documentation with PGVectorStore, ask Gemini for source IDs and quotes, and reject citations outside the supplied chunks.
Build multimodal RAG with chart source referencesRead
Retrieve chart descriptions with PGVectorStore, send selected PNGs to Gemini, and check image identity separately from quoted source text.
Inspect cosine distance under the hoodRead
Use NumPy and saved real embeddings to inspect cosine distance, reject invalid vector shapes, and see why metadata filters still matter.