# Build a RAG pipeline with citations

A self-contained lab using fictional internal developer documentation. This lesson uses the source, tests and recordings listed in its artifact manifest. The older pipeline lab remains in its original directory.

## Install

Use Python 3.12 or later. Install PostgreSQL 17 with its pgvector extension, and set `PG_BIN` to the server binary directory if it differs from the helper's default. The helper creates a temporary localhost-only cluster on an unused port other than 5432 and stops and deletes it on exit. Its temporary trusted role is for this local exercise, not a production authentication configuration.

```sh
python3 -m venv .venv
.venv/bin/pip install -r library-requirements-lock.txt
.venv/bin/python -m unittest -v test_library
.venv/bin/python rag.py
```

To create new embeddings, run Ollama with `qwen3-embedding:0.6b` already installed. Gemini generation uses `gemini-3.8-flash` through the Developer API; configure `GOOGLE_API_KEY` or `GEMINI_API_KEY` outside these files. Never include keys in output. Store/index/cosine tests replay saved real vectors and need no API key. Image tests use retained PNG files and Pillow.

`langchain-postgres==0.0.19` requires `pgvector<0.4`; the lock deliberately uses compatible Python package 0.3.6 rather than the incompatible newer 0.5.1. The PostgreSQL extension and Python package have separate versions. The code uses current `PGEngine`/`PGVectorStore`, not deprecated `PGVector`.

Recorded output is one run with the versions and settings it names. Chart numbers are supplied illustration values, not timings of this pipeline. Citation ID and excerpt checks do not grade whether an answer follows from its source.

When running the repository-wide registered suites, set `AIML_RAG_PYTHON` to this lab environment's Python executable. The standalone commands above do not need that variable.

## Current lesson files

- Entry point: `rag.py`.
- Shared adapters: `library.py`; database helper: `pg_cluster.py` where used.
- Dependency lock: `library-requirements-lock.txt`.
- Offline suite: `python -m unittest -v test_library`.