A repeated retrieval request can reuse a result only if its context still matches. The same query text from another principal or collection is a different request. A cached failure is also not a successful retrieval result.
When you need this. Use this pattern to reuse completed local retrieval sources while keeping caller, policy, corpus and pipeline changes separate.
The endpoint performs a fixed literal search followed by exact passage reads. It uses the same pinned Python summaries and synthetic collection policy as the authorization lesson. It neither rewrites queries nor generates answers. The cache stores materialized source data, not executable search handles.
Authorize before constructing the cache identity
POST /retrieve resolves the public fixture Bearer token and checks the requested collection before looking in the cache. The caller cannot submit a principal, policy identity or bypass field. The strict request model normalizes query case and fills declared version and top-k defaults.
The canonical key includes that normalized request, resolved principal, current policy hash, corpus snapshot and fixed pipeline configuration. An omitted default and its explicit equivalent share an entry. A changed principal, collection, version, allowance, policy or corpus does not.
Revoked grant. A previous hit cannot authorize a new request. Current collection access is checked before lookup, and trusted policy changes also change the key context. Changed context. The key regression checks principal, version, corpus and policy changes separately.
Store completed materialized source data rather than live handles
On a miss, the wrapper runs eligible native lexical search and validates passage reads through the same authorization core. A nonempty completed result contains pinned text, source identity, version, revision, span and hashes. It omits executable issued IDs. There is no public read endpoint in this cache lab.
A hit therefore returns source data that does not depend on the live handle registry. The cache keeps a deep copy on insertion and returns a fresh deep copy on lookup. The result hash describes the materialized payload; it is a consistency check, not server authentication.
Changed returned payload. Mutating a caller’s returned dictionary cannot change a later hit. Incomplete retrieval. A search with no eligible sources bypasses storage. Tool failure. A failed tool produces a safe service error and is attempted again on the next request instead of being cached.
Bound lifetime and least-recently-used retention
TTL means time to live. Each entry expires at a monotonic deadline measured from insertion. A hit promotes its least-recently-used position without extending that deadline. Entry and serialized-byte budgets bound retained cache content; an oversize entry is not stored.
A lock protects all cache bookkeeping, including expiry, promotion, byte accounting and eviction. The byte budget counts canonical serialized payload and key bytes rather than the full Python heap. Shutdown clears local state. This cache has no persistence, distributed coherence or single-flight suppression for concurrent misses.
Expired entry. Lookup removes an entry at its deadline before it can count as a hit. The recorded HTTP run waits on the real clock and observes a fresh miss. The separate named fake-clock unit test makes exact expiry and eviction boundaries reproducible; it is not an HTTP timing measurement.
| Executed case | HTTP status | Outcome |
|---|---|---|
| first miss | 200 | miss |
| normalized hit | 200 | hit |
| other principal miss | 200 | miss |
| unknown identity | 401 | unauthenticated |
| collection denied | 403 | collection_denied |
| empty bypass first | 200 | bypass |
| empty bypass again | 200 | bypass |
| real ttl expired miss | 200 | miss |
| tool failure first | 503 | tool_failure |
| tool failure again | 503 | tool_failure |
Execute misses, hits and failure bypass
Download the complete result-cache lab. Its README describes the pinned runtime and synthetic fixture credentials.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.lock"""Thread-safe bounded TTL/LRU for completed materialized retrieval results."""
import copy
import json
import math
import threading
import time
from collections import OrderedDict
class ResultCache:
def __init__(self, *, ttl=1.0,max_entries=4,max_bytes=65536,clock=time.monotonic):
if isinstance(ttl,bool) or not isinstance(ttl,(int,float)) or not math.isfinite(ttl) or not 0<ttl<=3600:raise ValueError('invalid_ttl')
if type(max_entries) is not int or not 1<=max_entries<=1024 or type(max_bytes) is not int or not 1<=max_bytes<=1048576:raise ValueError('invalid_capacity')
self.ttl=ttl;self.max_entries=max_entries;self.max_bytes=max_bytes;self.clock=clock;self.entries=OrderedDict();self.bytes=0;self.lock=threading.RLock()
def _drop(self,key):self.bytes-=self.entries.pop(key)[2]
def _expire(self):
now=self.clock()
for key,(deadline,_,_) in list(self.entries.items()):
if deadline<=now:self._drop(key)
def get(self,key):
with self.lock:
self._expire()
if key not in self.entries:return None
self.entries.move_to_end(key)
return copy.deepcopy(self.entries[key][1])
def put(self,key,value):
if not isinstance(value,dict) or value.get('status')!='completed':return False
frozen=copy.deepcopy(value);size=len(json.dumps(frozen,sort_keys=True,separators=(',',':'),allow_nan=False).encode('utf-8'))+len(key.encode('utf-8'))
with self.lock:
self._expire()
if key in self.entries:self._drop(key)
if size>self.max_bytes:return False
self.entries[key]=(self.clock()+self.ttl,frozen,size);self.bytes+=size
while len(self.entries)>self.max_entries or self.bytes>self.max_bytes:self._drop(next(iter(self.entries)))
return True
def clear(self):
with self.lock:self.entries.clear();self.bytes=0python run.py --output runs/my-run/result.json
python -m unittest discover -vfirst_miss: HTTP 200 cache=miss
normalized_hit: HTTP 200 cache=hit
other_principal_miss: HTTP 200 cache=miss
unknown_identity: HTTP 401
collection_denied: HTTP 403
empty_bypass_first: HTTP 200 cache=bypass
empty_bypass_again: HTTP 200 cache=bypass
real_ttl_expired_miss: HTTP 200 cache=miss
tool_failure_first: HTTP 503
tool_failure_again: HTTP 503The driver starts actual Uvicorn children and uses HTTPX over loopback. Compare the first miss with the normalized hit, then inspect the other-principal miss, repeated incomplete bypasses and repeated controlled failures. Explain which key component or cacheability rule accounts for each observation.
The short deterministic retrieval runs synchronously inside the async endpoint, with SQLite created and closed on the same thread. Thread-safe cache bookkeeping does not establish service capacity, cluster safety or a latency improvement. The recorded observations make no performance, cost or answer-quality claim.
Inspect the complete saved result
{
"schema_version": 1,
"mode": "actual-loopback-http-materialized-result-cache",
"model_called": false,
"observations": [
{
"name": "first_miss",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "miss",
"cache_key": "bd1000c3a6c6c91ea652b2379cd45fe0a187e4d1320f1cabea1260eca2da15a9",
"result": {
"status": "completed",
"principal": "reader",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "json",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [
{
"passage_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"source_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"start_line": 1,
"end_line": 1,
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"byte_count": 199,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps"
}
],
"result_sha256": "e1deddbeb398d30a550944effb72e35689ebedc37add665f7e2623180c5d41f8"
}
}
},
{
"name": "normalized_hit",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "hit",
"cache_key": "bd1000c3a6c6c91ea652b2379cd45fe0a187e4d1320f1cabea1260eca2da15a9",
"result": {
"status": "completed",
"principal": "reader",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "json",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [
{
"passage_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"source_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"start_line": 1,
"end_line": 1,
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"byte_count": 199,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps"
}
],
"result_sha256": "e1deddbeb398d30a550944effb72e35689ebedc37add665f7e2623180c5d41f8"
}
}
},
{
"name": "other_principal_miss",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "miss",
"cache_key": "69b69d2247a2f27251b8221586af80a6124f05d911a0ad9115c9874a90bd703e",
"result": {
"status": "completed",
"principal": "editor",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "json",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [
{
"passage_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"source_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"start_line": 1,
"end_line": 1,
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"byte_count": 199,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps"
}
],
"result_sha256": "049cc1bfec474d1e7811f1e933b70ec85116b602e1247bd48a7e0bc72f8448f7"
}
}
},
{
"name": "unknown_identity",
"method": "POST",
"path": "/retrieve",
"status_code": 401,
"body": {
"error": {
"code": "unauthenticated"
}
}
},
{
"name": "collection_denied",
"method": "POST",
"path": "/retrieve",
"status_code": 403,
"body": {
"error": {
"code": "collection_denied"
}
}
},
{
"name": "empty_bypass_first",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "bypass",
"cache_key": "daa86ed848608568b83c3a2e9d476c50cac3982d3e58d0080d22048bbe5a32d5",
"result": {
"status": "incomplete",
"principal": "reader",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "zzzz",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [],
"result_sha256": "4282f48c5cf1218fe7610533e3713fff5e352ccfee1ad99ac3183d4280069e9c"
}
}
},
{
"name": "empty_bypass_again",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "bypass",
"cache_key": "daa86ed848608568b83c3a2e9d476c50cac3982d3e58d0080d22048bbe5a32d5",
"result": {
"status": "incomplete",
"principal": "reader",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "zzzz",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [],
"result_sha256": "4282f48c5cf1218fe7610533e3713fff5e352ccfee1ad99ac3183d4280069e9c"
}
}
},
{
"name": "real_ttl_expired_miss",
"method": "POST",
"path": "/retrieve",
"status_code": 200,
"body": {
"cache": "miss",
"cache_key": "bd1000c3a6c6c91ea652b2379cd45fe0a187e4d1320f1cabea1260eca2da15a9",
"result": {
"status": "completed",
"principal": "reader",
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"policy_id": "0b9b91312bdcd4070944dbba0616396aa12e4d05b8893fc2d3dffdfcf85f2933",
"request": {
"query": "json",
"collection": "serialization",
"version": "3.12",
"k": 2
},
"pipeline": {
"name": "literal-search-read",
"version": 1,
"max_passages": 10
},
"evidence": [
{
"passage_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"source_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"start_line": 1,
"end_line": 1,
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"byte_count": 199,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps"
}
],
"result_sha256": "e1deddbeb398d30a550944effb72e35689ebedc37add665f7e2623180c5d41f8"
}
}
},
{
"name": "tool_failure_first",
"method": "POST",
"path": "/retrieve",
"status_code": 503,
"body": {
"error": {
"code": "tool_failure"
}
}
},
{
"name": "tool_failure_again",
"method": "POST",
"path": "/retrieve",
"status_code": 503,
"body": {
"error": {
"code": "tool_failure"
}
}
}
]
}Key takeaways
A hit reuses a completed source payload under the same normalized request and authorization context. It does not skip current access checks, validate an answer or prove that the cache improves latency.
Best practices
Authorize before lookup, include mutable policy and corpus identity in the key, store independent materialized payloads, and keep incomplete or failed work out of the result cache. State the retention budget and process boundary explicitly.
Read the explanation
A retrieval cache reuses data under matching request and authorization context; it does not grant access.
What’s next
Continue with Trace RAG Requests.
References
The local identity and source-binding rules are implemented in Authorize Collection Searches. TTL, eviction, copy semantics and cacheability are explicit application choices verified by this lab’s tests and actual HTTP observations.