Build a hybrid retriever that runs native SQLite BM25 and the actual LangChain vector store on the same question. Reciprocal rank fusion combines their ranked results. The vector branch reuses the original saved Qwen embeddings; this run makes no new model request.
When you need this. Use it when lexical and vector retrieval return separate candidate lists and you need an inspectable rule for combining their order without treating their scores as the same unit.
Prerequisites. Complete the preceding retrieval module. Its vector retriever supplies the saved-vector adapter, and its filter lesson supplies the version constraint. Use Python 3.12 or later with the already prepared pinned environment.
Send the same question through both retrievers
The lexical branch now accepts the full question rather than a separately selected term. The application extracts lowercase ASCII alphanumeric tokens, removes duplicates, quotes each token and joins them with explicit OR. SQLite still indexes the passage text with its native unicode61 tokenizer. These are separate tokenization rules; the application parser is intentionally narrower and has no stemming, synonym expansion or stop-word removal.
The raw question goes unchanged to the saved-vector adapter. Lexical normalization never substitutes a different question embedding. Both branches apply the requested version equality policy before selecting candidates, and keep their native ordering: ascending SQLite BM25 and descending cosine similarity.
Unrecorded question. A question outside the development recording raises ValueError in the vector adapter. Lexical matches cannot turn an unrecorded query into a supported hybrid replay. Empty text, a query without usable terms and inputs beyond the documented bounds are also refused.
Fuse ranks instead of adding scores
Reciprocal rank fusion, or RRF, assigns each returned passage a contribution of 1 / (rank_constant + rank) from each branch that contains it. Ranks start at one. An absent branch contributes zero. The implementation sums the contributions with math.fsum, orders by descending fused score, then uses chunk ID to break a tie.
BM25 and cosine remain diagnostic fields. They do not enter that addition. The configured rank constant is 60; this fixed choice makes the calculation reproducible and is not a tuned quality result for this corpus.
The first recorded development question produces the following fused trace. Each row retains the native branch rank and its own contribution, so the combined result can be checked without guessing a score conversion.
| Source | Lexical rank | Vector rank | Lexical contribution | Vector contribution | Fused score |
|---|---|---|---|---|---|
| path-read | 1 | 1 | 0.01639344262295082 | 0.01639344262295082 | 0.03278688524590164 |
| path-write | 2 | 2 | 0.016129032258064516 | 0.016129032258064516 | 0.03225806451612903 |
Keep the candidate union consistent
The per-branch candidate budget is 3, while the final limit is 2. A passage missing from both truncated lists cannot appear in their fused union. More fusion logic cannot repair that candidate-generation miss.
Conflicting payload. If the same chunk ID carries different text or source metadata across the two lists, fusion raises ValueError instead of choosing one payload. A repeated ID within a branch is also refused; it must not receive extra rank contributions. Nonfinite native scores are rejected before writing a trace.
The UTF-8 writing question still ranks a reading passage before the writing passage. The unsupported backup question also produces candidates. Agreement between retrieval branches is not an answer-support check.
Execute the local hybrid contract
Download the complete lab, including source, tests, exact dependency pins, fixtures and recorded results. The CLI writes a JSON result; it does not serve an HTTP endpoint.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.lock"""Actual SQLite lexical + LangChain saved-vector retrieval, fused by rank."""
import math
import re
import sqlite3
from vector_retriever import VectorRetriever, rows_from_snapshot, version_filter
PAYLOAD_KEYS=('chunk_id','source_id','version','url','text')
def positive_int(value, name):
if type(value) is not int or not 1<=value<=100:
raise ValueError('invalid_'+name)
def question_terms(query):
if not isinstance(query,str) or not query.strip() or len(query)>512:
raise ValueError('invalid_query')
tokens=re.findall(r'[a-z0-9]+',query.lower())
if not tokens or len(tokens)>32 or any(len(token)>64 for token in tokens):
raise ValueError('invalid_query')
return list(dict.fromkeys(tokens))
class QuestionLexicalRetriever:
def __init__(self,snapshot):
self.rows=rows_from_snapshot(snapshot)
self.db=sqlite3.connect(':memory:')
try:
self.db.execute('CREATE VIRTUAL TABLE passages USING fts5(chunk_id UNINDEXED,source_id UNINDEXED,version UNINDEXED,url UNINDEXED,text, tokenize="unicode61")')
self.db.executemany('INSERT INTO passages VALUES(?,?,?,?,?)',[(r['chunk_id'],r['source_id'],r['version'],r['url'],r['text']) for r in self.rows])
except Exception:
self.db.close();raise
def search(self,query,k=3,policy=None):
positive_int(k,'candidate_budget')
version=version_filter(policy)
terms=question_terms(query)
expression=' OR '.join('"'+term+'"' for term in terms)
sql='SELECT chunk_id,source_id,version,url,text,bm25(passages) FROM passages WHERE passages MATCH ?'
params=[expression]
if version is not None:
sql+=' AND version=?';params.append(version)
sql+=' ORDER BY bm25(passages),chunk_id LIMIT ?';params.append(k)
return [dict(zip((*PAYLOAD_KEYS,'native_score'),row)) for row in self.db.execute(sql,params)]
def close(self):self.db.close()
def reciprocal_rank_fusion(lexical,vector,k=2,rank_constant=60):
positive_int(k,'k')
if type(rank_constant) is not int or rank_constant<=0 or rank_constant>1000000:
raise ValueError('invalid_rank_constant')
merged={}
for branch,rows,score_key in [('lexical',lexical,'native_score'),('vector',vector,'score')]:
if not isinstance(rows,list) or len(rows)>100:raise ValueError('invalid_candidates')
seen=set()
for rank,row in enumerate(rows,1):
if not isinstance(row,dict) or any(not isinstance(row.get(key),str) or not row[key] for key in PAYLOAD_KEYS):
raise ValueError('invalid_candidate_payload')
key=row['chunk_id']
if key in seen:raise ValueError('duplicate_branch_chunk_id')
seen.add(key)
score=row.get(score_key)
try:
valid_score=type(score) in (int,float) and math.isfinite(score)
except OverflowError:
valid_score=False
if not valid_score:raise ValueError('invalid_native_score')
payload={key:row[key] for key in PAYLOAD_KEYS}
if key not in merged:
merged[key]={**payload,'lexical_score':None,'vector_score':None,'lexical_rank':None,'vector_rank':None,'lexical_contribution':0.0,'vector_contribution':0.0,'fused_score':0.0}
elif any(merged[key][field]!=payload[field] for field in PAYLOAD_KEYS):
raise ValueError('inconsistent_chunk_payload')
merged[key][branch+'_score']=score
merged[key][branch+'_rank']=rank
merged[key][branch+'_contribution']=1.0/(rank_constant+rank)
for row in merged.values():
row['fused_score']=math.fsum([row['lexical_contribution'],row['vector_contribution']])
return sorted(merged.values(),key=lambda row:(-row['fused_score'],row['chunk_id']))[:k]
class HybridRetriever:
def __init__(self,snapshot,embeddings):
self.vector=VectorRetriever(snapshot,embeddings)
self.lexical=QuestionLexicalRetriever(snapshot)
def close(self):self.lexical.close()
def search(self,query,k=2,candidate_budget=3,rank_constant=60,policy=None):
positive_int(k,'k');positive_int(candidate_budget,'candidate_budget')
if candidate_budget<k:raise ValueError('candidate_budget_below_k')
if type(rank_constant) is not int or not 1<=rank_constant<=1000000:raise ValueError('invalid_rank_constant')
terms=question_terms(query);version_filter(policy)
lexical=self.lexical.search(query,k=candidate_budget,policy=policy)
# Original full question: no substitution with the lexical tokens.
vector=self.vector.search(query,k=candidate_budget,policy=policy)
return {'query':query,'lexical_query_terms':terms,'lexical':lexical,'vector':vector,
'results':reciprocal_rank_fusion(lexical,vector,k,rank_constant)}python run.py --output scratch-result.json
python -m unittest discover -v{
"development_cases": [
{
"case_id": "utf8-write",
"fused_order": [
"path-read",
"path-write"
],
"lexical_order": [
"path-read",
"path-write",
"json-text"
],
"vector_order": [
"path-read",
"path-write",
"json-text"
]
},
{
"case_id": "replace-file",
"fused_order": [
"path-write",
"path-read"
],
"lexical_order": [
"path-write",
"path-read",
"json-text"
],
"vector_order": [
"path-write",
"path-read",
"json-text"
]
},
{
"case_id": "json-result-type",
"fused_order": [
"json-text",
"path-read"
],
"lexical_order": [
"json-text",
"path-read",
"path-write"
],
"vector_order": [
"json-text",
"path-read",
"path-write"
]
},
{
"case_id": "backup-schedule",
"fused_order": [
"json-text",
"path-read"
],
"lexical_order": [
"json-text"
],
"vector_order": [
"path-read",
"path-write",
"json-text"
]
}
],
"execution_mode": "offline-saved-vector-replay-native-sqlite-langchain-rrf"
}The implementation builds local in-memory stores and scores this small fixture. Its vector adapter scores all eligible passages to make tie behavior explicit. A larger index needs separate capacity measurements and an appropriate retrieval backend; this recording contains no throughput or latency benchmark.
Inspect the complete saved result
{
"execution_mode": "offline-saved-vector-replay-native-sqlite-langchain-rrf",
"model_called": false,
"snapshot_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"sqlite_version": "3.45.3",
"configuration": {
"k": 2,
"candidate_budget_per_branch": 3,
"rank_constant": 60
},
"development_cases": [
{
"case_id": "utf8-write",
"case_class": "answerable",
"policy": {
"version": {
"$eq": "3.12"
}
},
"query": "How do I explicitly select UTF-8 when saving text with pathlib?",
"lexical_query_terms": [
"how",
"do",
"i",
"explicitly",
"select",
"utf",
"8",
"when",
"saving",
"text",
"with",
"pathlib"
],
"lexical": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"native_score": -0.8901430296001853
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"native_score": -4.904418516387322e-06
},
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"native_score": -9.765013054830289e-07
}
],
"vector": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"score": 0.637539995393079
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"score": 0.5762678845903342
},
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"score": 0.4948350231504922
}
],
"results": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"lexical_score": -0.8901430296001853,
"vector_score": 0.637539995393079,
"lexical_rank": 1,
"vector_rank": 1,
"lexical_contribution": 0.01639344262295082,
"vector_contribution": 0.01639344262295082,
"fused_score": 0.03278688524590164
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"lexical_score": -4.904418516387322e-06,
"vector_score": 0.5762678845903342,
"lexical_rank": 2,
"vector_rank": 2,
"lexical_contribution": 0.016129032258064516,
"vector_contribution": 0.016129032258064516,
"fused_score": 0.03225806451612903
}
]
},
{
"case_id": "replace-file",
"case_class": "answerable",
"policy": {
"version": {
"$eq": "3.12"
}
},
"query": "Will write_text replace the contents of an existing file?",
"lexical_query_terms": [
"will",
"write",
"text",
"replace",
"the",
"contents",
"of",
"an",
"existing",
"file"
],
"lexical": [
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"native_score": -2.3976548397386743
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"native_score": -4.084683995209214e-06
},
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"native_score": -3.3056246724488786e-06
}
],
"vector": [
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"score": 0.6357498076122466
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"score": 0.5547892797395033
},
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"score": 0.36746132784169144
}
],
"results": [
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"lexical_score": -2.3976548397386743,
"vector_score": 0.6357498076122466,
"lexical_rank": 1,
"vector_rank": 1,
"lexical_contribution": 0.01639344262295082,
"vector_contribution": 0.01639344262295082,
"fused_score": 0.03278688524590164
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"lexical_score": -4.084683995209214e-06,
"vector_score": 0.5547892797395033,
"lexical_rank": 2,
"vector_rank": 2,
"lexical_contribution": 0.016129032258064516,
"vector_contribution": 0.016129032258064516,
"fused_score": 0.03225806451612903
}
]
},
{
"case_id": "json-result-type",
"case_class": "answerable",
"policy": {
"version": {
"$eq": "3.12"
}
},
"query": "Does json.dumps produce text or a bytes object?",
"lexical_query_terms": [
"does",
"json",
"dumps",
"produce",
"text",
"or",
"a",
"bytes",
"object"
],
"lexical": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"native_score": -3.628237730877387
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"native_score": -3.0341221974564053e-06
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"native_score": -2.263641817358195e-06
}
],
"vector": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"score": 0.652333570145617
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"score": 0.42563890525675496
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"score": 0.4145266500154773
}
],
"results": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"lexical_score": -3.628237730877387,
"vector_score": 0.652333570145617,
"lexical_rank": 1,
"vector_rank": 1,
"lexical_contribution": 0.01639344262295082,
"vector_contribution": 0.01639344262295082,
"fused_score": 0.03278688524590164
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"lexical_score": -3.0341221974564053e-06,
"vector_score": 0.42563890525675496,
"lexical_rank": 2,
"vector_rank": 2,
"lexical_contribution": 0.016129032258064516,
"vector_contribution": 0.016129032258064516,
"fused_score": 0.03225806451612903
}
]
},
{
"case_id": "backup-schedule",
"case_class": "unsupported",
"policy": {
"version": {
"$eq": "3.12"
}
},
"query": "How frequently does our cloud provider back up these files?",
"lexical_query_terms": [
"how",
"frequently",
"does",
"our",
"cloud",
"provider",
"back",
"up",
"these",
"files"
],
"lexical": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"native_score": -0.8273874668010814
}
],
"vector": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"score": 0.3202785226491292
},
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"score": 0.26563513588901855
},
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"score": 0.2488070454064458
}
],
"results": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"lexical_score": -0.8273874668010814,
"vector_score": 0.2488070454064458,
"lexical_rank": 1,
"vector_rank": 3,
"lexical_contribution": 0.01639344262295082,
"vector_contribution": 0.015873015873015872,
"fused_score": 0.032266458495966696
},
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"lexical_score": null,
"vector_score": 0.3202785226491292,
"lexical_rank": null,
"vector_rank": 1,
"lexical_contribution": 0.0,
"vector_contribution": 0.01639344262295082,
"fused_score": 0.01639344262295082
}
]
}
],
"interpretation": "Ranking only. Retrieval does not establish answer correctness or support; no held-out queries executed."
}The CLI prints lexical, vector and fused source order for all four development questions. It uses saved vectors while computing retrieval and reciprocal-rank fusion locally.
Compare the recorded behavior
Inspect the unsupported case in the raw result. Find a candidate present in only one branch and verify that its other contribution is zero. Then compare the UTF-8 writing candidates with the question: explain why agreement at the first rank still leaves the wrong source ahead.
Read the explanation
Fusion changes ordering within the candidate union; it does not make the passages support an answer.
What’s next
Continue with Rerank Retrieved Passages.
References
The original reciprocal rank fusion paper, section 1, supplies the rank-based formula. The official SQLite FTS5 documentation supplies quoted-string, Boolean-OR and native BM25 behavior. These passages were checked on 23 October 2026. The exact installed LangChain Core 1.7.0 adapter behavior and preserved-vector provenance remain inspectable in the lab; no new embedding inference occurs here.