A documentation version and a file revision answer different questions. “Python 3.12” names the API being described. A document revision identifies the particular text and metadata the application used. Editing a summary can change its revision while its API version stays the same.
A chunk ID identifies one searchable passage under a particular splitting configuration. A corpus snapshot binds the complete document and chunk records. Keeping these identities explicit prevents new text from silently inheriting old retrieval records.
When you need this. Use it when a changed document must produce a new corpus snapshot without confusing API versions with local file revisions.
Follow one example. The local text-change fixture adds a labelled teaching paragraph to a supplied summary. The edited source gets a new document revision and new chunk identities; unchanged sources keep theirs. A separate title-only change keeps the raw file bytes but still changes the revision because the title belongs to the recorded metadata.
You will compare those two changes with the existing local loader and the original summaries. The lab builds and validates snapshots. It does not regenerate embeddings or measure whether a new snapshot improves retrieval.
Make document and chunk revisions explicit
- Read the pinned document inputs.
- Derive the document revision.
- Bind paragraph spans and configuration.
- Retain versioned chunks in the snapshot.
Give each identity one job
A source ID names one logical document. Its documentation version names the API described by that document. Neither is a fingerprint of the file: editing a summary can leave the source ID and API version unchanged.
A document revision binds the logical source ID to its raw byte hash and metadata hash. The metadata hash covers the path, title, documentation version, URL and supplied fact status. A chunk ID hashes the document revision together with the paragraph text hash, line span and complete chunker configuration. The corpus snapshot ID hashes the full canonical document and chunk records, including normalized text, metadata and chunker configuration. The code sorts documents by source ID before it computes that hash, so manifest order does not change the snapshot ID.
Store retrieval records under the chunk ID, not the source ID or API version. In the text-change fixture, path-write keeps its source ID and version 3.12, but its document revision changes.
The supplied summaries are the same original Python API teaching text used in the preceding ingestion module; the lab does not fetch upstream documentation. The changed fixture adds a labelled local teaching paragraph, not an upstream Python documentation update. Its fact-status field records that local change. A separate fixture changes only a title.
Keep byte identity separate from normalized text
The loader checks pinned file bytes before decoding them. It then normalizes CRLF line endings to LF. The raw byte hash still identifies the original file, and paragraph spans point into the normalized string. This differs from the preceding source-span exercise, which preserved original line endings.
The paragraph chunker groups nonblank lines separated by blank lines. It is a small deterministic policy for these fixtures, not a general chunking recommendation. A blank-line boundary in the changed summary creates another chunk. The vector retrieval course remains the prerequisite for embedding and retrieval internals.
Unpinned edit. Changing a source while keeping its old manifest hash raises IngestionError with hash_mismatch. Review the new text, then pin its hash in the manifest before you compare it. A hash computed after the edit identifies the new bytes; it cannot show that they are accurate.
Unsupported chunker. The builder accepts its one implemented configuration. Passing another configuration raises IngestionError with unsupported_chunker; changing a label does not implement another splitting algorithm.
Compare text changes with metadata changes
The text-change fixture creates new revision and chunk IDs for the edited source. Unchanged sources retain their identifiers. The metadata-only fixture keeps the same raw byte hash but still changes the document revision and chunk IDs, because the title is part of the hashed metadata.
Altered saved snapshot. The separate validate_snapshot function rebuilds metadata revisions and paragraph chunks from the normalized saved text and checks the complete canonical identity. A changed field with an unchanged snapshot ID raises IngestionError with invalid_snapshot. It catches inconsistent saved artifacts. It cannot recover the original CRLF bytes from normalized text, and it does not authenticate the source author. Call validate_snapshot on a saved snapshot before you write a checkpoint or replace an index with it.
This rule is deliberately conservative: an edit anywhere in a document gives all its chunks a new document revision. It gives up chunk reuse so that one document’s chunks never mix two revisions. The lab does not keep chunk IDs stable across document edits or synchronize an index.
A changed snapshot tells a later ingestion or update step that work is needed. The lab writes local JSON files and an HTML report. It does not write to a database, remove old chunks, rewrite held-out eval labels or show that a passage is relevant to a question.
Execute the local corpus operation
Download the complete lab. The implementation, fixtures, tests, README and raw execution record travel together.
"""Version pinned documentation bytes, metadata, paragraphs and corpus snapshots."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
from load_documents import IngestionError, load_documents
CHUNKER = {'name': 'paragraphs', 'version': 1, 'newline_policy': 'CRLF-to-LF', 'separator': 'blank-line'}
METADATA_KEYS = ('path', 'title', 'version', 'url', 'fact_status')
def canonical_hash(value: object) -> str:
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':'), ensure_ascii=False).encode('utf-8')).hexdigest()
def paragraphs(text: str):
"""Yield exact normalized-text slices with 1-based inclusive line spans."""
lines = text.splitlines(keepends=True)
start = None
for index, line in enumerate(lines):
if line.strip():
if start is None:
start = index
elif start is not None:
yield start + 1, index, ''.join(lines[start:index])
start = None
if start is not None:
yield start + 1, len(lines), ''.join(lines[start:])
def version_document(document: dict, config: dict) -> dict:
metadata = {key: document[key] for key in METADATA_KEYS if key in document}
metadata_sha256 = canonical_hash(metadata)
revision = canonical_hash({'source_id': document['source_id'], 'raw_sha256': document['sha256'], 'metadata_sha256': metadata_sha256})
chunks = []
for start, end, text in paragraphs(document['text']):
text_hash = hashlib.sha256(text.encode('utf-8')).hexdigest()
identity = {'document_revision': revision, 'chunker': config, 'start_line': start, 'end_line': end, 'text_sha256': text_hash}
chunks.append({'chunk_id': canonical_hash(identity), 'source_id': document['source_id'], 'document_revision': revision,
'start_line': start, 'end_line': end, 'text': text, 'text_sha256': text_hash})
return {**document, 'metadata_sha256': metadata_sha256, 'document_revision': revision, 'chunks': chunks}
def snapshot_identity(documents: list, config: dict) -> str:
# Bind normalized text, metadata and every chunk too, not only claimed revisions.
return canonical_hash({'schema_version': 1, 'chunker': config, 'documents': documents})
def build_snapshot(manifest: Path, chunker: dict | None = None) -> dict:
config = dict(CHUNKER if chunker is None else chunker)
if canonical_hash(config) != canonical_hash(CHUNKER):
raise IngestionError('unsupported_chunker')
loaded = load_documents(manifest)['documents']
documents = [version_document(document, config) for document in sorted(loaded, key=lambda d: d['source_id'])]
return {'schema_version': 1, 'execution_mode': 'offline-local-files', 'chunker': config,
'snapshot_id': snapshot_identity(documents, config), 'documents': documents}
def validate_snapshot(snapshot: dict) -> dict:
"""Reject inconsistent saved artifacts; this does not authenticate raw files.
Original byte hashes were checked while loading files. Normalized saved text
cannot reconstruct original CRLF bytes, so this validator checks its canonical
identity and internal derivations, not the original files or their provenance.
"""
try:
if not isinstance(snapshot, dict) or type(snapshot.get('schema_version')) is not int or snapshot['schema_version'] != 1:
raise ValueError()
if snapshot.get('execution_mode') != 'offline-local-files' or canonical_hash(snapshot.get('chunker')) != canonical_hash(CHUNKER):
raise ValueError()
documents = snapshot['documents']
if not isinstance(documents, list) or not documents:
raise ValueError()
seen_ids, seen_paths = set(), set()
allowed = ('source_id', 'path', 'title', 'sha256', 'version', 'url', 'fact_status', 'text')
for document in documents:
if not isinstance(document, dict):
raise ValueError()
original = {key: document[key] for key in allowed if key in document}
if any(not isinstance(original.get(key), str) or not original[key] for key in ('source_id','path','title','sha256','text')):
raise ValueError()
if any(key in original and (not isinstance(original[key],str) or not original[key]) for key in ('version','url','fact_status')):
raise ValueError()
if len(original['sha256']) != 64 or any(c not in '0123456789abcdef' for c in original['sha256']):
raise ValueError()
if len(original['text'].encode('utf-8')) > 65536 or not original['text'].strip() or '\x00' in original['text'] or '\r\n' in original['text']:
raise ValueError()
if original['source_id'] in seen_ids or original['path'] in seen_paths:
raise ValueError()
seen_ids.add(original['source_id']); seen_paths.add(original['path'])
if canonical_hash(document) != canonical_hash(version_document(original, CHUNKER)):
raise ValueError()
if documents != sorted(documents, key=lambda d:d['source_id']):
raise ValueError()
if snapshot.get('snapshot_id') != snapshot_identity(documents, CHUNKER):
raise ValueError()
except (KeyError, TypeError, ValueError, UnicodeError, OverflowError) as error:
raise IngestionError('invalid_snapshot') from error
return snapshot
def compare(before: dict, after: dict) -> dict:
previous = {d['source_id']: d for d in before['documents']}
current = {d['source_id']: d for d in after['documents']}
return {'snapshot_changed': before['snapshot_id'] != after['snapshot_id'],
'added': sorted(current.keys() - previous.keys()), 'removed': sorted(previous.keys() - current.keys()),
'changed': [{'source_id': key, 'raw_bytes_changed': previous[key]['sha256'] != current[key]['sha256'],
'metadata_changed': previous[key]['metadata_sha256'] != current[key]['metadata_sha256'],
'chunk_ids_changed': [c['chunk_id'] for c in previous[key]['chunks']] != [c['chunk_id'] for c in current[key]['chunks']]}
for key in sorted(previous.keys() & current.keys()) if previous[key]['document_revision'] != current[key]['document_revision']]}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--manifest', type=Path, default=Path(__file__).parent / 'fixtures/initial/source-manifest.json')
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
result = build_snapshot(args.manifest)
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(json.dumps(result, indent=2) + '\n', encoding='utf-8')
print(json.dumps({'snapshot_id': result['snapshot_id'], 'documents': len(result['documents']), 'chunks': sum(len(d['chunks']) for d in result['documents'])}, sort_keys=True))
if __name__ == '__main__': main()python3 version_documents.py --output /tmp/corpus-snapshot.json
python3 demo.py --output /tmp/corpus-version-comparison
python3 -m unittest discover -v{
"comparisons": {
"metadata-change": {
"added": [],
"changed": [
{
"chunk_ids_changed": true,
"metadata_changed": true,
"raw_bytes_changed": false,
"source_id": "path-write"
}
],
"removed": [],
"snapshot_changed": true
},
"text-change": {
"added": [],
"changed": [
{
"chunk_ids_changed": true,
"metadata_changed": true,
"raw_bytes_changed": true,
"source_id": "path-write"
}
],
"removed": [],
"snapshot_changed": true
}
},
"execution_mode": "offline-local-files"
}This output is from a saved local run (offline-local-files); it needs no network or API key. Compare the snapshot IDs, document revisions and chunk IDs in the complete saved result.
Inspect the complete saved result
{
"execution_mode": "offline-local-files",
"scope": "deliberate local fixture changes; no upstream refresh or model execution",
"snapshots": {
"initial": {
"schema_version": 1,
"execution_mode": "offline-local-files",
"chunker": {
"name": "paragraphs",
"version": 1,
"newline_policy": "CRLF-to-LF",
"separator": "blank-line"
},
"snapshot_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"documents": [
{
"source_id": "json-text",
"path": "docs/json-text.md",
"title": "json-text",
"sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"fact_status": "original summary; primary docs checked",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"metadata_sha256": "fa80a47c53e04f67d9481651bd0b6fae72e8094b2df37fc9a375cabfd930bc10",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"chunks": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"start_line": 1,
"end_line": 1,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288"
}
]
},
{
"source_id": "path-read",
"path": "docs/path-read.md",
"title": "path-read",
"sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "original summary; primary docs checked",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"metadata_sha256": "1f1afc1a151fe961715da50dad4c18a19fa67b1f94117123153731c63dd9a1a8",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"chunks": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"start_line": 1,
"end_line": 1,
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0"
}
]
},
{
"source_id": "path-read-3-13",
"path": "docs/path-read-3-13.md",
"title": "path-read-3-13",
"sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f",
"version": "3.13",
"url": "https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "documented only; Python 3.13 not executed",
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"metadata_sha256": "a757e6201998dfaf780a87d8544ee4079bb5f1e5ab97ad035a77cd3afdbf0d01",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"chunks": [
{
"chunk_id": "bdaeff5ae3ab88b70b12b9b6d4a0a46de45ea14520b971a7756cb03bbd384f35",
"source_id": "path-read-3-13",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"start_line": 1,
"end_line": 1,
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"text_sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f"
}
]
},
{
"source_id": "path-write",
"path": "docs/path-write.md",
"title": "path-write",
"sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"fact_status": "original summary; primary docs checked",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"metadata_sha256": "b31d26389cf0da944c0817a580f736d8009f2b297b45ec631849c0e219818768",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"chunks": [
{
"chunk_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"start_line": 1,
"end_line": 1,
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e"
}
]
}
]
},
"text-change": {
"schema_version": 1,
"execution_mode": "offline-local-files",
"chunker": {
"name": "paragraphs",
"version": 1,
"newline_policy": "CRLF-to-LF",
"separator": "blank-line"
},
"snapshot_id": "d1ee6e652c1c6d08e6e4e10314f53153d6e0b5e5850739fd135f455196d4ec0d",
"documents": [
{
"source_id": "json-text",
"path": "docs/json-text.md",
"title": "json-text",
"sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"fact_status": "original summary; primary docs checked",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"metadata_sha256": "fa80a47c53e04f67d9481651bd0b6fae72e8094b2df37fc9a375cabfd930bc10",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"chunks": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"start_line": 1,
"end_line": 1,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288"
}
]
},
{
"source_id": "path-read",
"path": "docs/path-read.md",
"title": "path-read",
"sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "original summary; primary docs checked",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"metadata_sha256": "1f1afc1a151fe961715da50dad4c18a19fa67b1f94117123153731c63dd9a1a8",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"chunks": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"start_line": 1,
"end_line": 1,
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0"
}
]
},
{
"source_id": "path-read-3-13",
"path": "docs/path-read-3-13.md",
"title": "path-read-3-13",
"sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f",
"version": "3.13",
"url": "https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "documented only; Python 3.13 not executed",
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"metadata_sha256": "a757e6201998dfaf780a87d8544ee4079bb5f1e5ab97ad035a77cd3afdbf0d01",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"chunks": [
{
"chunk_id": "bdaeff5ae3ab88b70b12b9b6d4a0a46de45ea14520b971a7756cb03bbd384f35",
"source_id": "path-read-3-13",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"start_line": 1,
"end_line": 1,
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"text_sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f"
}
]
},
{
"source_id": "path-write",
"path": "docs/path-write.md",
"title": "path-write",
"sha256": "fc153a13320bf10c3bd95073064be8ca79fc49d1baf94020aab6e1f9098bcb03",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"fact_status": "deliberate local teaching revision; not an upstream documentation update",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.\n\nLocal teaching revision: callers must review overwrite behavior before saving.\n",
"metadata_sha256": "a26519682f3f5ec1043876ce74b1eafe0eff98421e486c9dd1a2c949bb19f0ad",
"document_revision": "a036f69e0484af8c5ddd938bb292a0cb63ec2f661379d871d3901a03dc987dd2",
"chunks": [
{
"chunk_id": "d27f0074d2d1a0c980e8bf541c26a83cf970e2f71e2c7f9a7d3da5442214b66b",
"source_id": "path-write",
"document_revision": "a036f69e0484af8c5ddd938bb292a0cb63ec2f661379d871d3901a03dc987dd2",
"start_line": 1,
"end_line": 1,
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.\n",
"text_sha256": "415990e67efb5d9b048c1af2ced2984f364e21bed39c04f58ed2d0b147e7e97e"
},
{
"chunk_id": "99e7deb4a60f1d2a3e3e8f33f3283b654e2ea80568aa6e2aa819d4666714c42f",
"source_id": "path-write",
"document_revision": "a036f69e0484af8c5ddd938bb292a0cb63ec2f661379d871d3901a03dc987dd2",
"start_line": 3,
"end_line": 3,
"text": "Local teaching revision: callers must review overwrite behavior before saving.\n",
"text_sha256": "f3540576a85312166e10df3c002f98b9b6503b5e88248e1bbe373485e4851ae0"
}
]
}
]
},
"metadata-change": {
"schema_version": 1,
"execution_mode": "offline-local-files",
"chunker": {
"name": "paragraphs",
"version": 1,
"newline_policy": "CRLF-to-LF",
"separator": "blank-line"
},
"snapshot_id": "970f70ddaeb7239d1c44a716174361be7b0cba4a552d011f830da98ed9ec95b5",
"documents": [
{
"source_id": "json-text",
"path": "docs/json-text.md",
"title": "json-text",
"sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/json.html#json.dumps",
"fact_status": "original summary; primary docs checked",
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"metadata_sha256": "fa80a47c53e04f67d9481651bd0b6fae72e8094b2df37fc9a375cabfd930bc10",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"chunks": [
{
"chunk_id": "e5455737b11664449f653280d89dfc2faaa9a14de16e9c3eb24296ca2ceea51c",
"source_id": "json-text",
"document_revision": "0c4b565b2b1900787d6da9f8539c471f8cb2245c00abf887a57d460364abc57a",
"start_line": 1,
"end_line": 1,
"text": "json.dumps(obj, ensure_ascii=True) returns a JSON string. Setting ensure_ascii=False leaves non-ASCII characters in that string. It does not write a file or select the encoding used by a file writer.",
"text_sha256": "0f5efa9669a97f93d5e7e8fe91de80a0d954643947ac9619dba41d3904166288"
}
]
},
{
"source_id": "path-read",
"path": "docs/path-read.md",
"title": "path-read",
"sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "original summary; primary docs checked",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"metadata_sha256": "1f1afc1a151fe961715da50dad4c18a19fa67b1f94117123153731c63dd9a1a8",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"chunks": [
{
"chunk_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"start_line": 1,
"end_line": 1,
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0"
}
]
},
{
"source_id": "path-read-3-13",
"path": "docs/path-read-3-13.md",
"title": "path-read-3-13",
"sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f",
"version": "3.13",
"url": "https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.read_text",
"fact_status": "documented only; Python 3.13 not executed",
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"metadata_sha256": "a757e6201998dfaf780a87d8544ee4079bb5f1e5ab97ad035a77cd3afdbf0d01",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"chunks": [
{
"chunk_id": "bdaeff5ae3ab88b70b12b9b6d4a0a46de45ea14520b971a7756cb03bbd384f35",
"source_id": "path-read-3-13",
"document_revision": "d68df3e7ea1e24af3f40366a61402fc80558fefe2ef5f3eda928de0e01820db9",
"start_line": 1,
"end_line": 1,
"text": "In Python 3.13, Path.read_text accepts encoding, errors and newline. The newline parameter was added in Python 3.13. The method returns decoded file text and closes the file.",
"text_sha256": "0dd5becb0c04458ad5f8bbf3fdef4f71034c584ef863bf748793bef7309b9f4f"
}
]
},
{
"source_id": "path-write",
"path": "docs/path-write.md",
"title": "Write UTF-8 text with pathlib",
"sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"version": "3.12",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"fact_status": "original summary; primary docs checked",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"metadata_sha256": "fa7085e5dcd23a3cc6a63261a19b1601c12750c05110a900e3251b59c4585572",
"document_revision": "db8807f30a17947915f67ea5208f5768a4f75c2b2233ad8ff0776327a96e3706",
"chunks": [
{
"chunk_id": "12f06d42c15746b564ca36a775a7bd5b88d6376a79b47473d360cb53cc60c815",
"source_id": "path-write",
"document_revision": "db8807f30a17947915f67ea5208f5768a4f75c2b2233ad8ff0776327a96e3706",
"start_line": 1,
"end_line": 1,
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e"
}
]
}
]
}
},
"comparisons": {
"text-change": {
"snapshot_changed": true,
"added": [],
"removed": [],
"changed": [
{
"source_id": "path-write",
"raw_bytes_changed": true,
"metadata_changed": true,
"chunk_ids_changed": true
}
]
},
"metadata-change": {
"snapshot_changed": true,
"added": [],
"removed": [],
"changed": [
{
"source_id": "path-write",
"raw_bytes_changed": false,
"metadata_changed": true,
"chunk_ids_changed": true
}
]
}
}
}Compare the state before and after the operation
Make a scratch copy of the fixture. Change only its title and compare the snapshots. The raw byte hash should remain the same, while the document revision and chunk IDs change. Then edit its text without changing the manifest hash, and check that the build fails with hash_mismatch. Do not modify the published baseline or silently update eval targets.
Read the explanation
Check the specific stored state and input contract; a completed command alone does not prove corpus consistency.
What’s next
Continue with Resume document ingestion.
References
APIs were checked on 23 October 2026 against official Python hashlib, Python sqlite3, SQLite transactions and SQLite FTS5 documentation as applicable. The source summaries retain their version-specific Python documentation URLs.