A service can start and return valid HTTP while still failing its application tasks. A release gate combines different checks before selecting a candidate: the expected files must be intact, the candidate must run, and its task report must meet the declared acceptance policy.
An artifact identity binds the candidate’s file manifest to an expected value retained by the controller. A smoke check exercises basic runtime behavior. Neither can turn a failed retrieval task into a pass; each check answers a different question.
When you need this. Use distinct release checks when a successful startup must not override a failed application task.
Follow one example. The local controller verifies the supplied candidate and starts that staged copy. Its HTTP smoke checks pass. It then reads the archived task report, whose scripted-policy gate fails. Promotion is refused and the existing artifact pointer stays unchanged. Follow that sequence to see why successful startup cannot override the task result.
You will build a local drill that selects an artifact and rolls it back. It is not a production deployment: the controller changes a temporary registry file and does not switch live traffic.
Refuse promotion when the task gate fails
- Verify the sealed artifact against its identity.
- Start and smoke-test the staged service.
- Evaluate the release gate after runtime checks.
- Recompute the pinned archived task policy.
- Refuse promotion and retain the old pointer.
Retain an identity outside the candidate
A bundle is a directory containing the explicit runtime file closure, pinned source snapshot, dependency lock and archived task report. A manifest lists each file’s SHA-256 digest. The release ID is a digest of that manifest, retained separately by the controller.
If the controller checked a candidate only against the candidate’s own manifest, someone could change the code and its listed hashes together, and the two would still agree. The controller therefore also requires the separately retained expected ID to match. Verification also checks the exact file set and refuses symbolic links. These hashes check consistency with trusted input; they do not authenticate an untrusted publisher.
Changed artifact. Editing code fails file verification. Replaced manifest. Replacing a listed hash also changes the release identity. Extra file. An undeclared file or symbolic link is refused rather than included implicitly.
Start the actual staged copy before considering promotion
The controller starts the sealed directory’s Uvicorn service on a temporary loopback socket. HTTPX checks readiness, a normal retrieval, a strict invalid request and the exact serialized response digest. The response also confirms that the service still runs without a model. After these checks the controller shuts down and reaps the child process; the record keeps its raw signal exit status. Verify the bundle again after the service stops, as smoke does: Python can write bytecode cache files next to the modules it imports.
The recorded runtime checks pass: true. That result covers the HTTP boundary and the supplied request. It does not show that every supported task succeeds.
The archived report is an exact copy of the preceding task evaluator’s actual run. The controller checks that the report names the bundle’s corpus snapshot, then recomputes the frozen scripted-policy release threshold with the copied checker. It does not run another evaluation, change target labels or generate an answer.
Let the failed task check stop the write
The archived scripted task gate returns FAIL. Promotion is refused with task_gate_not_accepted, and the existing artifact pointer stays byte-identical: true.
Failed task gate. Passing HTTP smoke cannot waive the recorded task misses. Wrong smoke binding. A result for another release ID, a failed check or an incomplete check set is refused before the pointer changes. Smoke records are trusted controller values, not authenticated remote attestations.
Accept only an explicit PASS: promote refuses every other decision, including the NOTCOMPARABLE result that task_gate returns when the report’s corpus does not match the bundle’s snapshot.
One unit test stubs a passing task gate to exercise promotion’s success branch. That stub is not an observed task-quality pass; the actual CLI run keeps the failed gate and the refused promotion.
Restore a previous artifact without claiming traffic rollback
An artifact pointer is a JSON record naming the current directory and its retained release ID, plus the previous entry. Before replacing it, the writer flushes a temporary file in the same directory and uses os.replace. This assumes one writer; it is not a distributed transaction or a guarantee against every power-loss case.
The rollback drill starts from a synthetic two-entry registry. It revalidates the prior bundle, then restores that entry. The recorded restoration check is true. The two seeded entries are test data, not earlier production deployments.
Changed rollback target. If the prior bundle changed, rollback refuses it and keeps the pointer intact. Failed replacement. A controlled replace error preserves the old pointer and removes the temporary candidate file. A real traffic rollback would also need routing, process health, compatible state and an authorized deployment controller.
Execute the local release drill
Download the complete local release lab. It contains the unchanged retrieval runtime, pinned packages, original summary corpus, archived task report, release controller and failure tests. Create a virtual environment and install requirements.lock before running the drill.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.lock"""Sealed local artifacts and runtime smoke checks, not a production deployer."""
from pathlib import Path
import hashlib
import importlib.util
import json
import os
import shutil
import tempfile
FILES = ('api.py','app.py','service.py','tools.py','refine.py','limits.py',
'bm25.py','version_documents.py','load_documents.py','loopback.py',
'serve.py','requirements.txt','requirements.lock','LICENSE',
'fixtures/snapshot.json','fixtures/development-cases.json',
'evaluator.py','compare.py','fixtures/task-evaluation.json')
QUESTION = 'Will write_text replace the contents of an existing file?'
def digest(data):
return hashlib.sha256(data).hexdigest()
def canonical(value):
return json.dumps(value,sort_keys=True,separators=(',',':'),ensure_ascii=True).encode()
def package(source, destination, label):
"""Copy the explicit runtime closure into a new immutable candidate directory."""
source, destination = Path(source), Path(destination)
if destination.exists():
raise ValueError('destination_exists')
if not isinstance(label,str) or not label or len(label)>64:
raise ValueError('invalid_label')
destination.mkdir(parents=True)
for name in FILES:
path=source/name
if path.is_symlink() or not path.is_file():
raise ValueError('invalid_source_file')
target=destination/name
target.parent.mkdir(parents=True,exist_ok=True)
shutil.copyfile(path,target)
(destination/'release-config.json').write_bytes(canonical({'schema':1,'label':label,'deployment':'local-staging-only'}))
manifest={'schema':1,'files':{name:digest((destination/name).read_bytes()) for name in (*FILES,'release-config.json')}}
release_id=digest(canonical(manifest))
(destination/'manifest.json').write_bytes(canonical(manifest))
verify(destination,release_id)
return release_id
def verify(directory, expected_id):
"""The caller retains expected_id separately from the candidate it checks."""
directory=Path(directory)
if directory.is_symlink() or any(p.is_symlink() for p in directory.rglob('*')):
raise ValueError('unexpected_bundle_entry')
manifest=json.loads((directory/'manifest.json').read_bytes())
if set(manifest)!= {'schema','files'} or type(manifest['schema']) is not int or manifest['schema']!=1:
raise ValueError('manifest_schema')
if digest(canonical(manifest))!=expected_id:
raise ValueError('release_identity_mismatch')
expected=set(FILES)|{'release-config.json'}
if set(manifest['files'])!=expected:
raise ValueError('file_closure_mismatch')
actual={str(p.relative_to(directory)) for p in directory.rglob('*') if p.is_file()}
if actual != expected|{'manifest.json'} or any(p.is_symlink() for p in directory.rglob('*')):
raise ValueError('unexpected_bundle_entry')
for name,sha in manifest['files'].items():
if digest((directory/name).read_bytes())!=sha:
raise ValueError('file_hash_mismatch')
return manifest
def smoke(directory, expected_id):
"""Start the actual staged service, send real HTTP, then reap its child."""
directory=Path(directory)
verify(directory,expected_id)
spec=importlib.util.spec_from_file_location('staged_loopback',directory/'loopback.py')
harness=importlib.util.module_from_spec(spec)
# Load the helper without writing __pycache__ inside the sealed candidate.
exec(compile((directory/'loopback.py').read_bytes(), str(directory/'loopback.py'), 'exec'), harness.__dict__)
# Prevent the Python child from adding bytecode to the sealed runtime closure.
previous=os.environ.get('PYTHONDONTWRITEBYTECODE')
os.environ['PYTHONDONTWRITEBYTECODE']='1'
try:
with harness.running_server() as (client,lifecycle):
health=client.get('/health')
valid=client.post('/retrieve',json={'question':QUESTION})
invalid=client.post('/retrieve',json={'question':QUESTION,'max_steps':True})
body=valid.json()
checks={'health':health.status_code==200,
'valid_request':valid.status_code==200 and body['status']=='completed',
'strict_refusal':invalid.status_code==422,
'wire_digest':digest(valid.content)==valid.headers.get('X-Retrieval-Body-SHA256'),
'no_model':body['model_called'] is False}
finally:
if previous is None:os.environ.pop('PYTHONDONTWRITEBYTECODE',None)
else:os.environ['PYTHONDONTWRITEBYTECODE']=previous
verify(directory,expected_id)
return {'release_id':expected_id,'checks':checks,'passed':all(checks.values()),
'scope':'local-runtime-smoke-only','lifecycle':lifecycle}
def atomic_state(path, value):
path=Path(path)
path.parent.mkdir(parents=True,exist_ok=True)
fd,name=tempfile.mkstemp(prefix='.candidate-',dir=path.parent)
try:
with os.fdopen(fd,'wb') as stream:
stream.write(canonical(value));stream.flush();os.fsync(stream.fileno())
os.replace(name,path)
finally:
if Path(name).exists():Path(name).unlink()
def task_gate(directory, expected_id):
"""Recheck the pinned preceding task report; do not run a model or rerank it."""
from evaluator import release_gate
directory=Path(directory)
verify(directory,expected_id)
raw=(directory/'fixtures/task-evaluation.json').read_bytes()
saved=json.loads(raw)
report=saved['normal']
snapshot=json.loads((directory/'fixtures/snapshot.json').read_bytes())
if report['identity']['corpus_id']!=snapshot['snapshot_id']:
return {'decision':'NOTCOMPARABLE','release_id':expected_id,'reason':'corpus_mismatch'}
result=release_gate(report,report['identity'],'scripted_refinement')
return {'decision':result['status'],'release_id':expected_id,'report_sha256':digest(raw),
'result':result,'scope':'archived-scripted-development-fixtures-only'}
def promote(pointer, directory, expected_id, smoke_record):
"""Move only a local artifact pointer; this does not switch running traffic."""
verify(directory,expected_id)
names={'health','valid_request','strict_refusal','wire_digest','no_model'}
checks=smoke_record.get('checks',{})
if (smoke_record.get('release_id')!=expected_id or smoke_record.get('passed') is not True
or set(checks)!=names or any(value is not True for value in checks.values())):
raise ValueError('smoke_not_accepted')
if task_gate(directory,expected_id)['decision']!='PASS':
raise ValueError('task_gate_not_accepted')
pointer=Path(pointer)
old=json.loads(pointer.read_bytes()) if pointer.exists() else None
state={'current':{'directory':str(Path(directory).resolve()),'release_id':expected_id},
'previous':old['current'] if old else None}
atomic_state(pointer,state)
return state
def rollback(pointer):
pointer=Path(pointer);state=json.loads(pointer.read_bytes());old=state.get('previous')
if not old:raise ValueError('no_previous_release')
verify(old['directory'],old['release_id'])
restored={'current':old,'previous':state['current']}
atomic_state(pointer,restored)
return restoredpython run.py --output scratch-result.json
python -m unittest -v test_releasestaged runtime smoke: True
archived scripted task gate: FAIL
promotion: task_gate_not_accepted pointer unchanged: True
seeded local rollback: True
changed artifact: file_hash_mismatchThe driver creates and removes temporary candidate directories. A successful exit means the expected refusal and rollback assertions held, not that the candidate was approved for release. The lab has no cloud resource, publisher identity service, production process manager or migration.
Inspect the complete saved result
This saved result predates the helper-loading fix described in the README; current tests also check that loading the helper leaves the sealed bundle unchanged.
{
"schema_version": 1,
"execution_mode": "actual-local-staging-http-and-artifact-registry",
"model_called": false,
"release_ids": {
"previous_fixture": "c1d499c3dbac6c76eb213339d5cd5c2109aa476df55d3420e655fae06ce0670c",
"candidate": "7375e5ceb066fdf6e88eee9d12b5bfada7023f2cd064ee741b5a518cfb272e74"
},
"runtime_smoke": {
"release_id": "7375e5ceb066fdf6e88eee9d12b5bfada7023f2cd064ee741b5a518cfb272e74",
"checks": {
"health": true,
"valid_request": true,
"strict_refusal": true,
"wire_digest": true,
"no_model": true
},
"passed": true,
"scope": "local-runtime-smoke-only",
"lifecycle": {
"command": [
"python3",
"serve.py",
"--fd",
"<inherited-loopback-socket>",
"--app",
"api:app"
],
"startup_observed": true,
"shutdown_exit_status": -15,
"stdout": "",
"stderr": "INFO: Started server process [58084]\nINFO: Waiting for application startup.\nINFO: Application startup complete.\nINFO: Shutting down\nINFO: Waiting for application shutdown.\nINFO: Application shutdown complete.\nINFO: Finished server process [58084]\n",
"shutdown_complete_observed": true,
"server_process_reaped": true
}
},
"task_gate": {
"decision": "FAIL",
"release_id": "7375e5ceb066fdf6e88eee9d12b5bfada7023f2cd064ee741b5a518cfb272e74",
"report_sha256": "f89e7459226cee8cb810b59a6dae61246f04d0ce805c988c38d9e66fe4a6fb39",
"result": {
"status": "FAIL",
"reason": "fixture_thresholds_not_met",
"thresholds": {
"minimum_supported_pass_fraction": 1.0,
"maximum_integrity_failures": 0,
"require_deterministic_repeats": true
},
"supported_trials": 6,
"supported_outcomes": {
"pass": 4,
"fail": 0,
"incomplete": 2
},
"unsupported_trials": 2,
"integrity_failures": 0,
"deterministic_repeats": true
},
"scope": "archived-scripted-development-fixtures-only"
},
"promotion": {
"refused": "task_gate_not_accepted",
"pointer_unchanged": true
},
"rollback": {
"setup": "synthetic two-entry artifact registry; not prior deployment",
"restored_previous_identity": true
},
"changed_artifact": {
"refused": "file_hash_mismatch"
},
"scope": "The staged service really ran over loopback HTTP. Promotion was blocked by archived task failures. Only a local artifact pointer is changed in the rollback drill; no running traffic, cloud release or answer-quality approval."
}Explain why a healthy candidate stayed inactive
Locate the passing runtime checks and the failed task gate. Explain why the pointer stayed unchanged despite a successful top-level CLI exit. Then change a previous bundle in the rollback test and identify which check prevents restoration. Distinguish restoring an artifact record from routing requests to a running process.
Read the explanation
Runtime health, task acceptance and artifact identity are separate checks.
The next capstone uses a different corpus and application contract, so read its release result separately from this refused Python-task promotion. This lesson’s failed cases stay in its saved results.
What’s next
Bring the pieces together in Build the developer knowledge assistant: a local browser UI and CLI with versioned retrieval, collection permissions and cited answers.
References
The official Python file-replacement documentation describes os.replace; the temporary-file documentation covers temporary-file creation and cleanup. hashlib supplies the file digests. The retained identity, explicit closure, task threshold and refusal behavior are application policies tested by this local drill.