A first search can return a valid passage that lacks the detail needed for the task. Build a local controller that reads the candidate, applies a declared rule and either stops or issues a different query. Its trace makes that decision inspectable.
When you need this. Use an explicit retrieval loop when the next action should depend on what was read, and you need to understand why another search happened.
This exercise executes a handwritten policy over real local search/read tools. The policy contains development-specific question cues and literal text markers. It demonstrates control flow, not a language model’s tool selection, rewriting ability or answer quality.
Let an observation control the next retrieval round
An observation records a completed tool operation: its name, arguments and validated result. ScriptedPolicy receives prior observations and returns a search action, a read action or a stop decision. It retains the original question while issuing smaller search terms.
The declared existing-file rule starts with a broad term. After reading a passage without its configured marker, it issues the more specific term. Once a validated paragraph contains the literal marker, the controller reports completion of that configured context search. It produces no answer.
The rule is visible in RULES. Selecting it from question words is application code, and its marker was chosen during development. A substring match does not test whether arbitrary retrieved text supports an answer or whether the whole task has succeeded.
Keep every query and read in the trace
The result retains the original question, requested version, policy configuration, query history and complete observations. A changed query does not overwrite an earlier unsuccessful attempt. Reads use issued handles and validate their results against the same pinned corpus.
| Actual operation | Query or source |
|---|---|
| search | text |
| read | path-read |
| search | overwritten |
| read | path-write |
The policy receives the question, requested version and tool observations. It receives no expected answer, expected source label or outcome grade. This separation prevents the evaluation label from directly selecting the next tool call; the handcrafted rule still reflects development choices.
Stop when another round cannot help this policy
Duplicate query. The controller compares a proposed refinement with previous queries and stops rather than issuing the same query again. The controlled duplicate case in the raw output returns duplicate_query.
No progress. A search that returns only already-read passage identities cannot keep the loop moving. Missing rules and exhausted round limits also stop with an incomplete result. Those reasons describe this policy’s limits; they do not certify that the corpus contains no answer.
Invalid read. A changed tool payload is refused before the controller adds it as a successful observation. Local tool failures remain distinct from a normal stop. A separate hard step guard protects against a faulty policy loop; it does not interrupt a blocked function call or impose a wall-clock timeout.
Run the declared refinement policy
Download the query-refinement lab. The implementation includes the question-only rule table and exact copies of the same pinned search/read backend.
"""A deterministic retrieval controller with explicit question-only rewrite rules."""
import copy
import re
from tools import DocumentationTools,load_snapshot,ToolError
RULES=(
('existing-path',('existing','overwrite','replace'),'overwritten','overwritten'),
('json-result',('json',),'json','JSON string'),
('read-text',('reading','read'),'read','opens a file'),
('write-text',('saving','write','pathlib'),'write','write_text'),
)
def rule_for(question):
if not isinstance(question,str) or not question.strip() or len(question)>512:raise ValueError('invalid_question')
words=set(re.findall('[a-z0-9]+',question.lower()))
for name,cues,query,marker in RULES:
if words.intersection(cues):return {'name':name,'initial_query':'text','refinement_query':query,'required_marker':marker,'max_rounds':2}
return {'name':'no-matching-rule','initial_query':'text','refinement_query':None,'required_marker':None,'max_rounds':2}
class ScriptedPolicy:
def __init__(self,question,version,config=None):
default=rule_for(question)
self.question=question;self.config=default if config is None else copy.deepcopy(config)
if not isinstance(version,str) or re.fullmatch(r'[0-9]+\.[0-9]+',version) is None:raise ValueError('invalid_version')
self.version=version;self.terminal_status=None;self.reason=None
config=self.config
if not isinstance(config,dict) or set(config)!={'name','initial_query','refinement_query','required_marker','max_rounds'}:raise ValueError('invalid_policy')
if not isinstance(config['name'],str) or not config['name']:raise ValueError('invalid_policy')
for key in ('initial_query','refinement_query'):
value=config[key]
if key=='refinement_query' and value is None:continue
if not isinstance(value,str) or re.fullmatch('[a-z0-9]{1,64}',value) is None:raise ValueError('invalid_policy')
if config['required_marker'] is not None and (not isinstance(config['required_marker'],str) or not config['required_marker'] or len(config['required_marker'])>128):raise ValueError('invalid_policy')
if type(config['max_rounds']) is not int or not 1<=config['max_rounds']<=3:raise ValueError('invalid_policy')
def stop(self,status,reason):self.terminal_status=status;self.reason=reason;return None
def __call__(self,observations):
if not isinstance(observations,list) or any(not isinstance(o,dict) or set(o)!={'tool','arguments','result'} or o['tool'] not in ('search','read') for o in observations):raise ValueError('invalid_observations')
searches=[o for o in observations if o['tool']=='search'];reads=[o for o in observations if o['tool']=='read']
if not observations:return {'tool':'search','arguments':{'query':self.config['initial_query'],'version':self.version,'k':2}}
last=observations[-1]
if last['tool']=='search':
already={o['result']['passage_id'] for o in reads}
eligible=[row for row in last['result']['results'] if row['passage_id'] not in already]
if eligible:return {'tool':'read','arguments':{'handle':copy.deepcopy(eligible[0])}}
reason='no_match' if not last['result']['results'] else 'no_new_passages'
if reason=='no_new_passages':return self.stop('incomplete',reason)
else:
marker=self.config['required_marker']
if marker is not None and marker in last['result']['text']:return self.stop('completed','configured_context_marker_found')
reason='marker_missing'
if len(searches)>=self.config['max_rounds']:return self.stop('incomplete','round_limit')
query=self.config['refinement_query']
if query is None:return self.stop('incomplete','no_refinement_rule')
previous=[o['arguments']['query'] for o in searches]
if query in previous:return self.stop('incomplete','duplicate_query')
return {'tool':'search','arguments':{'query':query,'version':self.version,'k':2}}
def run_controller(tools,question,version='3.12',config=None):
policy=ScriptedPolicy(question,version,config);observations=[]
try:
# Local policy's maximum is 3 search/read rounds; guard against bugs too.
for _ in range(7):
action=policy(observations)
if action is None:break
result=getattr(tools,action['tool'])(**action['arguments'])
if action['tool']=='read':tools.validate_read(result,action['arguments']['handle'])
observations.append({'tool':action['tool'],'arguments':copy.deepcopy(action['arguments']),'result':result})
else:return {'status':'failed','reason':'controller_step_guard','question':question,'version':version,'policy':policy.config,'observations':observations,'query_history':[o['arguments']['query'] for o in observations if o['tool']=='search']}
except (ToolError,ValueError) as error:
return {'status':'failed','reason':str(error),'question':question,'version':version,'policy':policy.config,'observations':observations,'query_history':[o['arguments']['query'] for o in observations if o['tool']=='search']}
return {'status':policy.terminal_status,'reason':policy.reason,'question':question,'version':version,'policy':policy.config,'observations':observations,'query_history':[o['arguments']['query'] for o in observations if o['tool']=='search']}python run.py --output scratch-result.json
python -m unittest discover -v{
"controlled_duplicate_reason": "duplicate_query",
"execution_mode": "offline-scripted-retrieval-controller",
"query_history": [
"text",
"overwritten"
],
"read_sources": [
"path-read",
"path-write"
],
"reason": "configured_context_marker_found",
"status": "completed"
}The output’s normal run finishes with completed because configured_context_marker_found. Read those fields together with the actual query history. They describe a configured source-search condition, not a generated or graded answer.
This standalone program has no HTTP deployment, provider adapter or model inference. A live agent would need a separately checked decision adapter, action admission and task-outcome evaluation. No such adapter, concurrency behavior or latency was measured here.
View the complete local execution JSON
{
"execution_mode": "offline-scripted-retrieval-controller",
"model_called": false,
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"controller": {
"status": "completed",
"reason": "configured_context_marker_found",
"question": "Will write_text replace the contents of an existing file?",
"version": "3.12",
"policy": {
"name": "existing-path",
"initial_query": "text",
"refinement_query": "overwritten",
"required_marker": "overwritten",
"max_rounds": 2
},
"observations": [
{
"tool": "search",
"arguments": {
"query": "text",
"version": "3.12",
"k": 2
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"query": "text",
"version": "3.12",
"k": 2,
"results": [
{
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124"
},
{
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"source_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"start_line": 1,
"end_line": 1,
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"byte_count": 224,
"native_score": -1.320388349514563e-06,
"rank": 2,
"read_token": "397d6f236743e576b50940b06d3df84a0712dc67157f5efdeb544448555729f1"
}
]
}
},
{
"tool": "read",
"arguments": {
"handle": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124"
}
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"read_request_id": "b2ce4d1e809a197df6c864261264b583737b1f41c59f8a4b6e353911b84c5bfe",
"result_sha256": "2d855cfa0b4f7e90c3711783b7779709f3b964760fd789d0d65b11f9b3604ac5"
}
},
{
"tool": "search",
"arguments": {
"query": "overwritten",
"version": "3.12",
"k": 2
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "aeffc68efdee46b8d23c597f651143424c575c0c1a4699ab08a48734d011f127",
"query": "overwritten",
"version": "3.12",
"k": 2,
"results": [
{
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "aeffc68efdee46b8d23c597f651143424c575c0c1a4699ab08a48734d011f127",
"passage_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"source_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"start_line": 1,
"end_line": 1,
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"byte_count": 224,
"native_score": -0.7992166451067193,
"rank": 1,
"read_token": "08b866d8cf00533f3484aac4d78ac512bd4df8767c9ea11fbd2e817eeeb1dd75"
}
]
}
},
{
"tool": "read",
"arguments": {
"handle": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "aeffc68efdee46b8d23c597f651143424c575c0c1a4699ab08a48734d011f127",
"passage_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"source_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"start_line": 1,
"end_line": 1,
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"byte_count": 224,
"native_score": -0.7992166451067193,
"rank": 1,
"read_token": "08b866d8cf00533f3484aac4d78ac512bd4df8767c9ea11fbd2e817eeeb1dd75"
}
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "aeffc68efdee46b8d23c597f651143424c575c0c1a4699ab08a48734d011f127",
"passage_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"source_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"start_line": 1,
"end_line": 1,
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"byte_count": 224,
"native_score": -0.7992166451067193,
"rank": 1,
"read_token": "08b866d8cf00533f3484aac4d78ac512bd4df8767c9ea11fbd2e817eeeb1dd75",
"text": "Path.write_text(data, encoding=None, errors=None, newline=None) writes text to a file and closes it. Set encoding=\"utf-8\" to select UTF-8. An existing file is overwritten. The method returns the number of characters written.",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.write_text",
"read_request_id": "a9ec642e4ed3206ac85e5dd03d4de9ff57de3167e134079eafe4a31de23cec67",
"result_sha256": "662c96d3b972224ac1549f9a6c3ad495b3ed885b6db477d52aeb7527ec1dbbb7"
}
}
],
"query_history": [
"text",
"overwritten"
]
},
"controlled_duplicate_stop": {
"status": "incomplete",
"reason": "duplicate_query",
"question": "Will write_text replace the contents of an existing file?",
"version": "3.12",
"policy": {
"name": "controlled-duplicate-query",
"initial_query": "text",
"refinement_query": "text",
"required_marker": "not-present",
"max_rounds": 2
},
"observations": [
{
"tool": "search",
"arguments": {
"query": "text",
"version": "3.12",
"k": 2
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"query": "text",
"version": "3.12",
"k": 2,
"results": [
{
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124"
},
{
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c66643a4609c6ee157458bf4353f7446b9c6a200a3ef453846d6a18d5b5069a7",
"source_id": "path-write",
"version": "3.12",
"document_revision": "28c6349eaf80b629322edb310bb871dc45a04af9fad3b752bc75d12f0d7314e7",
"source_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"start_line": 1,
"end_line": 1,
"text_sha256": "a48b4b9f763cd6309ebe32af19c5a814adcf810c2259683da9f328ccee46948e",
"byte_count": 224,
"native_score": -1.320388349514563e-06,
"rank": 2,
"read_token": "397d6f236743e576b50940b06d3df84a0712dc67157f5efdeb544448555729f1"
}
]
}
},
{
"tool": "read",
"arguments": {
"handle": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124"
}
},
"result": {
"corpus_id": "0615d22a3315e97c0a88c6c566c546877304f5a28a97e83f68dd36881d4bbd0e",
"search_request_id": "3f33383e706e36449e22c380704d880d5e2b8a4b1f6b779b8aa54206e072ad5f",
"passage_id": "c737db948e8111ae00e3442d8a22c80c876e43446dc1ee176ffe8ce142a41900",
"source_id": "path-read",
"version": "3.12",
"document_revision": "b58385b9bf597a951c16c035653d639929924e53eae3e5ada255afd6ee1592f2",
"source_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"start_line": 1,
"end_line": 1,
"text_sha256": "dc097ca63486ace45279f412ea266bf754fcd2f0b1153f5d763ce905dafbc2c0",
"byte_count": 154,
"native_score": -1.4220532319391636e-06,
"rank": 1,
"read_token": "72abbf5e0022ef780447ed8c2322b711cac8b4971d817a440bc02dfa28b07124",
"text": "Path.read_text(encoding=None, errors=None) opens a file, returns its text as a string and closes the file. Set encoding=\"utf-8\" when reading a UTF-8 file.",
"url": "https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.read_text",
"read_request_id": "b2ce4d1e809a197df6c864261264b583737b1f41c59f8a4b6e353911b84c5bfe",
"result_sha256": "2d855cfa0b4f7e90c3711783b7779709f3b964760fd789d0d65b11f9b3604ac5"
}
}
],
"query_history": [
"text"
]
},
"interpretation": "Declared question-only rules and literal passage checks; not an executed model agent or semantic quality experiment."
}The command prints the text-to-overwritten query sequence, reads path-read then path-write, and retains the duplicate-query stop. The controller follows authored rules rather than model decisions.
Change the stopping condition and compare
Run the normal fixture and inspect the passage read before the rewritten query. Then compare the controlled duplicate case. Explain which observation triggered refinement, why the second query differs, and what a literal marker can fail to tell you about the original question.
Read the explanation
The trace preserves the question, queries and observations so the actual stop can be inspected.
What’s next
Continue with Limit Retrieval-Agent Runs.
References
The Python copying documentation, checked on 23 October 2026, explains copying the structured policy configuration and selected handles. Python unittest documents the local assertion framework. Query cues, literal markers and stopping rules are supplied application choices; these sources do not validate semantic reasoning or agent performance.