Problem
When you need this. Use this when your application talks to more than one model API format and you need a single typed adapter backed by shared contract tests.
The request tracer and the first local client both speak Ollama’s native API, /api/chat, directly. Their code reads done_reason, prompt_eval_count and eval_count. Move to another server and those names are gone. Many servers, including llama.cpp’s llama-server, vLLM and Ollama itself, also offer an OpenAI-compatible API at /v1/chat/completions, with different field names: finish_reason, usage.prompt_tokens, max_tokens.
Renaming fields is the easy part. The hard part is that two APIs can accept the same intent and behave differently without an error: a setting one of them cannot carry, a default that changes the reply, an error in another shape. Application code that is ported field by field inherits each of these silently.
This build puts one contract in front of both wire formats and tests each adapter against it. Success check: the same contract test methods pass for both adapters against traffic recorded from both endpoints with two models, the two adapters return equal results for every recorded case, and a deliberate omission in either adapter makes a named test fail.
Prerequisites: the request tracer, or the same knowledge of /api/chat, finish reasons and failure paths; Python dataclasses, exceptions and unittest from Programming for the builds.
Concepts
An adapter translates; a contract says what it must preserve
An adapter is code that translates between the shape your application uses and the shape one external system uses. A contract is the set of rules every adapter must follow, written so a test can check them. Ours has three parts, all in adapters.py:
- Request
ChatRequest: messages as (role, text) pairs, an output limit, temperature, seed and an optional context size.validatechecks it before any adapter builds a body: known roles, a system message only first, the last message from the user, limits in range.- Result
ChatResult: nonblank text, a finish ofstoporlength, and input and output token counts. One shared function,result, builds it for both adapters, so neither can accept a reply the other would refuse.- Errors
AdapterErrorwith one of six kinds:connection,timeout,model_not_found,http,protocolandunsupported. The application handles kinds, never a server’s error text.
Two further rules are about behavior, not shapes. Thinking is off in every request, because a model that spends its output limit on hidden reasoning returns no text. And an option an API cannot carry is refused, never dropped: dropping it would let two adapters answer the same request under different settings.
A contract test runs once per adapter
A contract test checks an adapter against the contract, not against its own implementation. The suite is written once, as a class of test methods, and each adapter gets a subclass that only names the adapter. A new adapter is supported when it passes every method unchanged. This is different from a unit test of one function and from an outcome check of answer quality: a contract test says the adapter preserves the contract’s meaning, and nothing about whether the model’s answer is right.
The two wire formats, side by side
These are the bodies each adapter sent for the same ChatRequest, taken from the recording:
| Contract field | Native /api/chat |
OpenAI-compatible /v1/chat/completions |
|---|---|---|
| Output limit | options.num_predict: 48 |
max_tokens: 48 |
| Thinking off | think: false |
reasoning_effort: "none" |
| Context size | options.num_ctx |
No field; the adapter refuses it |
| Finish | done_reason |
choices[0].finish_reason |
| Token counts | prompt_eval_count, eval_count |
usage.prompt_tokens, usage.completion_tokens |
| Missing model | HTTP 404, {"error":"model 'aiml-missing:never' not found"} |
HTTP 404, an error object with message and type "not_found_error" |
Ollama documents that its OpenAI-compatible API “does not have a way of setting the context size for a model”; changing it there means creating a new model with a different num_ctx parameter. That is why the contract’s context_tokens is optional, and why one adapter must refuse it.
Architecture
Two wire formats share one application contract and one local model; unsupported options are refused
Application code builds a ChatRequest and calls complete(model, request) on whichever adapter it was given. The request is checked first, so a malformed request fails the same way on both. Each adapter then writes its own body and posts it with the same standard-library function, http_post, which returns the HTTP status and body bytes and turns transport failures into connection or timeout. Both paths reach the same Ollama server and the same model. Each adapter reads its own response shape, and both pass what they read to the same result function. The process is one Python program and one local server; nothing is cached or retried.
Working code
Choose a model
Both models were checked on their publishers’ model cards on 2026-10-20 and recorded through both adapters. The adapters do not depend on the model; using two shows the contract holds for more than one.
| Choice | Model | License | Runtime | Task compatibility | Verification |
|---|---|---|---|---|---|
| Tested default | Gemma 4 E2B IT, Ollama gemma4:e2b, digest b37049369adf |
Apache 2.0 | Ollama 0.32.14, both endpoints |
Text chat with a system role | Four cases recorded through each adapter |
| Smaller local option | Qwen3.5 0.8B, Ollama qwen3.5:0.8b, digest de63045f2975 |
Apache 2.0 | Ollama 0.32.14, both endpoints |
Text chat with a system role | Four cases recorded through each adapter |
Servers other than Ollama that offer /v1/chat/completions are candidates for the same adapter. None was tested here, so none is claimed as supported: each needs its own recorded traffic and a passing contract suite.
Read the adapters
Download the files listed in the README into one directory.
"""One chat contract, two wire formats: Ollama's native /api/chat and the OpenAI-compatible
/v1/chat/completions. Application code depends on ChatRequest, ChatResult and AdapterError only.
Python 3.11+, standard library only.
python3 adapters.py --api openai-compatible "What does JSON stand for?"
"""
import argparse
import json
import math
import socket
from dataclasses import dataclass
from urllib.error import HTTPError, URLError
from urllib.parse import urlsplit
from urllib.request import Request, urlopen
ROLES = ('system', 'user', 'assistant')
FINISH = ('stop', 'length')
KINDS = ('connection', 'timeout', 'model_not_found', 'http', 'protocol', 'unsupported')
class AdapterError(RuntimeError):
"""A failure in one of the six kinds the contract names. No adapter retries."""
def __init__(self, kind, message):
assert kind in KINDS
super().__init__(message)
self.kind = kind
@dataclass(frozen=True)
class ChatRequest:
messages: tuple
max_tokens: int = 64
temperature: float = 0.0
seed: int | None = 42
context_tokens: int | None = None
@dataclass(frozen=True)
class ChatResult:
text: str
finish: str
input_tokens: int
output_tokens: int
def validate(request):
"""The request half of the contract. Runs before any adapter builds a body."""
messages = request.messages
if not isinstance(messages, tuple) or not messages:
raise ValueError('messages must be a nonempty tuple of (role, content) pairs.')
for index, item in enumerate(messages):
if not (isinstance(item, tuple) and len(item) == 2 and item[0] in ROLES
and isinstance(item[1], str) and item[1].strip()):
raise ValueError(f'Message {index} must be (role, text) with a known role and nonblank text.')
if item[0] == 'system' and index != 0:
raise ValueError('Only the first message may be a system message.')
if messages[-1][0] != 'user':
raise ValueError('The last message must come from the user.')
if type(request.max_tokens) is not int or not 1 <= request.max_tokens <= 4096:
raise ValueError('max_tokens must be an integer from 1 to 4096.')
if (isinstance(request.temperature, bool) or not isinstance(request.temperature, (int, float))
or not 0 <= request.temperature <= 2):
raise ValueError('temperature must be a number from 0 to 2.')
if request.seed is not None and type(request.seed) is not int:
raise ValueError('seed must be an integer or None.')
if request.context_tokens is not None and (type(request.context_tokens) is not int
or request.context_tokens < 256):
raise ValueError('context_tokens must be an integer of at least 256, or None.')
def http_post(url, payload, headers, timeout):
"""Send one JSON POST. Returns (status, body bytes); raises only for transport failures."""
request = Request(url, data=json.dumps(payload).encode(), headers=headers)
try:
with urlopen(request, timeout=timeout) as response:
return response.status, response.read()
except HTTPError as error:
return error.code, error.read()
except (TimeoutError, socket.timeout) as error:
raise AdapterError('timeout', 'The model server did not answer in time. No retry was sent.') from error
except URLError as error:
if isinstance(error.reason, (TimeoutError, socket.timeout)):
raise AdapterError('timeout', 'The model server did not answer in time. No retry was sent.') from error
raise AdapterError('connection', f'Cannot reach the model server: {error.reason}') from error
except ConnectionError as error:
raise AdapterError('connection', f'Cannot reach the model server: {error}') from error
def local_origin(base_url):
url = urlsplit(base_url)
if url.scheme != 'http' or url.hostname not in ('localhost', '127.0.0.1', '::1'):
raise ValueError('These adapters accept only a local HTTP model server.')
if url.username or url.password or url.query or url.fragment or url.path not in ('', '/'):
raise ValueError('Use a server origin without credentials or a path.')
return base_url.rstrip('/')
def decode(status, body):
"""Shared status and JSON handling. Each adapter then reads its own response shape."""
try:
data = json.loads(body)
except (ValueError, UnicodeError):
data = None
if status == 404:
raise AdapterError('model_not_found', 'The server does not have this model. Pull it explicitly.')
if status != 200:
raise AdapterError('http', f'The model server returned HTTP {status}.')
if not isinstance(data, dict):
raise AdapterError('protocol', 'The response body is not a JSON object.')
return data
def count(value, name):
if type(value) is not int or value < 0:
raise AdapterError('protocol', f'Missing or invalid token count: {name}.')
return value
def result(text, finish, input_tokens, output_tokens):
"""The result half of the contract, shared so both adapters accept exactly the same results."""
if not isinstance(text, str) or not text.strip():
raise AdapterError('protocol', 'The reply has no text. Check that thinking is turned off.')
if finish not in FINISH:
raise AdapterError('protocol', f'Unsupported finish reason: {finish!r}.')
return ChatResult(text.strip(), finish, input_tokens, output_tokens)
class OllamaChatAdapter:
"""Ollama's native API: POST /api/chat."""
name = 'ollama-native'
path = '/api/chat'
def __init__(self, base_url='http://127.0.0.1:11434', post=http_post, timeout=120):
if not isinstance(timeout, (int, float)) or not math.isfinite(timeout) or timeout <= 0:
raise ValueError('Timeout must be positive and finite.')
self.base_url, self.post, self.timeout = local_origin(base_url), post, timeout
def body(self, model, request):
options = {'num_predict': request.max_tokens, 'temperature': request.temperature}
if request.seed is not None:
options['seed'] = request.seed
if request.context_tokens is not None:
options['num_ctx'] = request.context_tokens
return {'model': model, 'messages': [{'role': r, 'content': c} for r, c in request.messages],
'stream': False, 'think': False, 'options': options}
def parse(self, status, body):
data = decode(status, body)
message = data.get('message')
if data.get('done') is not True or not isinstance(message, dict) or message.get('role') != 'assistant':
raise AdapterError('protocol', 'Expected a completed assistant message.')
if message.get('tool_calls'):
raise AdapterError('protocol', 'The reply asks for a tool call; this contract has none.')
return result(message.get('content'), data.get('done_reason'),
count(data.get('prompt_eval_count'), 'prompt_eval_count'),
count(data.get('eval_count'), 'eval_count'))
def complete(self, model, request):
validate(request)
status, body = self.post(self.base_url + self.path, self.body(model, request),
{'Content-Type': 'application/json'}, self.timeout)
return self.parse(status, body)
class OpenAICompatibleAdapter(OllamaChatAdapter):
"""The OpenAI-compatible API that Ollama also serves: POST /v1/chat/completions."""
name = 'openai-compatible'
path = '/v1/chat/completions'
def body(self, model, request):
if request.context_tokens is not None:
raise AdapterError('unsupported', 'This API cannot set the context size; leave context_tokens unset.')
body = {'model': model, 'messages': [{'role': r, 'content': c} for r, c in request.messages],
'max_tokens': request.max_tokens, 'temperature': request.temperature,
'stream': False, 'reasoning_effort': 'none'}
if request.seed is not None:
body['seed'] = request.seed
return body
def parse(self, status, body):
data = decode(status, body)
choices = data.get('choices')
if not isinstance(choices, list) or len(choices) != 1 or not isinstance(choices[0], dict):
raise AdapterError('protocol', 'Expected exactly one choice.')
message, usage = choices[0].get('message'), data.get('usage')
if not isinstance(message, dict) or message.get('role') != 'assistant' or not isinstance(usage, dict):
raise AdapterError('protocol', 'Expected an assistant message and a usage object.')
if message.get('tool_calls'):
raise AdapterError('protocol', 'The reply asks for a tool call; this contract has none.')
return result(message.get('content'), choices[0].get('finish_reason'),
count(usage.get('prompt_tokens'), 'prompt_tokens'),
count(usage.get('completion_tokens'), 'completion_tokens'))
ADAPTERS = {adapter.name: adapter for adapter in (OllamaChatAdapter, OpenAICompatibleAdapter)}
def main():
parser = argparse.ArgumentParser(description='Ask one question through either adapter.')
parser.add_argument('question')
parser.add_argument('--api', choices=sorted(ADAPTERS), default='ollama-native')
parser.add_argument('--model', default='gemma4:e2b')
parser.add_argument('--base-url', default='http://127.0.0.1:11434')
parser.add_argument('--max-tokens', type=int, default=64)
args = parser.parse_args()
request = ChatRequest((('system', 'Answer in one short sentence.'), ('user', args.question)),
max_tokens=args.max_tokens)
try:
reply = ADAPTERS[args.api](args.base_url).complete(args.model, request)
except (ValueError, AdapterError) as error:
kind = getattr(error, 'kind', 'request')
parser.exit(1, f'{kind}: {error}\n')
print(json.dumps(reply.__dict__, ensure_ascii=False))
if __name__ == '__main__':
main()Start with validate and result: they are the contract, and neither belongs to an adapter. decode is shared too. It maps HTTP 404 to model_not_found before reading the body, because the two APIs put the reason in different places, and any other non-200 status to http.
OllamaChatAdapter.body puts the limits in options and sends think: false. OpenAICompatibleAdapter inherits complete and the constructor, and replaces only body, parse and path. Its body raises unsupported for a context size before anything is sent, and sends reasoning_effort: "none" as its way of turning thinking off. Each parse reads its own shape, rejects tool calls and a second choice, and hands text, finish reason and counts to result.
http_post returns an HTTP error as a status and body instead of raising, so decode decides what it means. It raises only when there is no response at all, and never retries: a timed-out request may still be running on the server.
Record both wire formats
record_wire.py record sends four cases per model through each adapter’s own body and saves the exact request, status and response body. The fourth case, thinking-left-on, removes the adapter’s thinking switch to show what the server does by default. Without record, it replays the saved responses through each adapter’s parse and writes the comparison:
Inspect record_wire.py
"""Record what each wire format sends and receives, then compare the adapters offline.
python3 record_wire.py record # needs Ollama with gemma4:e2b and qwen3.5:0.8b; writes wire-record.json
python3 record_wire.py # offline: replays wire-record.json through the adapters, writes contract-report.json
Python 3.11+, standard library only.
"""
import argparse
import json
import platform
import time
from pathlib import Path
from urllib.request import urlopen
from adapters import ADAPTERS, AdapterError, ChatRequest, http_post, local_origin, validate
HERE = Path(__file__).resolve().parent
MODELS = ('gemma4:e2b', 'qwen3.5:0.8b')
MESSAGES = (('system', 'Answer in one short sentence.'), ('user', 'What does HTTP stand for?'))
CASES = {
'complete': ('{model}', ChatRequest(MESSAGES, max_tokens=48)),
'output-limit': ('{model}', ChatRequest(MESSAGES, max_tokens=3)),
'missing-model': ('aiml-missing:never', ChatRequest(MESSAGES, max_tokens=48)),
'thinking-left-on': ('{model}', ChatRequest(MESSAGES, max_tokens=48)),
}
THINKING_KEYS = {'ollama-native': 'think', 'openai-compatible': 'reasoning_effort'}
def wire_body(adapter, model, case, request):
"""The adapter's own body, except the thinking-left-on case, which drops the thinking switch."""
body = adapter.body(model, request)
if case == 'thinking-left-on':
body.pop(THINKING_KEYS[adapter.name])
return body
def record(base_url='http://127.0.0.1:11434', post=http_post, clock=time.perf_counter):
rows = []
for model in MODELS:
for name, Adapter in ADAPTERS.items():
adapter = Adapter(base_url)
for case, (target, request) in CASES.items():
validate(request)
body = wire_body(adapter, target.format(model=model), case, request)
started = clock()
status, raw = post(adapter.base_url + adapter.path, body, {'Content-Type': 'application/json'}, 120)
rows.append({'model': model, 'adapter': name, 'case': case, 'request': body,
'status': status, 'body': raw.decode('utf-8'),
'client_ms': round((clock() - started) * 1000, 1)})
origin = local_origin(base_url)
with urlopen(origin + '/api/version', timeout=10) as response:
version = json.load(response)['version']
with urlopen(origin + '/api/tags', timeout=10) as response:
digests = {m['name']: m['digest'] for m in json.load(response)['models'] if m['name'] in MODELS}
return {'recorded_on': time.strftime('%Y-%m-%d'),
'environment': {'python': platform.python_version(), 'platform': platform.platform(),
'ollama': version, 'digests': digests},
'rows': rows}
def outcome(row):
"""Replay one recorded response through the adapter's own parser."""
adapter = ADAPTERS[row['adapter']]()
try:
r = adapter.parse(row['status'], row['body'].encode('utf-8'))
except AdapterError as error:
return {'error': error.kind, 'message': str(error)}
return {'text': r.text, 'finish': r.finish, 'input_tokens': r.input_tokens, 'output_tokens': r.output_tokens}
def report(record_data):
"""Offline and deterministic: one outcome per model, case and adapter, and whether the two agree."""
table = {}
for row in record_data['rows']:
table.setdefault(row['model'], {}).setdefault(row['case'], {})[row['adapter']] = outcome(row)
for cases in table.values():
for outcomes in cases.values():
native, compatible = outcomes['ollama-native'], outcomes['openai-compatible']
outcomes['agree'] = native == compatible
return table
def dumps(data):
return json.dumps(data, indent=2, ensure_ascii=False) + '\n'
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('mode', nargs='?', choices=['record', 'report'], default='report')
parser.add_argument('--base-url', default='http://127.0.0.1:11434')
args = parser.parse_args()
if args.mode == 'record':
(HERE / 'wire-record.json').write_text(dumps(record(args.base_url)))
data = report(json.loads((HERE / 'wire-record.json').read_text()))
(HERE / 'contract-report.json').write_text(dumps(data))
for model, cases in data.items():
for case, outcomes in cases.items():
print(f"{model:14} {case:17} agree={outcomes['agree']!s:5} "
f"{outcomes['ollama-native'].get('text', outcomes['ollama-native'].get('error'))!r:50.50} "
f"{outcomes['openai-compatible'].get('text', outcomes['openai-compatible'].get('error'))!r:.50}")
if __name__ == '__main__':
main()python3 record_wire.py record
python3 record_wire.py{
"gemma4:e2b": {
"complete": {
"ollama-native": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 26,
"output_tokens": 9
},
"openai-compatible": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 26,
"output_tokens": 9
},
"agree": true
},
"output-limit": {
"ollama-native": {
"text": "HTTP stands for",
"finish": "length",
"input_tokens": 26,
"output_tokens": 3
},
"openai-compatible": {
"text": "HTTP stands for",
"finish": "length",
"input_tokens": 26,
"output_tokens": 3
},
"agree": true
},
"missing-model": {
"ollama-native": {
"error": "model_not_found",
"message": "The server does not have this model. Pull it explicitly."
},
"openai-compatible": {
"error": "model_not_found",
"message": "The server does not have this model. Pull it explicitly."
},
"agree": true
},
"thinking-left-on": {
"ollama-native": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 28,
"output_tokens": 9
},
"openai-compatible": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 28,
"output_tokens": 9
},
"agree": true
}
},
"qwen3.5:0.8b": {
"complete": {
"ollama-native": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 29,
"output_tokens": 9
},
"openai-compatible": {
"text": "HTTP stands for Hypertext Transfer Protocol.",
"finish": "stop",
"input_tokens": 29,
"output_tokens": 9
},
"agree": true
},
"output-limit": {
"ollama-native": {
"text": "HTTP stands for",
"finish": "length",
"input_tokens": 29,
"output_tokens": 3
},
"openai-compatible": {
"text": "HTTP stands for",
"finish": "length",
"input_tokens": 29,
"output_tokens": 3
},
"agree": true
},
"missing-model": {
"ollama-native": {
"error": "model_not_found",
"message": "The server does not have this model. Pull it explicitly."
},
"openai-compatible": {
"error": "model_not_found",
"message": "The server does not have this model. Pull it explicitly."
},
"agree": true
},
"thinking-left-on": {
"ollama-native": {
"error": "protocol",
"message": "The reply has no text. Check that thinking is turned off."
},
"openai-compatible": {
"error": "protocol",
"message": "The reply has no text. Check that thinking is turned off."
},
"agree": true
}
}
}Inspect the recorded traffic · wire-record.json
{
"recorded_on": "2026-10-20",
"environment": {
"python": "3.15.0",
"platform": "macOS-27.0.1-arm64-arm-64bit-Mach-O",
"ollama": "0.32.14",
"digests": {
"gemma4:e2b": "b37049369adfe3d2b653af0ab301a062ca5cbe96ace6aa0d3f2559ee9b563fc2",
"qwen3.5:0.8b": "de63045f29755d4721c09d54c9b45c2dadb644f4fd7d604d4cac90d96049971c"
}
},
"rows": [
{
"model": "gemma4:e2b",
"adapter": "ollama-native",
"case": "complete",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"gemma4:e2b\",\"created_at\":\"2026-10-20T17:07:06.83998Z\",\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":3253627750,\"load_duration\":3024329250,\"prompt_eval_count\":26,\"prompt_eval_duration\":85003000,\"eval_count\":9,\"eval_duration\":143246000}",
"client_ms": 3264.1
},
{
"model": "gemma4:e2b",
"adapter": "ollama-native",
"case": "output-limit",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 3,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"gemma4:e2b\",\"created_at\":\"2026-10-20T17:07:12.478553Z\",\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for\"},\"done\":true,\"done_reason\":\"length\",\"total_duration\":5637498417,\"load_duration\":5518662834,\"prompt_eval_count\":26,\"prompt_eval_duration\":84000000,\"eval_count\":3,\"eval_duration\":33523000}",
"client_ms": 5638.1
},
{
"model": "gemma4:e2b",
"adapter": "ollama-native",
"case": "missing-model",
"request": {
"model": "aiml-missing:never",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 404,
"body": "{\"error\":\"model 'aiml-missing:never' not found\"}",
"client_ms": 0.9
},
{
"model": "gemma4:e2b",
"adapter": "ollama-native",
"case": "thinking-left-on",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"gemma4:e2b\",\"created_at\":\"2026-10-20T17:07:17.95317Z\",\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":5473064625,\"load_duration\":5252614250,\"prompt_eval_count\":28,\"prompt_eval_duration\":80521000,\"eval_count\":9,\"eval_duration\":137231000}",
"client_ms": 5473.6
},
{
"model": "gemma4:e2b",
"adapter": "openai-compatible",
"case": "complete",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-943\",\"object\":\"chat.completion\",\"created\":1792516038,\"model\":\"gemma4:e2b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":26,\"completion_tokens\":9,\"total_tokens\":35}}\n",
"client_ms": 945.9
},
{
"model": "gemma4:e2b",
"adapter": "openai-compatible",
"case": "output-limit",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 3,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-840\",\"object\":\"chat.completion\",\"created\":1792516044,\"model\":\"gemma4:e2b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for\"},\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":26,\"completion_tokens\":3,\"total_tokens\":29}}\n",
"client_ms": 5362.8
},
{
"model": "gemma4:e2b",
"adapter": "openai-compatible",
"case": "missing-model",
"request": {
"model": "aiml-missing:never",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 404,
"body": "{\"error\":{\"message\":\"model 'aiml-missing:never' not found\",\"type\":\"not_found_error\",\"param\":null,\"code\":null}}\n",
"client_ms": 0.8
},
{
"model": "gemma4:e2b",
"adapter": "openai-compatible",
"case": "thinking-left-on",
"request": {
"model": "gemma4:e2b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-561\",\"object\":\"chat.completion\",\"created\":1792516045,\"model\":\"gemma4:e2b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":28,\"completion_tokens\":9,\"total_tokens\":37}}\n",
"client_ms": 915.0
},
{
"model": "qwen3.5:0.8b",
"adapter": "ollama-native",
"case": "complete",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"qwen3.5:0.8b\",\"created_at\":\"2026-10-20T17:07:25.484113Z\",\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":305279250,\"load_duration\":153923417,\"prompt_eval_count\":29,\"prompt_eval_duration\":25572000,\"eval_count\":9,\"eval_duration\":124392000}",
"client_ms": 305.8
},
{
"model": "qwen3.5:0.8b",
"adapter": "ollama-native",
"case": "output-limit",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 3,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"qwen3.5:0.8b\",\"created_at\":\"2026-10-20T17:07:25.703836Z\",\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for\"},\"done\":true,\"done_reason\":\"length\",\"total_duration\":219233666,\"load_duration\":155552208,\"prompt_eval_count\":29,\"prompt_eval_duration\":32194000,\"eval_count\":3,\"eval_duration\":30362000}",
"client_ms": 219.7
},
{
"model": "qwen3.5:0.8b",
"adapter": "ollama-native",
"case": "missing-model",
"request": {
"model": "aiml-missing:never",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"think": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 404,
"body": "{\"error\":\"model 'aiml-missing:never' not found\"}",
"client_ms": 0.8
},
{
"model": "qwen3.5:0.8b",
"adapter": "ollama-native",
"case": "thinking-left-on",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"stream": false,
"options": {
"num_predict": 48,
"temperature": 0.0,
"seed": 42
}
},
"status": 200,
"body": "{\"model\":\"qwen3.5:0.8b\",\"created_at\":\"2026-10-20T17:07:26.390877Z\",\"message\":{\"role\":\"assistant\",\"content\":\"\",\"thinking\":\"Thinking Process:\\n\\n1. **Analyze the Request:** The user is asking for the acronym \\\"HTTP\\\" and wants the answer in one short sentence.\\n\\n2. **Identify the Acronym:** HTTP stands for Hypertext\"},\"done\":true,\"done_reason\":\"length\",\"total_duration\":685886500,\"load_duration\":153232916,\"prompt_eval_count\":27,\"prompt_eval_duration\":11564000,\"eval_count\":48,\"eval_duration\":519955000}",
"client_ms": 686.3
},
{
"model": "qwen3.5:0.8b",
"adapter": "openai-compatible",
"case": "complete",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-351\",\"object\":\"chat.completion\",\"created\":1792516046,\"model\":\"qwen3.5:0.8b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for Hypertext Transfer Protocol.\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":29,\"completion_tokens\":9,\"total_tokens\":38}}\n",
"client_ms": 299.5
},
{
"model": "qwen3.5:0.8b",
"adapter": "openai-compatible",
"case": "output-limit",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 3,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-52\",\"object\":\"chat.completion\",\"created\":1792516046,\"model\":\"qwen3.5:0.8b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"HTTP stands for\"},\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":29,\"completion_tokens\":3,\"total_tokens\":32}}\n",
"client_ms": 248.6
},
{
"model": "qwen3.5:0.8b",
"adapter": "openai-compatible",
"case": "missing-model",
"request": {
"model": "aiml-missing:never",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"reasoning_effort": "none",
"seed": 42
},
"status": 404,
"body": "{\"error\":{\"message\":\"model 'aiml-missing:never' not found\",\"type\":\"not_found_error\",\"param\":null,\"code\":null}}\n",
"client_ms": 5.0
},
{
"model": "qwen3.5:0.8b",
"adapter": "openai-compatible",
"case": "thinking-left-on",
"request": {
"model": "qwen3.5:0.8b",
"messages": [
{
"role": "system",
"content": "Answer in one short sentence."
},
{
"role": "user",
"content": "What does HTTP stand for?"
}
],
"max_tokens": 48,
"temperature": 0.0,
"stream": false,
"seed": 42
},
"status": 200,
"body": "{\"id\":\"chatcmpl-631\",\"object\":\"chat.completion\",\"created\":1792516047,\"model\":\"qwen3.5:0.8b\",\"system_fingerprint\":\"fp_ollama\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"\",\"reasoning\":\"Thinking Process:\\n\\n1. **Analyze the Request:** The user is asking for the acronym \\\"HTTP\\\" and wants the answer in one short sentence.\\n\\n2. **Identify the Acronym:** HTTP stands for Hypertext\"},\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":27,\"completion_tokens\":48,\"total_tokens\":75}}\n",
"client_ms": 633.4
}
]
}The run used Ollama 0.32.14 on 2026-10-20.
Read the result
The two adapters agree on every recorded case: 8 of 8. For the complete case with gemma4:e2b, both returned {"text":"HTTP stands for Hypertext Transfer Protocol.","finish":"stop","input_tokens":26,"output_tokens":9}. The input token counts match too, 26 through each endpoint, which is consistent with the server formatting the same messages the same way. With an output limit of 3, both returned the cut-off text “HTTP stands for” with finish length, which the contract treats as a result for the caller to inspect, not an error.
Leaving thinking at the server default changes what happens, through both endpoints. In the thinking-left-on case, qwen3.5:0.8b spent all 48 output tokens on reasoning, returned empty content, and finished with length on both APIs. Both adapters turned that into a protocol error instead of an empty answer. The Qwen3.5 0.8B model card describes non-thinking as the model’s default; this Ollama packaging behaved otherwise. Recorded traffic shows this kind of difference; the model card does not. gemma4:e2b still answered within the limit, but its input grew from 26 to 28 tokens: the server changed the prompt it built even though the reply looked the same.
The thinking switch has two names. The native API’s think: false and the OpenAI-compatible reasoning_effort: "none" produced the same input token counts and the same text. An adapter that forgets its switch is not visibly broken with gemma4:e2b; it breaks with qwen3.5:0.8b. The contract tests catch it either way, because they compare the body each adapter sends with the recorded one.
Use either adapter from application code
The command-line example asks one question through the adapter named by --api. The application code is the same for both:
$ python3 adapters.py --api ollama-native 'What does JSON stand for?'
{"text": "JSON stands for JavaScript Object Notation.", "finish": "stop", "input_tokens": 26, "output_tokens": 8}
$ python3 adapters.py --api openai-compatible 'What does JSON stand for?'
{"text": "JSON stands for JavaScript Object Notation.", "finish": "stop", "input_tokens": 26, "output_tokens": 8}
$ python3 adapters.py --api openai-compatible --model aiml-missing:never 'What does JSON stand for?'
model_not_found: The server does not have this model. Pull it explicitly.
exit status 1
$ python3 adapters.py --base-url http://127.0.0.1:1 'What does JSON stand for?'
connection: Cannot reach the model server: [Errno 61] Connection refused
exit status 1The last two commands show the error kinds reaching the caller: model_not_found from the OpenAI-compatible error object, and connection from a port where nothing listens.
Failure cases
Run the contract suite. It replays recorded responses and edited copies of them; no server is needed:
python3 -m unittest -v test_contract.pytest_a_completed_reply_is_normalized (test_contract.CompatibleContract.test_a_completed_reply_is_normalized) ... ok
test_a_missing_model_is_model_not_found (test_contract.CompatibleContract.test_a_missing_model_is_model_not_found) ... ok
test_an_option_the_api_cannot_carry_is_refused_not_dropped (test_contract.CompatibleContract.test_an_option_the_api_cannot_carry_is_refused_not_dropped) ... ok
test_requests_are_checked_before_anything_is_sent (test_contract.CompatibleContract.test_requests_are_checked_before_anything_is_sent) ... ok
test_server_errors_and_malformed_bodies_get_a_named_kind (test_contract.CompatibleContract.test_server_errors_and_malformed_bodies_get_a_named_kind) ... ok
test_the_output_limit_is_a_result_not_an_error (test_contract.CompatibleContract.test_the_output_limit_is_a_result_not_an_error) ... ok
test_thinking_is_turned_off_in_every_request (test_contract.CompatibleContract.test_thinking_is_turned_off_in_every_request) ... ok
test_transport_failures_are_reported_once_without_retry (test_contract.CompatibleContract.test_transport_failures_are_reported_once_without_retry) ... ok
test_a_completed_reply_is_normalized (test_contract.NativeContract.test_a_completed_reply_is_normalized) ... ok
test_a_missing_model_is_model_not_found (test_contract.NativeContract.test_a_missing_model_is_model_not_found) ... ok
test_an_option_the_api_cannot_carry_is_refused_not_dropped (test_contract.NativeContract.test_an_option_the_api_cannot_carry_is_refused_not_dropped) ... ok
test_requests_are_checked_before_anything_is_sent (test_contract.NativeContract.test_requests_are_checked_before_anything_is_sent) ... ok
test_server_errors_and_malformed_bodies_get_a_named_kind (test_contract.NativeContract.test_server_errors_and_malformed_bodies_get_a_named_kind) ... ok
test_the_output_limit_is_a_result_not_an_error (test_contract.NativeContract.test_the_output_limit_is_a_result_not_an_error) ... ok
test_thinking_is_turned_off_in_every_request (test_contract.NativeContract.test_thinking_is_turned_off_in_every_request) ... ok
test_transport_failures_are_reported_once_without_retry (test_contract.NativeContract.test_transport_failures_are_reported_once_without_retry) ... ok
test_both_wire_formats_give_the_same_result_for_every_recorded_case (test_contract.RecordedComparison.test_both_wire_formats_give_the_same_result_for_every_recorded_case) ... ok
test_the_report_is_reproduced_from_the_recording (test_contract.RecordedComparison.test_the_report_is_reproduced_from_the_recording) ... ok
----------------------------------------------------------------------
Ran 18 tests in 0.002s
OKtest_contract.py defines 10 test methods and the runner ran 18: the contract methods in Contract run once for each adapter subclass, and two more check the recorded comparison. The subclasses add only the adapter class, whether it can set a context size, and how to break a recorded response of its own shape.
| Case | Expected behavior |
|---|---|
| The model is not installed | model_not_found from either error shape. |
| Thinking is left at the server default | Each body carries its thinking switch; an empty reply caused by reasoning is a protocol error, never an empty answer. |
| An HTTP error, invalid JSON, or a response with the wrong shape | http for a non-200 status; protocol for a body that is not a JSON object, an unfinished reply, a wrong role, empty text, a tool call, an unknown finish reason, a missing or negative count, or the wrong number of choices. |
| A refused connection or a timeout | connection or timeout, after exactly one attempt. |
| A request the contract does not allow | ValueError before any request: no messages, blank text, an unknown role, a late system message, a reply as the last message, or a limit out of range. |
| An option the API cannot carry | The native adapter sends num_ctx; the OpenAI-compatible adapter raises unsupported and sends nothing. |
| The reply stops at the output limit | A ChatResult with finish length, the same for both adapters. |
To check that each test catches a real defect, we removed each rule from a copy of adapters.py and ran the suite: each thinking switch, each output-limit field, the context mapping and the refusal, the seed, the 404 mapping, the status check, every check in result, decode and both parse methods, the timeout mapping, the call to validate, every rule inside validate, and the local-origin check. Every removal made at least one test fail.
Inspect the tests · test_contract.py
"""The adapter contract, run once per adapter, against recorded wire traffic. No model or network is used."""
import json
import socket
import unittest
from pathlib import Path
from unittest.mock import patch
from urllib.error import URLError
import adapters as lab
from record_wire import CASES, MESSAGES, report
HERE = Path(__file__).resolve().parent
RECORD = json.loads((HERE / 'wire-record.json').read_text())
MODELS = sorted({row['model'] for row in RECORD['rows']})
def recorded(model, adapter, case):
return next(r for r in RECORD['rows'] if (r['model'], r['adapter'], r['case']) == (model, adapter, case))
class Replay:
"""A post() that answers with one recorded or edited response and keeps what was sent."""
def __init__(self, status, body):
self.status, self.body, self.sent = status, body if isinstance(body, bytes) else body.encode(), []
def __call__(self, url, payload, headers, timeout):
self.sent.append((url, payload))
return self.status, self.body
def never(*args):
raise AssertionError('The adapter sent a request it should have refused.')
class Contract:
"""Every adapter must pass every test here. Subclasses only name the adapter and its wire shape."""
adapter_class = None
sets_context = None
def make(self, post=never):
return self.adapter_class(post=post)
def replay(self, model, case):
row = recorded(model, self.adapter_class.name, case)
post = Replay(row['status'], row['body'])
return post, row
def test_a_completed_reply_is_normalized(self):
for model in MODELS:
with self.subTest(model=model):
post, row = self.replay(model, 'complete')
got = self.make(post).complete(model, CASES['complete'][1])
self.assertEqual(post.sent[0][1], row['request'])
self.assertEqual(got.finish, 'stop')
self.assertTrue(got.text.startswith('HTTP stands for'))
self.assertGreater(got.input_tokens, 0)
self.assertGreater(got.output_tokens, 0)
def test_the_output_limit_is_a_result_not_an_error(self):
for model in MODELS:
with self.subTest(model=model):
post, _ = self.replay(model, 'output-limit')
got = self.make(post).complete(model, CASES['output-limit'][1])
self.assertEqual((got.finish, got.output_tokens), ('length', 3))
def test_a_missing_model_is_model_not_found(self):
post, _ = self.replay(MODELS[0], 'missing-model')
with self.assertRaises(lab.AdapterError) as caught:
self.make(post).complete('aiml-missing:never', CASES['missing-model'][1])
self.assertEqual(caught.exception.kind, 'model_not_found')
def test_thinking_is_turned_off_in_every_request(self):
body = self.make().body('qwen3.5:0.8b', CASES['complete'][1])
self.assertIn(body.get('think', body.get('reasoning_effort')), (False, 'none'))
post, _ = self.replay('qwen3.5:0.8b', 'thinking-left-on')
with self.assertRaises(lab.AdapterError) as caught:
self.make(post).complete('qwen3.5:0.8b', CASES['complete'][1])
self.assertEqual(caught.exception.kind, 'protocol')
def test_server_errors_and_malformed_bodies_get_a_named_kind(self):
good = json.loads(recorded(MODELS[0], self.adapter_class.name, 'complete')['body'])
cases = [(500, good, 'http'), (400, {'error': 'bad request'}, 'http'), (200, b'{not json', 'protocol'),
(200, [], 'protocol')] + [(200, body, 'protocol') for body in self.broken(good)]
for status, body, kind in cases:
with self.subTest(status=status, body=str(body)[:70]):
raw = body if isinstance(body, bytes) else json.dumps(body).encode()
with self.assertRaises(lab.AdapterError) as caught:
self.make(Replay(status, raw)).complete(MODELS[0], CASES['complete'][1])
self.assertEqual(caught.exception.kind, kind)
def test_transport_failures_are_reported_once_without_retry(self):
adapter = self.adapter_class()
for error, kind in ((URLError(ConnectionRefusedError(61, 'refused')), 'connection'),
(URLError(socket.timeout('timed out')), 'timeout'), (TimeoutError(), 'timeout')):
with self.subTest(kind=kind), patch.object(lab, 'urlopen', side_effect=error) as opened:
with self.assertRaises(lab.AdapterError) as caught:
lab.http_post(adapter.base_url + adapter.path, {}, {}, 5)
self.assertEqual((caught.exception.kind, opened.call_count), (kind, 1))
def test_requests_are_checked_before_anything_is_sent(self):
adapter = self.make()
bad = [(), (('user', ' '),), (('assistant', 'Hi'),), (('user', 'a'), ('system', 'late'), ('user', 'b')),
(('tool', 'x'), ('user', 'a')), [('user', 'not a tuple')]]
for messages in bad:
with self.subTest(messages=messages), self.assertRaises(ValueError):
adapter.complete('gemma4:e2b', lab.ChatRequest(messages))
for change in ({'max_tokens': 0}, {'max_tokens': True}, {'temperature': 3}, {'temperature': '0'},
{'seed': 1.5}, {'context_tokens': 10}):
with self.subTest(change=change), self.assertRaises(ValueError):
adapter.complete('gemma4:e2b', lab.ChatRequest(MESSAGES, **change))
for origin in ('https://example.com', 'http://example.com:11434', 'http://localhost/v1'):
with self.subTest(origin=origin), self.assertRaises(ValueError):
self.adapter_class(base_url=origin)
def test_an_option_the_api_cannot_carry_is_refused_not_dropped(self):
request = lab.ChatRequest(MESSAGES, context_tokens=4096)
if self.sets_context:
post, _ = self.replay(MODELS[0], 'complete')
self.make(post).complete(MODELS[0], request)
self.assertEqual(post.sent[0][1]['options']['num_ctx'], 4096)
else:
with self.assertRaises(lab.AdapterError) as caught:
self.make().complete(MODELS[0], request)
self.assertEqual(caught.exception.kind, 'unsupported')
class NativeContract(Contract, unittest.TestCase):
adapter_class = lab.OllamaChatAdapter
sets_context = True
@staticmethod
def broken(good):
message = good['message']
return [{**good, 'done': False}, {**good, 'message': {**message, 'role': 'user'}},
{**good, 'message': {**message, 'content': ''}}, {**good, 'message': {**message, 'tool_calls': [{}]}},
{**good, 'done_reason': 'unload'}, {**good, 'eval_count': -1}, {**good, 'prompt_eval_count': '26'}]
class CompatibleContract(Contract, unittest.TestCase):
adapter_class = lab.OpenAICompatibleAdapter
sets_context = False
@staticmethod
def broken(good):
choice, message = good['choices'][0], good['choices'][0]['message']
return [{**good, 'choices': []}, {**good, 'choices': [choice, choice]},
{**good, 'choices': [{**choice, 'message': {**message, 'content': ''}}]},
{**good, 'choices': [{**choice, 'message': {**message, 'tool_calls': [{}]}}]},
{**good, 'choices': [{**choice, 'finish_reason': 'content_filter'}]},
{**good, 'usage': None}, {**good, 'usage': {**good['usage'], 'completion_tokens': -1}}]
class RecordedComparison(unittest.TestCase):
def test_the_report_is_reproduced_from_the_recording(self):
expected = (HERE / 'contract-report.json').read_text()
self.assertEqual(json.dumps(report(RECORD), indent=2, ensure_ascii=False) + '\n', expected)
def test_both_wire_formats_give_the_same_result_for_every_recorded_case(self):
for model, cases in report(RECORD).items():
for case, outcomes in cases.items():
with self.subTest(model=model, case=case):
self.assertTrue(outcomes['agree'])
if __name__ == '__main__':
unittest.main()Deployment
The deployment unit is a library: adapters.py, imported by the application, plus the recorded traffic and the contract suite that justify each supported adapter. Choose the adapter in configuration, not in application code, and log the adapter name, model and digest with each request so a change of wire format is visible in the logs.
Before switching a deployment to another server or version, record its traffic with record_wire.py, run the contract suite, and keep the previous recording. Rolling back means pointing the configuration at the previous adapter and server, which the application can do without code changes because it depends only on the contract. Both adapters accept loopback servers only; a remote or hosted API also needs credentials, TLS and a decision about what may be sent, and none of that is part of this build.
Scaling
No load was measured. An adapter adds one dictionary translation in each direction to a request whose cost is model inference, so its own overhead is not the bottleneck worth measuring first. Scaling here means more adapters. Each new wire format adds a subclass with body, parse and broken, a recording, and nothing else; the suite grows by the number of contract methods per adapter.
When several adapters run in production, the contract also needs the pieces this build leaves out: a streaming variant, timeouts per adapter, and a recorded comparison for each model you support. Keep comparing normalized results across adapters, as contract-report.json does, so a server change that alters the reply without an error shows up as a disagreement.
Try it: remove one adapter’s thinking switch
In a copy of adapters.py, delete 'reasoning_effort': 'none' from OpenAICompatibleAdapter.body. Before you run the suite, predict which tests fail and for which adapter, and why the native adapter’s tests are unaffected.
python3 -m unittest test_contract.pyCompare your result
Only CompatibleContract tests fail: test_thinking_is_turned_off_in_every_request, because the body no longer turns thinking off, and test_a_completed_reply_is_normalized for both models, because the body the adapter sends no longer matches the recorded request. NativeContract runs the same methods with its own adapter, which still sends think: false. The suite failed even for gemma4:e2b, whose recorded reply did not change without the switch: the contract checks what is sent, not only what comes back.
Check your understanding
Read the explanation
A portable contract either carries an option to the server or refuses it. Dropping it quietly would make the two adapters behave differently for the same request.
Key takeaways
- An adapter translates field names; a contract says what the translation must preserve: the request, a normalized result and named error kinds.
- Write the contract tests once and run them against every adapter, using recorded traffic from the real server.
- The dangerous differences between APIs raise no error: a default such as thinking, or an option one API cannot carry. Make the adapter send the setting explicitly or refuse the request.
- A new server or version is supported only after its own recorded traffic passes the same suite.
Best practices
- Keep validation and result checks outside the adapters, so no adapter can be more permissive than another.
- Compare normalized results across adapters for the same recorded cases, and treat a disagreement as a failure to explain.
- Hand errors to the application as kinds, not as server text, and never retry inside an adapter.
References
Official sources checked on 2026-10-20, for Ollama 0.32.14.
- Ollama chat API: native request fields including
thinkandoptions,done_reasonand token counts. - Ollama OpenAI compatibility: supported
/v1/chat/completionsfields, includingmax_tokens,seedandreasoning_effort, and the statement that this API cannot set the context size. - llama.cpp server README and vLLM online serving: OpenAI-compatible
/v1/chat/completionsroutes in other servers. Neither was tested here. - Qwen3.5-0.8B model card and Gemma 4 E2B IT model card: Apache 2.0 licenses and each model’s documented thinking behavior.
- Python
dataclassesandunittest: frozen request and result types, and test classes shared through a mixin.
The recorded traffic, report and tests belong to this build. The documentation describes the APIs; it does not validate these results.