Get structured output and validate it with Pydantic

You validate a model reply as a Pydantic DocsAnswer. The recorded answer passes the schema with a null command field; prose and truncated JSON fail validation.

A free-form answer is difficult for an application to parse reliably. When the docs assistant feeds a CI job, a UI component or the next step of an agent, the application needs fields it can read: the answer, the page it came from, and the command to run. Here you get that answer as a typed Pydantic object, and see the three ways a reply can fail to become one.

When you need this. The docs assistant’s answer goes into a pull request comment with a link to the source page. If the model writes rollbacks.md, wraps the JSON in prose, or stops halfway through, the comment breaks, so the application has to reject that reply before anything uses it.

Recorded example. The structured responses are original model outputs for shipctl; the current docuctl schema activity keeps those examples separate from a new generation.

1. Define the answer as a Pydantic model

Download the json-and-files lab and extract its code, docs, tests and README together.

unzip json-and-files.zip
cd json-and-files
python3.12 -m venv .venv
.venv/bin/python -m pip install -r requirements-lock.txt

# Offline tests: no model calls
.venv/bin/python -m pytest -q

# Run after reading the implementation below
ollama pull gemma4:e2b
.venv/bin/python structured_answer.py

The tests exercise schema validation with fixed replies. The script requests new structured responses from Ollama.

"""Ask for a typed answer and validate it with Pydantic. Run: python structured_answer.py"""
from pathlib import Path
from typing import Literal

import ollama
from pydantic import BaseModel, Field, ValidationError

MODEL = "gemma4:e2b"
DOCS = {page.name: page.read_text() for page in sorted((Path(__file__).parent / "docs").glob("*.md"))}
Page = Literal["config.md", "deploy.md", "rollback.md", "tokens.md"]


class DocsAnswer(BaseModel):
    """What the docs assistant returns to the application."""
    answer: str = Field(min_length=1, max_length=400)
    source: Page = Field(description="The docs page the answer comes from.")
    command: str | None = Field(default=None, description="A docuctl command to run, if the answer has one.")


def prompt(question: str) -> list[dict]:
    pages = "\n\n".join(f"<page name={name!r}>\n{text}</page>" for name, text in DOCS.items())
    return [{"role": "system", "content": f"Answer from these docuctl docs pages.\n\n{pages}"},
            {"role": "user", "content": question}]


def ask(client: ollama.Client, question: str, schema: bool = True, num_predict: int = 300) -> DocsAnswer:
    """With schema=True the runtime constrains decoding to the JSON schema; Pydantic still checks the result."""
    reply = client.chat(model=MODEL, messages=prompt(question), think=False,
                        format=DocsAnswer.model_json_schema() if schema else None,
                        options={"temperature": 0, "num_predict": num_predict})
    return DocsAnswer.model_validate_json(reply.message.content)


def main() -> None:
    client = ollama.Client()
    question = "How do I roll back production to release r41?"
    print(ask(client, question).model_dump_json(indent=2))

    for label, kwargs in [("no schema in the request", {"schema": False}), ("num_predict=20", {"num_predict": 20})]:
        try:
            ask(client, question, **kwargs)
        except ValidationError as error:
            first = error.errors()[0]
            print(f"\n{label} -> {error.error_count()} validation error(s): {first['type']}: {first['msg'][:70]}")
            print("   input started:", repr(str(first["input"])[:60]))


if __name__ == "__main__":
    main()

DocsAnswer defines exactly what the application accepts from the model. Every field is a decision:

• source is a Literal of the four page names. In the JSON schema it becomes an enum, so validation rejects a page name outside this list.
• answer has length bounds. An empty answer or an essay fails validation instead of reaching the UI.
• command is optional. Not every answer has a command, so None is allowed. Step 3 shows the price of that choice.

2. Send the schema as the output format

DocsAnswer.model_json_schema() turns the model into a JSON Schema document, and format= sends it with the request. Ollama enforces that schema on the response, a feature it calls structured outputs; many hosted APIs offer the same feature under names such as response schema.

Tip: Ollama’s documentation also recommends putting the schema in the prompt and using a low temperature. The lab uses temperature 0 and lists the page names in the system message.

Constraining the output is not the same as trusting it. DocsAnswer.model_validate_json() parses the text and checks every field and constraint, and raises ValidationError when anything is wrong. Here is the recorded run:

{
  "answer": "To roll back production to release r41, you would use the command: `shipctl rollback --env production --to r41`.",
  "source": "rollback.md",
  "command": null
}

no schema in the request -> 1 validation error(s): json_invalid: Invalid JSON: expected value at line 1 column 1
   input started: 'To roll back production to release r41, you would use the fo'

num_predict=20 -> 1 validation error(s): json_invalid: Invalid JSON: EOF while parsing a string at line 2 column 66
   input started: '{\n  "answer": "To roll back production to release r41, you w'

The schema was not sent

Without format=, the model answered in friendly prose, and validation failed at the first character with json_invalid. Prompt instructions such as “reply in JSON” make JSON more likely; they do not guarantee it. For this typed answer, send the schema and validate the returned text before using it.

The reply was cut off

With num_predict 20, generation stopped in the middle of the answer string, and validation failed with EOF while parsing a string. A schema cannot finish JSON that the output limit cut off. Set the limit above your largest valid reply, read done_reason, and on length retry with a higher limit instead of repairing the JSON by hand.

3. Read what came back

The first reply validated: a DocsAnswer with source set to rollback.md. Code can now read answer.source and link to the page, with no string parsing.

Look again at command: it is null, while the original recorded answer text contains shipctl rollback --env production --to r41. The reply is valid, because the schema made command optional, and it is still not what the application wanted. If the CI job runs answer.command, it gets nothing. Validation checks the declared fields and constraints; it does not establish factual correctness. Make a field required when the application cannot work without it, and test content with an eval, as the last lesson does.

Separate constrained decoding from validation

Two separate mechanisms run here. Enforcing the schema happens inside the runtime, during generation. Runtimes typically do it with constrained decoding: at each step of the generation loop, the decoder restricts which tokens can follow the generated prefix to match the supported schema. An output limit can still stop generation before the JSON is complete. Validation happens in your process after the reply arrives, and it also checks rules a schema cannot express, such as your own Pydantic validators.

You need both. Enforcing the schema makes valid output likely, which saves retries. Application code must use the validated object and handle validation errors, including truncated responses and differences between providers.

Try it: require the command field

Change command to a required str and run the rollback question again. Check that the model now fills command. Then ask “Who has to approve a production deploy?” and see what the model puts in command when the honest answer has no single command. Decide which schema the CI job should use. The offline tests check the schema and validation without a model: .venv/bin/python -m pytest -q.

Rank documents with embeddings and cosine similarity finds the right page for a question, so that the assistant no longer needs every page in every prompt.

References

Checked October 21, 2026.

• Ollama: structured outputs
• Pydantic: JSON Schema
• Pydantic: validators and ValidationError