AI Fundamentals
Build the parts of a docs assistant: model requests, context, structured output, document ranking, validated tool calls and evaluations.
- Prerequisites
- Basic Python, a terminal and JSON. Local labs use Ollama; the hosted comparison also needs a Gemini API key.
- Duration
- about 2 hours of reading
- Cost
- Free, no sign-up
What you'll learn
- Trace tokenization, next-token generation, context limits and sampling in recorded model runs.
- Call local and hosted models, validate structured responses, and distinguish runtime errors from incorrect answers.
- Rank docs with embeddings and check a model-proposed tool call before executing it.
- Test prompt injection controls and compare answers against fixed evaluation cases.
Lessons
Module 1 · 4 lessonsHow LLMs work
Run inference with a local LLMRead
Run a language model with and without supplied documentation, and explain how request context differs from trained model weights.
Count tokens with the model's tokenizerRead
Tokenize docuctl docs, commands and IDs with Gemma's own tokenizer, then compare your counts with what Ollama actually processed, chat template included.
Trace next-token generationRead
Run a language model one token at a time, record the top candidates it scores at each step, and see how the docs in the context change every choice.
Send chat messages within the context windowRead
Build a multi-turn chat request for the docuctl docs assistant, resend the history Ollama needs, and detect when the configured context window loses input.
Module 2 · 4 lessonsCalling models from code
Control sampling, output limits and streamingRead
Set temperature and seed, cap the output with num_predict, stream the reply and time its first chunk, and handle a reply that was cut off or timed out.
Call a local model and a hosted APIRead
Send one docuctl docs question to Gemma through Ollama and to Gemini through its API, compare latency and token usage, and handle each failure.
Get structured output and validate it with PydanticRead
Make the docuctl docs assistant return a typed answer: send a Pydantic JSON schema as the output format, validate every reply, and see its limits.
Rank documents with embeddingsRead
Embed the docuctl docs pages and questions with a local model, rank pages by cosine similarity, and see the two ways similarity search misleads.
Module 3 · 3 lessonsTools, safety and evals
Validate a model-proposed tool callRead
Send a read tool schema to a local model, inspect one proposed call, and reject unknown names or invalid page arguments before the tool function runs.
Defend a tool-using assistant against prompt injectionRead
Test whether document text can trigger an unauthorized tool call, and compare prompt warnings with application permission checks.
Write your first eval for an LLM appRead
Write fixed cases for the docuctl docs assistant, grade every answer in code, run three trials per case, and use the results to judge two attempted fixes.