aiml
SearchMy learning
Menu

Lesson 2 of 7Module 1

Read live events and manage live sessions

Fold ADK live events into captions, handle interruptions, inspect saved session history, and test reconnect behavior with and without a handle.

Prerequisites

A live agent running through a queue and event loop.

By the end, you can

  • Build captions from partial and finished transcription events and react to interrupted and turn_complete.
  • Tell what run_live saves to the ADK session from what it only streams.
  • Keep one conversation across connections with session resumption, replayed history and session state.
Runtime
Python 3.12+ and a Gemini Live API key
Dependencies
google-adk 2.12.0

Lab code, tests and fixtures (ZIP, 145 KB)Quiz check your understanding

Progress is saved in this browser.

A voice UI needs more than audio: it must show captions, track an unfinished turn and stop playback when the caller interrupts. Captions turns ADK events into that display state. An interrupted event tells the client to discard pending playback; reconnecting requires separate decisions about retained conversation history.

The saved connections on gemini-2.5-flash-native-audio-preview-12-2025 include a caller talking over an answer, followed by two reconnects on the same ADK session. ADK replays saved history to each new Live API session. The examples show why carrying history forward does not guarantee that the next answer recalls it correctly.

When you need this. A docs client needs captions while a caller speaks and must stop playback when the caller interrupts. The conversation can outlast a network connection, so keep caption state separate from retained ADK session history.

Two different things are called a session. The ADK Session lives in the session service and holds saved events and state. A Live API session holds conversational context on Google’s side; a resumption handle can preserve that context across connections. Each run_live() creates a Live API session from the ADK one. If the ADK session already has events and there is no resumption handle, ADK sets history_config.initial_history_in_client_content=True and sends the history first; the reconnection example below shows that history entry.

Event feeds captions and SessionService separately
data flowreturnartifact hand-offEventpartial and finishedCaptions.applylab caption reducerCaptionslines / live / stateInMemorySessionServiceappend finished eventsSessionevents and statetranscriptcaption fieldsappend_eventfinished eventsdata flowreturnartifact hand-offEventpartial and finishedCaptions.applylab caption reducerCaptionslines / live / stateInMemorySessionServiceappend finished eventsSessionevents and statetranscriptcaption fieldsappend_event3

3. finished events

Captions.apply folds transcript fields into the lab Captions object: lines, live text and microphone state. ADK persists finished events through InMemorySessionService; Session.events and state are separate from those caption fields, and raw audio bytes are streamed without being stored in the session. Recorded model: gemini-2.5-flash-native-audio-preview-12-2025.

Download the complete lab ZIP, extract it and open a terminal in adk-live-events/. Create the environment below and set GOOGLE_API_KEY in that terminal before running a live command. live_harness.py and docs/ are the same as in the first lesson. interrupt.wav and followup.wav are short questions synthesized locally.

Prepare the local labTerminal · bash
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-lock.txt

1. Fold events into captions

captions.pySource · pythonDownload
"""Fold a live event stream into captions and UI state, and keep one ADK session across connections."""
import asyncio
from dataclasses import dataclass, field
from pathlib import Path

from google.adk.agents import Agent
from google.adk.agents.run_config import RunConfig
from google.adk.events import Event, EventActions
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types

from live_harness import MODEL

DOCS = (Path(__file__).parent / "docs/live-streaming.md").read_text()
APP = "voice_docs"

RUN_CONFIG = RunConfig(
    response_modalities=["AUDIO"],
    # ADK caches each resumption handle and reconnects with it when the ~10 minute connection closes.
    session_resumption=types.SessionResumptionConfig(),
    # Retain recent turns and discard older ones past the trigger; this is not summarization.
    context_window_compression=types.ContextWindowCompressionConfig(
        trigger_tokens=100_000, sliding_window=types.SlidingWindow(target_tokens=80_000)),
)


@dataclass
class Captions:
    """What a voice UI shows: finished lines per speaker, the line in progress and the mic state."""
    lines: list = field(default_factory=list)
    live: str = ""
    state: str = "listening"
    audio_bytes: int = 0

    def apply(self, row: dict) -> None:
        if row.get("interrupted"):
            self.state, self.live = "interrupted", ""  # stop playback and drop the unfinished line
        for side, who in (("input_transcription", "user"), ("output_transcription", "agent")):
            if side in row:
                if row.get("partial"):
                    self.live += row[side]["text"]  # a fragment; the finished event repeats it merged
                elif row[side]["text"]:
                    self.lines.append((who, row[side]["text"].strip()))
                    self.live = ""
        if row.get("audio_bytes"):
            self.state, self.audio_bytes = "speaking", self.audio_bytes + row["audio_bytes"]
        if row.get("turn_complete"):
            self.state, self.live = "listening", ""


def build_agent(model=MODEL) -> Agent:
    # {asked?} renders from session state once per Live connection; it is empty on the first one.
    return Agent(name="docs_helper", model=model, instruction=(
        "You are the docs assistant. Answer from this page in short spoken sentences.\n"
        "Questions the caller already asked in this call: {asked?}\n\n" + DOCS))


def make_runner(agent) -> Runner:
    return Runner(app_name=APP, agent=agent, session_service=InMemorySessionService())


async def open_session(runner, session_id: str):
    """Get or create: a reconnecting client reuses its ADK session and therefore its history."""
    session = await runner.session_service.get_session(app_name=APP, user_id="dev", session_id=session_id)
    return session or await runner.session_service.create_session(app_name=APP, user_id="dev", session_id=session_id)


def stored(session) -> list:
    """What run_live persisted: one row per saved event, with the kinds of data it carries."""
    rows = []
    for event in session.events:
        parts = (event.content.parts or []) if event.content else []
        kinds = sorted({k for p in parts for k in ("text", "inline_data", "function_call") if getattr(p, k)})
        kinds += [k for k in ("input_transcription", "output_transcription", "usage_metadata") if getattr(event, k)]
        rows.append({"author": event.author, "kinds": kinds,
                     "text": (event.input_transcription or event.output_transcription or types.Transcription()).text})
    return rows


async def remember_questions(runner, session) -> list:
    """Copy the caller's saved questions into state, so the next connection's instruction has them."""
    asked = [e.input_transcription.text.strip() if e.input_transcription else e.content.parts[0].text
             for e in session.events if e.author == "user" and (e.input_transcription or e.content)]
    await runner.session_service.append_event(session, Event(
        author="user", actions=EventActions(state_delta={"asked": "; ".join(asked)})))
    return asked

Partial or finished. Transcription arrives twice. Events with partial=True carry fragments, a few characters each, for a live typing effect. Then one event with partial=False carries the merged text of the segment. Captions.apply() appends fragments to live and replaces them with the finished line, so each sentence is shown once.

Interrupted, then complete. When the caller talks over the agent, an event with interrupted=True arrives. The client must stop playback right away; audio already buffered keeps playing over the caller otherwise. In the recording, interrupted is event 74 of 168, and turn_complete follows two events later. Both turns end with turn_complete, 2 in all.

Run config per connection. RUN_CONFIG turns on session_resumption, so ADK caches the resumption handles the server sends (3 updates in the first connection) and reconnects with the latest one when a connection closes. The configuration also requests context_window_compression at trigger_tokens. The short recording does not show that threshold being reached or compression taking place. For a longer session, the configured sliding window removes older turns and retains recent conversation context; it does not generate a summary. System instructions stay in context. Check what the application must retain when earlier turns leave the model context.

Every fragment joined. Concatenating every output_transcription.text repeats each sentence, because the finished event holds the fragments merged. Use the finished events, or the fragments only for the line in progress. Check: test_joining_every_fragment_doubles_the_transcript.

2. Record a call with a barge-in and two reconnections

Terminal commandsTerminal · bash
python record.py

barge_in() sends a long question as text and starts streaming interrupt.wav as soon as the first audio arrives. Then the script opens two more run_live() calls on the same session, review-call, each with a new queue. Before the third call, remember_questions() copies the caller’s saved questions into session state.

Read record.py
record.pySource · pythonDownload
"""Record: a barge-in, the saved session, a second connection on the same session, a reused queue."""
import asyncio
import json
from pathlib import Path

from google.adk.agents import LiveRequestQueue

from captions import RUN_CONFIG, Captions, build_agent, make_runner, open_session, remember_questions, stored
from live_harness import RecordingGemini, environment, MODEL, read_pcm, say_audio, say_text, spoken, summarize

HERE = Path(__file__).parent
LONG = "Read me every limit on the page, one at a time, with a sentence about each."


async def barge_in(runner, session_id, queue):
    """Ask a long question, then talk over the answer once its audio starts."""
    rows, interrupted = [], False
    await say_text(LONG)(queue)
    async for event in runner.run_live(user_id="dev", session_id=session_id,
                                       live_request_queue=queue, run_config=RUN_CONFIG):
        row = summarize(event)
        rows.append(row)
        if row.get("audio_bytes") and not interrupted:
            interrupted = True
            await say_audio(read_pcm(HERE / "interrupt.wav"))(queue)
        if sum(1 for r in rows if r.get("turn_complete")) == 2:
            break
    queue.close()
    return rows


async def one_turn(runner, session_id, queue, turn, timeout=60):
    rows = []

    async def loop():
        await turn(queue)
        async for event in runner.run_live(user_id="dev", session_id=session_id,
                                           live_request_queue=queue, run_config=RUN_CONFIG):
            rows.append(summarize(event))
            if event.turn_complete:
                break
    try:
        await asyncio.wait_for(loop(), timeout)
        outcome = "turn complete"
    except asyncio.TimeoutError:
        outcome = f"no event for {timeout} s"
    queue.close()
    return rows, outcome


async def main():
    model = RecordingGemini(model=MODEL, connections=[])
    runner = make_runner(build_agent(model))
    session = await open_session(runner, "review-call")
    first = await asyncio.wait_for(barge_in(runner, session.id, LiveRequestQueue()), 120)
    captions = Captions()
    for row in first:
        captions.apply(row)
    saved_after_first = stored(await open_session(runner, "review-call"))
    second, _ = await one_turn(runner, session.id, LiveRequestQueue(), say_audio(read_pcm(HERE / "followup.wav")))
    asked = await remember_questions(runner, await open_session(runner, "review-call"))
    third, _ = await one_turn(runner, session.id, LiveRequestQueue(), say_text("What did I ask you a moment ago?"))
    closed = LiveRequestQueue()
    closed.close()
    reused, outcome = await one_turn(runner, session.id, closed, say_text("Are you still there?"), timeout=20)
    print(f"[first connection] interrupted={any(r.get('interrupted') for r in first)} "
          f"resumption updates={sum(1 for r in first if 'resumption_update' in r)}")
    for who, line in captions.lines:
        print(f"  {who}: {line}")
    print(f"[saved] {len(saved_after_first)} events, audio parts saved: "
          f"{sum('inline_data' in r['kinds'] for r in saved_after_first)}")
    print(f"[second connection] history sent: {model.connections[1]["traffic"][0]["turns"]}")
    print(f"  user: {spoken(second, 'input_transcription')}\n  agent: {spoken(second)}")
    print(f"[third connection, state asked={asked}]\n  agent: {spoken(third)}")
    print(f"[reused closed queue] {len(reused)} events, {outcome}")
    record = {"environment": environment(), "first": first, "captions": captions.lines,
              "saved_after_first": saved_after_first, "second": second, "third": third, "asked": asked,
              "reused_queue": {"events": reused, "outcome": outcome}, "connections": model.connections}
    (HERE / "events-run.json").write_text(json.dumps(record, indent=1) + "\n")


if __name__ == "__main__":
    asyncio.run(main())

3. Read what the session kept

Here is what run_live saved from the first connection, one row per event:

events-run.json: saved_after_firstRecorded output · jsonDownload
[
 {
  "author": "user",
  "kinds": [
   "text"
  ],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [
   "output_transcription"
  ],
  "text": "A Live API connection closes after about 10 minutes."
 },
 {
  "author": "docs_helper",
  "kinds": [
   "text"
  ],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [
   "usage_metadata"
  ],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [],
  "text": null
 },
 {
  "author": "user",
  "kinds": [
   "input_transcription"
  ],
  "text": " Wait, just tell me the input sample rate."
 },
 {
  "author": "docs_helper",
  "kinds": [
   "output_transcription"
  ],
  "text": "Microphone audio uses a 16 kHz sample rate."
 },
 {
  "author": "docs_helper",
  "kinds": [
   "usage_metadata"
  ],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [
   "text"
  ],
  "text": null
 },
 {
  "author": "docs_helper",
  "kinds": [],
  "text": null
 }
]

The connection streamed 168 events, and the session kept 13: the typed question, the finished transcripts of both speakers, thought text, usage and control events. No row holds audio. Raw inline_data and partial fragments are yielded for playback and display only; set save_live_blob=True to store audio in the artifact service, at about 1.92 MB per minute per session for 16 kHz input.

Audio expected in the session. Audio is streamed and not saved: no saved event has an inline_data part. Turn on save_live_blob only for the calls you must keep. Check: test_audio_is_streamed_but_not_saved.

4. Reconnect on the same session

When the second run_live() opened a new Live API session, ADK rebuilt the conversation from the saved events and sent it with send_history before any new input:

events-run.json: the history ADK sent on the second connectionRecorded output · jsonDownload
{
 "sent": "history",
 "turns": [
  {
   "role": "user",
   "text": "Read me every limit on the page, one at a time, with a sentence about each."
  },
  {
   "role": "model",
   "text": "A Live API connection closes after about 10 minutes."
  },
  {
   "role": "user",
   "text": " Wait, just tell me the input sample rate."
  },
  {
   "role": "model",
   "text": "Microphone audio uses a 16 kHz sample rate."
  }
 ]
}

All 4 turns arrived, including the caller’s spoken question as text. Then the caller asked what they had asked a moment ago, and the agent answered You haven't asked any questions yet.. That answer is wrong. The history was in the model’s context, and the model still did not use it. Sending history is ADK’s job, and ADK did it; recalling it is the model’s, and here it failed.

The next recording makes the question history explicit. remember_questions() writes the caller’s questions into session state as asked, and the instruction renders {asked?} once per connection. On the second connection the key did not exist yet, so the line was empty. On the third, the agent answered You asked, "Wait, just tell me the input sample rate.". This made those questions easier to use in the recorded call; it does not force the model to recall them. Keep facts needed by application logic in state and check the resulting answer separately.

Replayed history without recall. The second connection received all 4 turns and the agent still answered You haven't asked any questions yet.. Keep facts in session state and render them into the instruction. Check: test_replayed_history_is_not_recall.

A closed queue reused. A LiveRequestQueue that was closed ends the next run_live() at once: the recording got 0 events and no answer. Create a new queue for every run_live() call. Check: test_a_closed_queue_ends_run_live_with_no_events.

A drop without a resumption handle. ConnectionClosedError propagates out of run_live() when no handle was cached. Enable session_resumption when reconnection is needed; ADK can reconnect once it has cached a handle. Check: test_a_drop_without_a_handle_reaches_the_caller.

5. Test it without the network

Inspect the saved session output
Recorded call on the Live APIRecorded output (excerpt) · textFull output
[first connection] interrupted=True resumption updates=3
  agent: A Live API connection closes after about 10 minutes.
  user: Wait, just tell me the input sample rate.
  agent: Microphone audio uses a 16 kHz sample rate.
[saved] 13 events, audio parts saved: 0
[second connection] history sent: [{'role': 'user', 'text': 'Read me every limit on the page, one at a time, with a sentence about each.'}, {'role': 'model', 'text': 'A Live API connection closes after about 10 minutes.'}, {'role': 'user', 'text': ' Wait, just tell me the input sample rate.'}, {'role': 'model', 'text': 'Microphone audio uses a 16 kHz sample rate.'}]
  user: What did I ask you a moment ago?
  agent: You haven't asked any questions yet.
[third connection, state asked=['Read me every limit on the page, one at a time, with a sentence about each.', 'Wait, just tell me the input sample rate.', 'What did I ask you a moment ago?']]
  agent: You asked, "Wait, just tell me the input sample rate."
[reused closed queue] 0 events, turn complete
Terminal commandsTerminal · bash
python -m unittest -v test_captions.py
Recorded test outputRecorded output · textDownload
test_a_closed_queue_ends_run_live_with_no_events (test_captions.LiveEvents.test_a_closed_queue_ends_run_live_with_no_events) ... ok
test_a_drop_with_a_cached_handle_reconnects_without_history (test_captions.LiveEvents.test_a_drop_with_a_cached_handle_reconnects_without_history) ... ok
test_a_drop_without_a_handle_reaches_the_caller (test_captions.LiveEvents.test_a_drop_without_a_handle_reaches_the_caller) ... ok
test_a_new_connection_replays_the_saved_history (test_captions.LiveEvents.test_a_new_connection_replays_the_saved_history) ... ok
test_audio_is_streamed_but_not_saved (test_captions.LiveEvents.test_audio_is_streamed_but_not_saved) ... ok
test_barge_in_sets_interrupted_before_the_turn_completes (test_captions.LiveEvents.test_barge_in_sets_interrupted_before_the_turn_completes) ... ok
test_captions_keep_each_finished_line_once (test_captions.LiveEvents.test_captions_keep_each_finished_line_once) ... ok
test_joining_every_fragment_doubles_the_transcript (test_captions.LiveEvents.test_joining_every_fragment_doubles_the_transcript) ... ok
test_replayed_history_is_not_recall (test_captions.LiveEvents.test_replayed_history_is_not_recall) ... ok

----------------------------------------------------------------------
Ran 9 tests in 3.178s

OK

All 9 tests pass. Two of them change the recording on purpose: they insert a dropped connection (ConnectionClosedError) into the replayed traffic, once after the server sent a resumption handle and once with every handle removed. ADK’s own reconnect code handles both, so the tests show what it does with and without a handle.

Read the test source
test_captions.pySource · pythonDownload
"""Replay the recorded live session through ADK; inject a dropped connection; no network."""
import asyncio
import copy
import json
import logging
import unittest
from pathlib import Path

from google.adk.agents import LiveRequestQueue
from google.adk.agents.run_config import RunConfig
from websockets.exceptions import ConnectionClosedError

import live_harness
from captions import RUN_CONFIG, Captions, build_agent, make_runner, open_session
from live_harness import ReplayLive, read_pcm, say_audio, say_text, spoken, summarize
from record import barge_in, one_turn

HERE = Path(__file__).parent
RECORD = json.loads((HERE / "events-run.json").read_text())
live_harness.PACE = 0
logging.disable(logging.CRITICAL)  # ADK logs the injected failures; the assertions check them


def replay(*indexes, drop_after_handle=False, strip_handles=False):
    connections = [copy.deepcopy(RECORD["connections"][i]) for i in indexes]
    traffic = connections[0]["traffic"]
    if strip_handles:
        traffic[:] = [e for e in traffic if "live_session_resumption_update" not in e.get("received", {})]
    if drop_after_handle or strip_handles:
        mark = "live_session_resumption_update" if drop_after_handle else "output_transcription"
        at = next(i for i, e in enumerate(traffic) if e.get("received", {}).get(mark, {}).get(
            "new_handle" if drop_after_handle else "text"))
        traffic.insert(at + 1, {"raise": ConnectionClosedError(None, None)})
    return ReplayLive(model="replay", connections=connections, opened=[])


class LiveEvents(unittest.TestCase):
    def test_captions_keep_each_finished_line_once(self):
        captions = Captions()
        for row in RECORD["first"]:
            captions.apply(row)
        self.assertEqual([list(line) for line in captions.lines], RECORD["captions"])
        self.assertEqual(captions.state, "listening")

    def test_joining_every_fragment_doubles_the_transcript(self):
        naive = "".join(r["output_transcription"]["text"] for r in RECORD["first"] if "output_transcription" in r)
        first_line = RECORD["captions"][0][1]
        self.assertEqual(naive.count(first_line), 2)
        self.assertEqual(spoken(RECORD["first"]).count(first_line), 1)

    def test_barge_in_sets_interrupted_before_the_turn_completes(self):
        flags = [("interrupted" if r.get("interrupted") else "turn_complete")
                 for r in RECORD["first"] if r.get("interrupted") or r.get("turn_complete")]
        self.assertEqual(flags[:2], ["interrupted", "turn_complete"])

    def test_audio_is_streamed_but_not_saved(self):
        self.assertGreater(sum(r.get("audio_bytes", 0) for r in RECORD["first"]), 0)
        self.assertFalse(any("inline_data" in r["kinds"] for r in RECORD["saved_after_first"]))

    def test_a_new_connection_replays_the_saved_history(self):
        model = replay(0, 1)
        runner = make_runner(build_agent(model))

        async def run():
            session = await open_session(runner, "review-call")
            await barge_in(runner, session.id, LiveRequestQueue())
            await one_turn(runner, session.id, LiveRequestQueue(), say_audio(read_pcm(HERE / "followup.wav")))
        asyncio.run(run())
        self.assertEqual(model.opened[1][1].received[0], ("history", 4))

    def test_replayed_history_is_not_recall(self):
        self.assertNotIn("sample rate", spoken(RECORD["second"]))
        self.assertIn("sample rate", spoken(RECORD["third"]))

    def test_a_closed_queue_ends_run_live_with_no_events(self):
        runner = make_runner(build_agent(replay(0)))

        async def run():
            session = await open_session(runner, "review-call")
            queue = LiveRequestQueue()
            queue.close()
            return [summarize(e) async for e in runner.run_live(
                user_id="dev", session_id=session.id, live_request_queue=queue, run_config=RUN_CONFIG)]
        self.assertEqual(asyncio.run(run()), [])

    def test_a_drop_with_a_cached_handle_reconnects_without_history(self):
        model = replay(0, 1, drop_after_handle=True)
        runner = make_runner(build_agent(model))

        async def run():
            session = await open_session(runner, "review-call")
            await asyncio.wait_for(barge_in(runner, session.id, LiveRequestQueue()), 3)
        try:
            asyncio.run(run())
        except asyncio.TimeoutError:
            pass  # the second recording expects other input; reconnecting is what is under test
        request, connection = model.opened[1]
        self.assertTrue(request.live_connect_config.session_resumption.handle)
        self.assertNotIn("history", [kind for kind, _ in connection.received])

    def test_a_drop_without_a_handle_reaches_the_caller(self):
        runner = make_runner(build_agent(replay(0, strip_handles=True)))

        async def run():
            session = await open_session(runner, "review-call")
            await barge_in(runner, session.id, LiveRequestQueue())
        with self.assertRaises(ConnectionClosedError):
            asyncio.run(run())


if __name__ == "__main__":
    unittest.main()

How it works

The reconnection behavior depends on which session survives and whether a resumption handle is available.

Within one run_live(), the server sends session_resumption_update messages with a new handle, and ADK keeps the latest in the invocation context. When the connection closes with a handle cached and the queue still open, ADK reconnects with that handle, at most 5 failed attempts in a row, and sends no history, because the server still holds the context. The reconnect test checks both: the second connect carries the handle, and nothing is sent as history. Without a handle, the drop goes straight to your code.

ADK manages only its own connection to the Live API. The connection between your client and your server, and its reconnects, are your code; Build a custom WebSocket server writes that part.

What’s next

Check your understanding

A caption client appends every partial fragment and then appends the finished transcript too. Why do sentences repeat?

Choose one answer, then check your reasoning.

Read the explanation

Use partial fragments for the live line, then replace that line with the finished transcription instead of appending both.

References

Official sources checked October 21, 2026, against google-adk 2.12.0.

Written by Arjun Prabhulal.

Module 1

Stream media and manage live sessions