Provision Terraform infrastructure and deploy with agents-cli

Separate cloud infrastructure provisioning from application rollout, choose a deployment target and inspect a preflight dry-run. The local lab checks identity flags and Terraform ownership rules; actual provisioning and rollout require the configured cloud environment and a separate smoke test.

Problem

When you need this. Deploying an agent directly from your laptop with default settings works for a quick demo, but production deployments need least-privilege service accounts, Secret Manager bindings, telemetry storage buckets, and predictable ownership between Terraform and CLI code rollouts. If you run an imperative agents-cli deploy first and run terraform apply later, Terraform collides with the already-created cloud service. In this lesson, you will inspect the Terraform modules generated by agents-cli scaffold enhance, run Agent Runtime cloud preflight checks and inspect the Cloud Run command preview with agents-cli deploy --dry-run, and verify target-specific flag and state rules.

Concepts

agents-cli divides production cloud ownership into two layers: Terraform owns long-lived infrastructure (IAM service accounts, Secret Manager permissions, telemetry buckets, BigQuery tables, and the initial service shell), while agents-cli deploy owns application container rollouts.

Placeholder image + ignore_changes

In service.tf, Terraform creates the Cloud Run service using us-docker.pkg.dev/cloudrun/container/hello and sets lifecycle { ignore_changes = [template[0].containers[0].image] } so Terraform leaves the rollout image to the deployment step.

Agent Runtime (agent_runtime)

Deploys the agent container as a managed Vertex AI ReasoningEngine resource, supporting either --agent-identity (per-agent SPIFFE principal) or a custom --service-account.

Preflight dry-run (agents-cli deploy --dry-run)

On Agent Runtime, checks cloud APIs, caller permissions and engine state before printing the plan. On Cloud Run, prints the proposed gcloud command and skips the existing-service lookup; it does not verify deployed permissions. Neither path creates the deployment.

Operation lock (.agents-cli-operation.json)

A local state file written while a deployment is in progress so agents-cli status can poll the rollout and another deploy from that working tree can be refused. This local file does not coordinate deployments from other checkouts or machines.

Architecture

Terraform provisions persistent infrastructure; the deployment command rolls out application code to the selected target. The manifest, identity flags and deployment metadata connect those operations. The downloaded validator checks the illustrated flag and ownership rules without provisioning a resource.

Working code

Download the complete lesson lab, extract it, and open its agents-cli-deploy-infra directory. It includes the source, tests and fixtures. With Python 3.12+ and uv installed, prepare the local environment:

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-lock.txt
LAB_DIR="$PWD"

Replace the project path below with the directory created in Lesson 1. Use the model credentials configured there for model activities; use your cloud credentials for cloud activities.

export PROJECT_DIR="/absolute/path/to/docuctl-docs"
cd "$PROJECT_DIR"

The downloaded service.tf is an excerpt for inspection, not a complete Terraform root. Use the full deployment/terraform/single-project directory generated in Upgrade and extend an Agents CLI project for provisioning. You need Terraform, the Google Cloud CLI, a billing-enabled project and credentials with the documented deployment permissions. The commands below can create billable resources; the offline validator does not provision them.

Choose a deployment target and identity mode

agents-cli deploy supports three Google Cloud compute targets (--deployment-target or -d):

Notice that on agent_runtime, GOOGLE_CLOUD_PROJECT is injected automatically by the platform and is reserved: agents-cli deploy filters GOOGLE_CLOUD_PROJECT out of .env before sending deployment_spec.env so the API does not reject the deployment with FAILED_PRECONDITION. GOOGLE_CLOUD_LOCATION (global) is not reserved and is preserved so your model calls can target global even when the runtime engine lives in us-central1.

Inspect the scaffolded Terraform split

When you ran agents-cli scaffold enhance . --deployment-target cloud_run --cicd-runner github_actions --bq-analytics in Lesson 2, the CLI generated two Terraform roots:

• deployment/terraform/single-project/: provisions a single GCP project (APIs, the docuctl-docs-app service account in iam.tf, GCS logs bucket and BigQuery dataset/views in telemetry.tf, and the Cloud Run service in service.tf).
• deployment/terraform/cicd/: provisions Workload Identity Federation (github.tf), Cloud Build triggers, and staging/production environments.

Inspect deployment/terraform/single-project/service.tf:

resource "google_cloud_run_v2_service" "app" {
  name                = var.project_name
  location            = var.region
  project             = var.project_id
  deletion_protection = false
  ingress             = "INGRESS_TRAFFIC_ALL"
  labels = {
    "created-by" = "agents-cli"
  }

  template {
    containers {
      image = "us-docker.pkg.dev/cloudrun/container/hello"

      env {
        name  = "APP_URL"
        value = "https://${var.project_name}-${data.google_project.project.number}.${var.region}.run.app"
      }

      env {
        name  = "GOOGLE_CLOUD_PROJECT"
        value = var.project_id
      }

      env {
        name  = "GOOGLE_CLOUD_LOCATION"
        value = "global"
      }

      env {
        name  = "GOOGLE_GENAI_USE_ENTERPRISE"
        value = "True"
      }

      resources {
        limits = {
          cpu    = "1"
          memory = "4Gi"
        }
      }

      env {
        name  = "LOGS_BUCKET_NAME"
        value = google_storage_bucket.logs_data_bucket.name
      }

      env {
        name  = "OTEL_SERVICE_NAME"
        value = "docuctl-docs"
      }

      env {
        name  = "OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"
        value = "NO_CONTENT"
      }

      env {
        name  = "ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS"
        value = "false"
      }

      env {
        name  = "OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK"
        value = "upload"
      }

      env {
        name  = "OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH"
        value = "gs://${google_storage_bucket.logs_data_bucket.name}/completions"
      }
    }

    service_account                  = google_service_account.app_sa.email
    max_instance_request_concurrency = 8

    scaling {
      min_instance_count = 1
      max_instance_count = 10
    }

    session_affinity = true
  }

  lifecycle {
    ignore_changes = [
      template[0].containers[0].image,
    ]
  }
}

Look at two critical details in service.tf:

1. Placeholder image + ignore_changes: Terraform initializes google_cloud_run_v2_service.app with image = "us-docker.pkg.dev/cloudrun/container/hello" and sets lifecycle { ignore_changes = [template[0].containers[0].image] }. This lets terraform apply create the service, bind the google_service_account.app_sa (docuctl-docs-app) service account, configure max_instance_request_concurrency = 8 and session_affinity = true, and wire the telemetry bucket—while subsequent agents-cli deploy runs update the container image without Terraform reverting it on the next infrastructure plan.
2. Memory and concurrency sizing: Worker memory depends on the imported SDKs, concurrency and request size. Measure it under your workload rather than treating the template settings as a capacity estimate. The supplied configuration pairs memory = "4Gi" with max_instance_request_concurrency = 8 (--concurrency 8).

From the generated project, authenticate and review the infrastructure plan before applying it. Supply the variables requested by the generated root; its variable definitions are the authority for your template version.

export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
gcloud auth application-default login
gcloud config set project "$GOOGLE_CLOUD_PROJECT"
terraform -chdir=deployment/terraform/single-project init
terraform -chdir=deployment/terraform/single-project plan -out=reviewed.tfplan
# Only after reviewing the plan:
terraform -chdir=deployment/terraform/single-project apply reviewed.tfplan

Validate rollouts with deploy dry-run

Before creating or updating cloud resources, pass --dry-run to agents-cli deploy. On Agent Runtime, --dry-run connects to your GCP project (your own project) to verify that aiplatform.googleapis.com is enabled, checks whether an engine named docuctl-docs already exists, verifies your caller IAM permissions (aiplatform.reasoningEngines.create, aiplatform.reasoningEngines.update, resourcemanager.projects.getIamPolicy, resourcemanager.projects.setIamPolicy), and prints the resolved machine shape:

# Validate an Agent Runtime deployment without creating resources
agents-cli deploy \
  --deployment-target agent_runtime \
  --project "$GOOGLE_CLOUD_PROJECT" \
  --region us-central1 \
  --cpu 2 \
  --memory 4Gi \
  --concurrency 8 \
  --min-instances 1 \
  --max-instances 6 \
  --dry-run

# Validate a Cloud Run deployment with Secret Manager bindings
agents-cli deploy \
  --deployment-target cloud_run \
  --project "$GOOGLE_CLOUD_PROJECT" \
  --region us-central1 \
  --cpu 1 \
  --memory 4Gi \
  --concurrency 8 \
  --min-instances 1 \
  --max-instances 10 \
  --secrets DOCUCTL_TOKEN=docuctl-ci-token:latest \
  --dry-run

A successful dry run is not a deployed service or a completed Gemini request. After provisioning and reviewing the preflight output, run the matching deployment command without --dry-run to perform the rollout. When you run agents-cli deploy without --dry-run, the CLI claims a lock in .agents-cli-operation.json while the rollout is in flight (which you can poll at any time with agents-cli status) and writes deployment_metadata.json when the deployment completes.

Here is deploy_validator.py, a local teaching helper that validates target-only CLI flags and checks Terraform-vs-CLI rollout ordering:

"""Deployment flag validation, Terraform ownership checks, and metadata builder for agents-cli deploy."""

from __future__ import annotations

from typing import Any

TARGET_ONLY_FLAGS: dict[str, tuple[tuple[str, ...], str]] = {
    "--cpu": (
        ("agent_runtime", "cloud_run"),
        "On GKE, configure sizing via Terraform and the HorizontalPodAutoscaler under deployment/terraform/.",
    ),
    "--memory": (
        ("agent_runtime", "cloud_run"),
        "On GKE, configure sizing via Terraform and the HorizontalPodAutoscaler under deployment/terraform/.",
    ),
    "--concurrency": (
        ("agent_runtime", "cloud_run"),
        "On GKE, configure sizing via Terraform and the HorizontalPodAutoscaler under deployment/terraform/.",
    ),
    "--secrets": (
        ("agent_runtime", "cloud_run"),
        "On GKE, mount secrets via Kubernetes Secrets or the Secret Manager CSI driver.",
    ),
    "--update-only": (
        ("agent_runtime",),
        "",
    ),
    "--image": (
        ("cloud_run", "gke"),
        "Agent Runtime builds from the project Dockerfile and does not support prebuilt images.",
    ),
}

AGENT_RUNTIME_RESERVED_ENV = frozenset({"GOOGLE_CLOUD_PROJECT"})


def validate_deploy_flags(
    deployment_target: str, provided_flags: dict[str, Any]
) -> dict[str, Any]:
    """Validate target-specific flags for `agents-cli deploy` (mirrors cmd_deploy.py)."""
    if deployment_target not in ("agent_runtime", "cloud_run", "gke"):
        raise ValueError(f"Unsupported deployment target: {deployment_target!r}")

    for flag, value in provided_flags.items():
        if value is None or value is False:
            continue
        if flag in TARGET_ONLY_FLAGS:
            allowed_targets, hint = TARGET_ONLY_FLAGS[flag]
            if deployment_target not in allowed_targets:
                target_names = " and ".join(
                    {"agent_runtime": "Agent Runtime", "cloud_run": "Cloud Run", "gke": "GKE"}[t]
                    for t in allowed_targets
                )
                msg = (
                    f"{flag} is only supported for {target_names} deployments "
                    f"(current target: {deployment_target})."
                )
                if hint:
                    msg += f"\n  {hint}"
                raise ValueError(msg)

    return {
        "deployment_target": deployment_target,
        "validated_flags": {k: v for k, v in provided_flags.items() if v is not None},
    }


def check_terraform_rollout_order(
    *,
    imperative_deploy_ran_first: bool,
    terraform_state_has_service: bool,
    deployment_target: str,
    update_only: bool = False,
) -> dict[str, str]:
    """Verify that Terraform and imperative `agents-cli deploy` do not collide on resource ownership."""
    if imperative_deploy_ran_first and not terraform_state_has_service:
        raise ValueError(
            "Resource ownership conflict: imperative 'agents-cli deploy' created the service "
            "before 'terraform apply'. Running 'terraform apply' now fails with 409 Already Exists "
            "(or creates a duplicate Agent Runtime engine). Provision Terraform first or import the resource."
        )
    if deployment_target == "agent_runtime" and terraform_state_has_service and not update_only:
        raise ValueError(
            "Agent Runtime engine is managed by Terraform; pass --update-only to 'agents-cli deploy' "
            "so a missing engine ID fails fast instead of creating a duplicate ReasoningEngine."
        )
    return {
        "status": "aligned",
        "deployment_target": deployment_target,
        "image_lifecycle": "ignore_changes=[template[0].containers[0].image]",
    }

Inspect the supplied earlier preflight examples and metadata structure in recorded-deploy-infra.json:

{
  "cli_version": "1.11.0",
  "project": "vertex-ai-experminent",
  "project_number": "913703049174",
  "region": "us-central1",
  "agent_runtime_dry_run": {
    "command": "agents-cli deploy --deployment-target agent_runtime --project vertex-ai-experminent --region us-central1 --cpu 2 --memory 4Gi --concurrency 8 --min-instances 1 --max-instances 6 --dry-run",
    "exit_code": 0,
    "preflight_checks": {
      "apis_enabled": ["aiplatform.googleapis.com"],
      "target_engine": "creates a new engine named 'shipctl-docs'",
      "iam_permissions_granted": [
        "aiplatform.reasoningEngines.create",
        "aiplatform.reasoningEngines.update",
        "resourcemanager.projects.getIamPolicy",
        "resourcemanager.projects.setIamPolicy"
      ]
    },
    "runtime_shape": {
      "cpu": "2",
      "memory": "4Gi",
      "min_instances": 1,
      "max_instances": 6,
      "container_concurrency": 8
    }
  },
  "cloud_run_dry_run": {
    "command": "agents-cli deploy --deployment-target cloud_run --project vertex-ai-experminent --region us-central1 --cpu 1 --memory 4Gi --concurrency 8 --min-instances 1 --max-instances 10 --secrets SHIPCTL_TOKEN=shipctl-ci-token:latest --dry-run",
    "exit_code": 0,
    "would_run": "gcloud run deploy shipctl-docs --project vertex-ai-experminent --region us-central1 --source . --memory 4Gi --cpu 1 --min-instances 1 --max-instances 10 --concurrency 8 --no-allow-unauthenticated --no-cpu-throttling --update-env-vars 'GOOGLE_GENAI_USE_ENTERPRISE=***,GOOGLE_CLOUD_PROJECT=***,GOOGLE_CLOUD_LOCATION=***,AGENT_VERSION=***,ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=***,APP_URL=***' --update-secrets 'SHIPCTL_TOKEN=***' --update-labels 'created-by=***'"
  },
  "deployment_metadata": {
    "remote_agent_runtime_id": "projects/913703049174/locations/us-central1/reasoningEngines/7482910384756192000",
    "deployment_target": "agent_runtime",
    "is_a2a": true,
    "agent_name": "shipctl-docs",
    "region": "us-central1",
    "project": "vertex-ai-experminent"
  },
  "failures": {
    "gke_cpu_flag_rejected": {
      "command": "agents-cli deploy --deployment-target gke --project vertex-ai-experminent --region us-central1 --cpu 2 --dry-run",
      "exit_code": 1,
      "error": "--cpu is only supported for Agent Runtime and Cloud Run deployments (current target: gke).\n  On GKE, configure sizing via Terraform and the HorizontalPodAutoscaler under deployment/terraform/."
    },
    "terraform_after_imperative_deploy": {
      "error": "Resource ownership conflict: imperative 'agents-cli deploy' created the service before 'terraform apply'. Running 'terraform apply' now fails with 409 Already Exists (or creates a duplicate Agent Runtime engine). Provision Terraform first or import the resource."
    }
  }
}

Failure cases

Run the offline verification suite to test the service.tf contract, the recorded dry-run outputs, and the two failure boundaries:

cd "$LAB_DIR"
"$LAB_DIR/.venv/bin/python" -m unittest -v test_deploy_infra.py

Two failure boundaries frequently block cloud rollouts:

Passing –cpu or –memory when deployment target is gke. Because agents-cli delegates GKE pod sizing and autoscaling to Kubernetes manifests and the HorizontalPodAutoscaler under deployment/terraform/, passing --cpu, --memory, --concurrency, --min-instances, or --max-instances with --deployment-target gke is rejected immediately rather than silently ignored:

Error: --cpu is only supported for Agent Runtime and Cloud Run deployments (current target: gke).
  On GKE, configure sizing via Terraform and the HorizontalPodAutoscaler under deployment/terraform/.

Running terraform apply after an imperative agents-cli deploy. If you run agents-cli deploy in a fresh GCP project before running terraform apply, the CLI imperatively creates the Cloud Run service (docuctl-docs) or Agent Runtime ReasoningEngine. When you later run terraform apply in deployment/terraform/single-project/, Terraform can fail with Error 409: Resource 'docuctl-docs' already exists when it tries to create that same resource outside its state. An existing resource can instead be imported into the intended Terraform state after reviewing its configuration and ownership. Conversely, when Terraform owns an Agent Runtime engine, always pass --update-only to agents-cli deploy in CI/CD so a missing or misconfigured engine ID fails fast instead of creating a duplicate ReasoningEngine resource.

Deployment

Use the provisioning and rollout commands above in the intended project and environment. Retain the deployment metadata, then make a smoke request to the actual endpoint and check the selected identity. Keep a compatible previous revision and its configuration available for rollback; the offline helper cannot establish that a cloud rollout succeeded.

Scaling

Choose instance size, concurrency and replica limits from the target workload and model behavior. Test admission, timeouts and downstream quotas before increasing traffic. A valid deployment configuration and successful dry-run do not measure latency, capacity or cost.

Key takeaways

• Terraform owns IAM, storage, and the service shell (ignore_changes = [template[0].containers[0].image]), while agents-cli deploy owns container image rollouts.
• On Agent Runtime, agents-cli deploy --dry-run checks cloud APIs, permissions and engine state. Cloud Run dry-run prints the proposed command without checking the existing service.
• GKE sizing is controlled exclusively by Terraform and HorizontalPodAutoscaler; passing --cpu or --memory with --deployment-target gke fails fast.

Best practices

• For the Terraform-owned setup taught here, apply the reviewed Terraform plan before the first agents-cli deploy, and pass --update-only when deploying to a Terraform-provisioned Agent Runtime engine.
• Bind sensitive credentials through Secret Manager (--secrets DOCUCTL_TOKEN=docuctl-ci-token:latest) rather than baking secrets into .env or container images.
• Keep deployment_metadata.json as a build artifact so downstream agents-cli publish gemini and smoke-test steps can read the exact deployed resource ID and URL.

References

Commands reviewed against Google Agents CLI 1.11.0 and the linked official documentation on October 23, 2026.

• Google Agents CLI deployment guide: Agent Runtime, Cloud Run, and GKE deployment flags, --dry-run, and --update-only.
• Vertex AI Agent Engine (Reasoning Engine) documentation: container runtime requirements, Agent Identity, and IAM roles.