# 1. No identity token $ curl -s -o /dev/null -w "HTTP %{http_code}\n" $SERVICE_URL/list-apps HTTP 403 # 2. The runtime service account has no Agent Platform role yet $ ./request.sh "I run shipctl 1.x. How do I undo release r42 in production?" HTTP 200 in 0.119380 s session 4dfba80e-3d99-4762-a061-f5d2ae2c5d54 HTTP 500 in 3.082162 s Internal Server Error # The service log for that 500 $ gcloud logging read 'resource.type="cloud_run_revision" AND resource.labels.service_name="docs-helper" AND textPayload:"PERMISSION_DENIED"' --project $GOOGLE_CLOUD_PROJECT --freshness=15m --limit 1 google.genai.errors.ClientError: 403 PERMISSION_DENIED. {'error': {'code': 403, 'message': "Permission 'aiplatform.endpoints.predict' denied on resource '//aiplatform.googleapis.com/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-3.8-flash' (or it may not exist). Remediate access with this Trouble ... # 3. Grant the role to the runtime service account, then retry $ gcloud projects add-iam-policy-binding $GOOGLE_CLOUD_PROJECT --member="serviceAccount:docs-helper-run@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com" --role=roles/aiplatform.user --condition=None Updated IAM policy for project [PROJECT_ID]. # the first two retries, 30 s apart, still returned HTTP 500; the third, 68 s after the grant: $ ./request.sh "I run shipctl 1.x. How do I undo release r42 in production?" HTTP 200 in 0.087383 s session b9300104-e457-44b2-819c-4aef2e6d7e0e HTTP 200 in 5.006192 s call read_page {"page":"rollback"} result read_page answer In shipctl 1.x, use: ```bash shipctl release revert r42 ``` Note that database migrations are not automatically rolled back, so any migrations applied in r42 must be reversed manually. # 4. A new revision $ gcloud run services update docs-helper --region us-central1 --project $GOOGLE_CLOUD_PROJECT --update-env-vars=DOCS_LOG_LEVEL=info Done. Service [docs-helper] revision [docs-helper-00002-gpw] has been deployed and is serving 100 percent of traffic. # The session from before the update, on the new revision $ SESSION=b9300104-e457-44b2-819c-4aef2e6d7e0e ./request.sh "And on staging?" HTTP 404 in 1.399506 s { "detail": "Session not found: b9300104-e457-44b2-819c-4aef2e6d7e0e" } # A new session on the same instance $ ./request.sh "How do I undo the last production release? I run shipctl 2.x." HTTP 200 in 0.074428 s session 64e911d1-11c0-4852-830c-bcde5b41d74e HTTP 200 in 6.269696 s call read_page {"page":"rollback"} result read_page answer To undo the last production release in shipctl 2.x, run: ```bash shipctl rollback --env production ``` Omitting `--to` automatically rolls traffic back to the immediately preceding release. Note: Database migrations are not rolled back automatically. If the release ran a migration, you must deploy a separate migration to reverse it.