Workload Identity
Workload Identity automatically provides Google identity to Kubernetes workloads via short-lived access tokens—more secure than static JSON key files.
Applications get a default KSA named after the application. Configure it in gap.yaml. The service account is automatically mounted on deployments, cronjobs, and pre/post deploy pods.
| Cluster | Environment | Project Number | Project ID |
|---|---|---|---|
| gap-staging | EU staging | 15789844182 | ems-gap-stage |
| gap-production | EU production | 193988271297 | ems-gap-production |
| gap-s-us1-01 | US staging | 656527773986 | ems-gap-s-us1-01 |
| gap-p-us1-01 | US production | 1020362820938 | ems-gap-p-us1-01 |
There are three ways to grant GCP permissions to your KSA depending on your IAM setup.
If your project is managed in infra-hub resman, use the $iam_principals interpolation:
iam:
roles/pubsub.subscriber:
- $iam_principals:wi/gap/<NAMESPACE>/<KSA_NAME>
This resolves to the correct principal for each cluster automatically.
Grant permissions directly to the Kubernetes service account using the full principal format:
principal://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<PROJECT_ID>.svc.id.goog/subject/ns/<NAMESPACE>/sa/<KSA_NAME>
Use the cluster parameters table above for the <PROJECT_NUMBER> and <PROJECT_ID> values. You need one binding per cluster where your workload runs.
Terraform example (Pub/Sub):
resource "google_pubsub_subscription_iam_member" "subscriber" {
subscription = "your-subscription-name"
role = "roles/pubsub.subscriber"
member = "principal://iam.googleapis.com/projects/193988271297/locations/global/workloadIdentityPools/ems-gap-production.svc.id.goog/subject/ns/my-namespace/sa/my-app"
}
Some GCP services (e.g., Cloud SQL) require a GCP service account (GSA) rather than a direct principal binding. In this case, create a GSA, bind it to your KSA, then grant permissions to the GSA.
resource "google_service_account" "my_app" {
account_id = "my-app"
display_name = "Service account for my-app"
project = "my-gcp-project"
}
resource "google_service_account_iam_member" "workload_identity_binding" {
service_account_id = google_service_account.my_app.name
role = "roles/iam.workloadIdentityUser"
member = "serviceAccount:<cluster-project-id>.svc.id.goog[my-namespace/my-app]"
}
Then grant roles to the GSA as usual:
resource "google_project_iam_member" "example" {
project = "my-gcp-project"
role = "roles/cloudsql.client"
member = "serviceAccount:${google_service_account.my_app.email}"
}
See CloudSQL docs for a complete Cloud SQL example.
Workload Identity works with Application Default Credentials—Google client libraries authenticate automatically without configuration.
Workload Identity tokens are not always available immediately when a pod starts. If your application authenticates to GCP APIs during startup, it may fail with permission errors before the token is ready.
Set waitForWorkloadIdentity: true in gap.yaml to add an init container that blocks until the token is available:
deployments:
web:
command: ["node", "server.js"]
waitForWorkloadIdentity: true
This setting is available on all scopes: root level, per-deployment, cronjobs, preDeploy, and postDeploy.
This adds a few seconds to pod startup time. Only enable it if you experience authentication errors during application startup.
Service mesh note: If you rely on a custom init container to wait for workload identity, it will not work after onboarding to service mesh. Use waitForWorkloadIdentity instead—it is compatible with mesh.
kubectl run --rm -it \
--overrides='{"spec":{"serviceAccount":"my-app","securityContext":{"runAsUser":1000,"fsGroup":1000}}}' \
--image google/cloud-sdk:slim \
--namespace my-namespace \
workload-identity-test
# Inside the pod:
curl -H "Metadata-Flavor: Google" \
http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/email
The response should show your GCP service account email, not default.
If you use a GCP service account (Option 3), verify that the KSA is bound to it:
gcloud iam service-accounts get-iam-policy <GSA_EMAIL> --project <PROJECT_ID>
Look for your KSA in the members list with roles/iam.workloadIdentityUser.
For resource-level bindings (Options 1 & 2), check the IAM policy on the specific resource, e.g.:
# Pub/Sub example
gcloud pubsub subscriptions get-iam-policy <SUBSCRIPTION> --project <PROJECT_ID>
You can also verify on the GCP console: navigate to IAM & Admin → Service Accounts, open your service account’s Permissions tab, and look for ems-gap-<instance-key>.svc.id.goog[my-namespace/my-app] with Workload Identity User.
| Symptom | Likely Cause | Fix |
|---|---|---|
403 or PERMISSION_DENIED on startup, works after a few seconds | WI token not ready yet | Set waitForWorkloadIdentity: true in gap.yaml |
403 on all requests, even after pod is running | Missing IAM binding | Verify the principal/GSA binding matches your namespace and KSA name exactly |
Token email shows default instead of your GSA | KSA not annotated correctly | Check that gap.yaml has the correct name and namespace; redeploy |
| Init container hangs after mesh onboarding | Custom WI init container incompatible with mesh | Replace with waitForWorkloadIdentity: true (service mesh docs) |