GAP Documentation
GitHub Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

CloudSQL

Prerequisites

Create a GCP project for attached resources.

Create CloudSQL Instance

All dbs should be configured with the SAP Hardening guide in mind. This is an example with the relevant settings for compliance, please always ensure that the configuration is up-to-date and complies with the SAP Hardening guide.

resource "google_sql_database_instance" "main" {
  name             = "my-db-name"
  database_version = "POSTGRES_17"
  region           = "region"
  project          = "my-project-id"

  settings {
    tier              = "db-custom-2-8192"
    availability_type = "REGIONAL"

    ip_configuration {
      # SAP Hardening 8.3-4
      ipv4_enabled = false
      # SAP Hardening 8.3-12
      ssl_mode     = "TRUSTED_CLIENT_CERTIFICATE_REQUIRED"
      # SAP Hardening 8.3-2
      private_network = "projects/${var.network_project}/global/networks/vpc"
      # 100.64/10 is not implicity enabled in Cloud SQL
      authorized_networks {
        name = "gap-primary-pod-range"
        value = "100.65.0.0/16"
      }
      # authorized_networks {
      #   name = "gap-secondary-pod-range"
      #   value = "100.73.0.0/16"
      # }
    }

    # https://docs.cloud.google.com/sql/docs/postgres/iam-authentication#instance-config-iam-auth
    database_flags {
      name  = "cloudsql.iam_authentication"
      value = "on"
    }
    # https://docs.cloud.google.com/sql/docs/postgres/pg-audit#setting-up-database-auditing
    database_flags {
      name  = "cloudsql.enable_pgaudit"
      value = "on"
    }
    database_flags {
      name  = "pgaudit.log"
      value = "all"
    }
    # (...)
  }
  # lifecycle {
  #   prevent_destroy = true
  # }
}

The authorized_networks section is required for Multi-Region instances. If you need a unified, skeletonized version, use the snippet below:

    ip_configuration {
      ipv4_enabled    = false
      private_network = var.private_network
      ssl_mode        = "TRUSTED_CLIENT_CERTIFICATE_REQUIRED"

      dynamic "authorized_networks" {
        for_each = var.has_allowed_networks ? [1] : []
        content {
          name  = "gap-primary-pod-range"
          value = "100.65.0.0/16"
        }
      }
    }

Network project can be:

  • ems-vpchost-staging (gap-staging)
  • ems-vpchost-production (gap-production)
  • ems-base-infra-net-s-us1-01 (gap-s-us1-01)
  • ems-base-infra-net-p-us1-01 (gap-p-us1-01)

If you’re using HCP Terraform, you can dynamically select the appropriate network project for Multi-Region instances by using ems-base-infra-net-${var.instance_key}, which resolves based on the instance key.

IAM Authentication Setup

SAP Security requires credential rotation every 90 days. Use workload identity with Cloud SQL Auth Proxy for automatic handling.

For Cloud SQL IAM authentication, you need a GCP service account bound to the KSA.

1. Create the GCP Service Account with Workload Identity role

In infra-hub or by hand:

resource "google_service_account" "cloudsql_user" {
  account_id   = "my-app"
  display_name = "Cloud SQL user for my-app"
  project      = "my-gcp-project"
}
resource "google_service_account_iam_member" "workload_identity_binding" {
  service_account_id = google_service_account.cloudsql_user.name
  role               = "roles/iam.workloadIdentityUser"
  member             = "serviceAccount:<cluster-project-id>.svc.id.goog[my-namespace/my-app]"
}

2. Grant Cloud SQL Permissions

resource "google_project_iam_member" "cloudsql_client" {
  project = "my-gcp-project"
  role    = "roles/cloudsql.client"
  member  = "serviceAccount:${google_service_account.cloudsql_user.email}"
}

resource "google_project_iam_member" "cloudsql_instance_user" {
  project = "my-gcp-project"
  role    = "roles/cloudsql.instanceUser"
  member  = "serviceAccount:${google_service_account.cloudsql_user.email}"
}

3. Create IAM Database User

resource "google_sql_user" "iam_user" {
  name     = replace(google_service_account.cloudsql_user.email, ".gserviceaccount.com", "")
  instance = google_sql_database_instance.main.name
  type     = "CLOUD_IAM_SERVICE_ACCOUNT"
  project  = "my-gcp-project"
  database_roles = ["cloudsqlsuperuser"]
}

Note that database_roles is available starting with version 7.19 of the hashicorp/google provider. If you’re on an older version, update the provider to use it.

Application Configuration

Cloud SQL Auth Proxy is not required if using Google Cloud SQL connectors.
  1. Enable the cloudSQLProxy sidecar in gap.yaml
  2. Connection string format:
postgresql://{SA_SHORT_NAME}%40{PROJECT}.iam@127.0.0.1:5432/{DB_NAME}

Example:

postgresql://my-app%40my-project.iam@127.0.0.1:5432/mydb

Scaling

# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance#disk_size-1
resource "google_sql_database_instance" "main" {
  # (...)
  settings {
    tier = "db-custom-4-16384"  # 4 vCPUs, 16GB RAM

    disk_size       = 100  # GB
    disk_autoresize = true
  }
  # (...)
}

Backup and Recovery

# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance#enabled-2
# https://wiki.one.int.sap/wiki/spaces/itsec/pages/1916766708/Google+Cloud+Platform+-+Security+Hardening#GoogleCloudPlatformSecurityHardening-8.8SecureConfigurationofBackupServices
resource "google_sql_database_instance" "main" {
  settings {
    backup_configuration {
      enabled                        = true
      point_in_time_recovery_enabled = true
      start_time                     = "08:00"
      
      backup_retention_settings {
        retained_backups = 7
        retention_unit   = "COUNT"
      }
    }
  }
}

Migrating from Built-in to IAM Authentication

Complete IAM Authentication Setup first.
  1. Grant old user’s role to IAM user:
-- Connect as old user
GRANT {OLD_USER} TO "{IAM_USER}@{PROJECT}.iam";
  1. Update deployments to use IAM authentication (see Application Configuration)

  2. Reassign ownership:

-- Connect as IAM user
REASSIGN OWNED BY {OLD_USER} TO "{IAM_USER}@{PROJECT}.iam";
  1. Reassign privileges (if applicable):

    • Check privileges: use psql backslash commands
    • Check default privileges: \ddp
    • Grant same privileges to IAM user
    • Drop old privileges: DROP OWNED BY {OLD_USER}
  2. Remove old user via Terraform or console

Troubleshooting

Instance Creation Fails

Permission errors appear in Cloud Logging with:

status: {
  code: 13
}

Cause: Service Networking Service Agent lacks permissions in the VPC host project.

Resolution: Contact GAP Team via #infra-support Slack or GAP Jira with:

  • Project hosting the CloudSQL instance
  • Service account: service-<project-number>@service-networking.iam.gserviceaccount.com
  • Target VPC:
    • ems-vpchost-staging for staging
    • ems-vpchost-production for production

Connecting to CloudSQL

To ensure the security of the Cloud SQL instances (Postgres in this case, but this method should work for other DBs as well), public IP has to be disabled, and TLS must be enforced. This poses a few issues when one wants to connect for operational tasks. This guide shows you the recommended way to do this on GAP using Google’s SQL Auth Proxy.

To simplify connecting to the Cloud SQL instance from a local machine, you can use a CLI tool that automates the process, instead of running the below steps manually. Check the GitHub repository emartech/google-cloud-sql-cli for installation and usage instructions.

Initialization

A few initial configuration steps must be taken before one can go on opening a pod to connect.

  • SQL Admin API needs to be enabled on the project hosting the instance (gcloud does it if you run gcloud sql instances list --project <project>)
  • Get the connection name
    • gcloud sql instances describe --project <project> <instance> --format json | jq '.connectionName'
    • or manually: <project>:<region>:<instance>
    • example: sap-gap-example-p:europe-west3:example
  • A service account must exist that has
    • Cloud SQL Client IAM role in the project hosting your postgres instance
  • on GAP a service account must exist that is configured for workload identity with the above mentioned service account

There are two options for one-off database connection:

  • run the CloudSQL proxy in GAP, port-forward it to local machine and use whatever client to interact with the database
  • run the CloudSQL proxy and a psql in a pod on GAP with interactive shell and use psql cli directly in the pod

Port-forward the SQL Auth Proxy to local machine

This method uses a SQL Auth Proxy on GAP and port-forwarding to your local machine:

kubectl run -i --rm -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='{
  "apiVersion": "v1",
  "spec": { 
    "serviceAccount": "<SERVICE_ACCOUNT>",
    "securityContext": {
      "runAsUser": 1000,
      "fsGroup": 1000
    },
  },
}' \
--image=gcr.io/cloudsql-docker/gce-proxy:latest psql-client-proxy -- /cloud_sql_proxy -ip_address_types=PRIVATE -instances=<CONNECTION_NAME>=tcp:5432

If you have set up IAM authentication, you have to add the -enable_iam_login flag to the command:

kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='{
  "apiVersion": "v1",
  "spec": { 
    "serviceAccount": "<SERVICE_ACCOUNT>",
    "securityContext": {
      "runAsUser": 1000,
      "fsGroup": 1000
    },
  },
}' \
--image=gcr.io/cloudsql-docker/gce-proxy:latest psql-client-proxy -- /cloud_sql_proxy -enable_iam_login -ip_address_types=PRIVATE -instances=<CONNECTION_NAME>=tcp:5432

This will still create an interactive pod, so pressing Ctrl+C will stop and delete the container. And as usual, if there is an error, an investigation and manual deletion are necessary.

As for the port-forwarding:

kubectl port-forward -n <NAMESPACE> psql-client-proxy 5432

After this it is possible to connect using the 127.0.0.1:5432 on the local machine. In case of IAM authentication the user is your-service-account@your-project.iam.

Use PSQL cli in GAP pod

The following command will exec into a shell on the postgres image with the SQL Auth Proxy running as a sidecar. From the shell one can connect to postgres as usual replacing the host with 127.0.0.1.

  • SERVICE_ACCOUNT with workload identity bound to the account that got permission above
  • CONNECTION_NAME the connectionName extracted above (PROJECT:REGION:INSTANCE)
  • NAMESPACE your namespace
kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='
{
  "apiVersion": "v1",
    "spec": {
      "serviceAccount": "<SERVICE_ACCOUNT>",
      "securityContext": {
        "runAsUser": 1000,
        "fsGroup": 1000
      },
      "containers": [
        {"name": "psql-client-proxy",
          "image": "postgres",
          "stdin": true,
          "stdinOnce": true,
          "tty": true,
          "command": ["/bin/shell"]
        }, {
          "name": "sql-proxy",
          "image": "gcr.io/cloudsql-docker/gce-proxy:latest",
          "command": ["/cloud_sql_proxy", "-ip_address_types=PRIVATE", "-instances=<CONNECTION_NAME>=tcp:5432"]}]
    }
}' --image=postgres psql-client-proxy

If you have set up IAM authentication, you have to add the -enable_iam_login flag to the command:

kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='
{
  "apiVersion": "v1",
    "spec": {
      "serviceAccount": "<SERVICE_ACCOUNT>",
      "securityContext": {
        "runAsUser": 1000,
        "fsGroup": 1000
      },
      "containers": [
        {"name": "psql-client-proxy",
          "image": "postgres",
          "stdin": true,
          "stdinOnce": true,
          "tty": true,
          "command": ["/bin/bash"]
        }, {
          "name": "sql-proxy",
          "image": "gcr.io/cloudsql-docker/gce-proxy:latest",
          "command": ["/cloud_sql_proxy", "-enable_iam_login", "-ip_address_types=PRIVATE", "-instances=<CONNECTION_NAME>=tcp:5432"]}]
    }
}' --image=postgres psql-client-proxy

Run in the shell something like: psql -h 127.0.0.1 -p 5432 …

If you are using IAM login: psql -h 127.0.0.1 -p 5432 -u <your-service-account@your-project.iam> -d <your-database>