CloudSQL
Create a GCP project for attached resources.
All dbs should be configured with the SAP Hardening guide in mind. This is an example with the relevant settings for compliance, please always ensure that the configuration is up-to-date and complies with the SAP Hardening guide.
resource "google_sql_database_instance" "main" {
name = "my-db-name"
database_version = "POSTGRES_17"
region = "region"
project = "my-project-id"
settings {
tier = "db-custom-2-8192"
availability_type = "REGIONAL"
ip_configuration {
# SAP Hardening 8.3-4
ipv4_enabled = false
# SAP Hardening 8.3-12
ssl_mode = "TRUSTED_CLIENT_CERTIFICATE_REQUIRED"
# SAP Hardening 8.3-2
private_network = "projects/${var.network_project}/global/networks/vpc"
# 100.64/10 is not implicity enabled in Cloud SQL
authorized_networks {
name = "gap-primary-pod-range"
value = "100.65.0.0/16"
}
# authorized_networks {
# name = "gap-secondary-pod-range"
# value = "100.73.0.0/16"
# }
}
# https://docs.cloud.google.com/sql/docs/postgres/iam-authentication#instance-config-iam-auth
database_flags {
name = "cloudsql.iam_authentication"
value = "on"
}
# https://docs.cloud.google.com/sql/docs/postgres/pg-audit#setting-up-database-auditing
database_flags {
name = "cloudsql.enable_pgaudit"
value = "on"
}
database_flags {
name = "pgaudit.log"
value = "all"
}
# (...)
}
# lifecycle {
# prevent_destroy = true
# }
}
The authorized_networks section is required for Multi-Region instances. If you need a unified, skeletonized version, use the snippet below:
ip_configuration {
ipv4_enabled = false
private_network = var.private_network
ssl_mode = "TRUSTED_CLIENT_CERTIFICATE_REQUIRED"
dynamic "authorized_networks" {
for_each = var.has_allowed_networks ? [1] : []
content {
name = "gap-primary-pod-range"
value = "100.65.0.0/16"
}
}
}
Network project can be:
- ems-vpchost-staging (gap-staging)
- ems-vpchost-production (gap-production)
- ems-base-infra-net-s-us1-01 (gap-s-us1-01)
- ems-base-infra-net-p-us1-01 (gap-p-us1-01)
If you’re using HCP Terraform, you can dynamically select the appropriate network project for Multi-Region instances by using ems-base-infra-net-${var.instance_key}, which resolves based on the instance key.
SAP Security requires credential rotation every 90 days. Use workload identity with Cloud SQL Auth Proxy for automatic handling.
For Cloud SQL IAM authentication, you need a GCP service account bound to the KSA.
In infra-hub or by hand:
resource "google_service_account" "cloudsql_user" {
account_id = "my-app"
display_name = "Cloud SQL user for my-app"
project = "my-gcp-project"
}
resource "google_service_account_iam_member" "workload_identity_binding" {
service_account_id = google_service_account.cloudsql_user.name
role = "roles/iam.workloadIdentityUser"
member = "serviceAccount:<cluster-project-id>.svc.id.goog[my-namespace/my-app]"
}
resource "google_project_iam_member" "cloudsql_client" {
project = "my-gcp-project"
role = "roles/cloudsql.client"
member = "serviceAccount:${google_service_account.cloudsql_user.email}"
}
resource "google_project_iam_member" "cloudsql_instance_user" {
project = "my-gcp-project"
role = "roles/cloudsql.instanceUser"
member = "serviceAccount:${google_service_account.cloudsql_user.email}"
}
resource "google_sql_user" "iam_user" {
name = replace(google_service_account.cloudsql_user.email, ".gserviceaccount.com", "")
instance = google_sql_database_instance.main.name
type = "CLOUD_IAM_SERVICE_ACCOUNT"
project = "my-gcp-project"
database_roles = ["cloudsqlsuperuser"]
}
Note that database_roles is available starting with version 7.19 of the hashicorp/google provider. If you’re on an older version, update the provider to use it.
Cloud SQL Auth Proxy is not required if using Google Cloud SQL connectors.
- Enable the cloudSQLProxy sidecar in gap.yaml
- Connection string format:
postgresql://{SA_SHORT_NAME}%40{PROJECT}.iam@127.0.0.1:5432/{DB_NAME}
Example:
postgresql://my-app%40my-project.iam@127.0.0.1:5432/mydb
# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance#disk_size-1
resource "google_sql_database_instance" "main" {
# (...)
settings {
tier = "db-custom-4-16384" # 4 vCPUs, 16GB RAM
disk_size = 100 # GB
disk_autoresize = true
}
# (...)
}
# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance#enabled-2
# https://wiki.one.int.sap/wiki/spaces/itsec/pages/1916766708/Google+Cloud+Platform+-+Security+Hardening#GoogleCloudPlatformSecurityHardening-8.8SecureConfigurationofBackupServices
resource "google_sql_database_instance" "main" {
settings {
backup_configuration {
enabled = true
point_in_time_recovery_enabled = true
start_time = "08:00"
backup_retention_settings {
retained_backups = 7
retention_unit = "COUNT"
}
}
}
}
Complete IAM Authentication Setup first.
- Grant old user’s role to IAM user:
-- Connect as old user
GRANT {OLD_USER} TO "{IAM_USER}@{PROJECT}.iam";
Update deployments to use IAM authentication (see Application Configuration)
Reassign ownership:
-- Connect as IAM user
REASSIGN OWNED BY {OLD_USER} TO "{IAM_USER}@{PROJECT}.iam";
Reassign privileges (if applicable):
- Check privileges: use
psqlbackslash commands - Check default privileges:
\ddp - Grant same privileges to IAM user
- Drop old privileges:
DROP OWNED BY {OLD_USER}
- Check privileges: use
Remove old user via Terraform or console
Permission errors appear in Cloud Logging with:
status: {
code: 13
}
Cause: Service Networking Service Agent lacks permissions in the VPC host project.
Resolution: Contact GAP Team via #infra-support Slack or GAP Jira with:
- Project hosting the CloudSQL instance
- Service account:
service-<project-number>@service-networking.iam.gserviceaccount.com - Target VPC:
ems-vpchost-stagingfor stagingems-vpchost-productionfor production
To ensure the security of the Cloud SQL instances (Postgres in this case, but this method should work for other DBs as well), public IP has to be disabled, and TLS must be enforced. This poses a few issues when one wants to connect for operational tasks. This guide shows you the recommended way to do this on GAP using Google’s SQL Auth Proxy.
To simplify connecting to the Cloud SQL instance from a local machine, you can use a CLI tool that automates the process, instead of running the below steps manually. Check the GitHub repositoryemartech/google-cloud-sql-clifor installation and usage instructions.
A few initial configuration steps must be taken before one can go on opening a pod to connect.
- SQL Admin API needs to be enabled on the project hosting the instance (gcloud does it if you run
gcloud sql instances list --project <project>) - Get the connection name
gcloud sql instances describe --project <project> <instance> --format json | jq '.connectionName'- or manually:
<project>:<region>:<instance> - example:
sap-gap-example-p:europe-west3:example
- A service account must exist that has
- Cloud SQL Client IAM role in the project hosting your postgres instance
- on GAP a service account must exist that is configured for workload identity with the above mentioned service account
There are two options for one-off database connection:
- run the CloudSQL proxy in GAP, port-forward it to local machine and use whatever client to interact with the database
- run the CloudSQL proxy and a
psqlin a pod on GAP with interactive shell and usepsql clidirectly in the pod
This method uses a SQL Auth Proxy on GAP and port-forwarding to your local machine:
kubectl run -i --rm -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='{
"apiVersion": "v1",
"spec": {
"serviceAccount": "<SERVICE_ACCOUNT>",
"securityContext": {
"runAsUser": 1000,
"fsGroup": 1000
},
},
}' \
--image=gcr.io/cloudsql-docker/gce-proxy:latest psql-client-proxy -- /cloud_sql_proxy -ip_address_types=PRIVATE -instances=<CONNECTION_NAME>=tcp:5432
If you have set up IAM authentication, you have to add the -enable_iam_login flag to the command:
kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='{
"apiVersion": "v1",
"spec": {
"serviceAccount": "<SERVICE_ACCOUNT>",
"securityContext": {
"runAsUser": 1000,
"fsGroup": 1000
},
},
}' \
--image=gcr.io/cloudsql-docker/gce-proxy:latest psql-client-proxy -- /cloud_sql_proxy -enable_iam_login -ip_address_types=PRIVATE -instances=<CONNECTION_NAME>=tcp:5432
This will still create an interactive pod, so pressing Ctrl+C will stop and delete the container. And as usual, if there is an error, an investigation and manual deletion are necessary.
As for the port-forwarding:
kubectl port-forward -n <NAMESPACE> psql-client-proxy 5432
After this it is possible to connect using the 127.0.0.1:5432 on the local machine. In case of IAM authentication the user is your-service-account@your-project.iam.
The following command will exec into a shell on the postgres image with the SQL Auth Proxy running as a sidecar. From the shell one can connect to postgres as usual replacing the host with 127.0.0.1.
- SERVICE_ACCOUNT with workload identity bound to the account that got permission above
- CONNECTION_NAME the connectionName extracted above (PROJECT:REGION:INSTANCE)
- NAMESPACE your namespace
kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='
{
"apiVersion": "v1",
"spec": {
"serviceAccount": "<SERVICE_ACCOUNT>",
"securityContext": {
"runAsUser": 1000,
"fsGroup": 1000
},
"containers": [
{"name": "psql-client-proxy",
"image": "postgres",
"stdin": true,
"stdinOnce": true,
"tty": true,
"command": ["/bin/shell"]
}, {
"name": "sql-proxy",
"image": "gcr.io/cloudsql-docker/gce-proxy:latest",
"command": ["/cloud_sql_proxy", "-ip_address_types=PRIVATE", "-instances=<CONNECTION_NAME>=tcp:5432"]}]
}
}' --image=postgres psql-client-proxy
If you have set up IAM authentication, you have to add the -enable_iam_login flag to the command:
kubectl run -i --rm --tty -n <NAMESPACE> --annotations="cluster-autoscaler.kubernetes.io/safe-to-evict=true" --overrides='
{
"apiVersion": "v1",
"spec": {
"serviceAccount": "<SERVICE_ACCOUNT>",
"securityContext": {
"runAsUser": 1000,
"fsGroup": 1000
},
"containers": [
{"name": "psql-client-proxy",
"image": "postgres",
"stdin": true,
"stdinOnce": true,
"tty": true,
"command": ["/bin/bash"]
}, {
"name": "sql-proxy",
"image": "gcr.io/cloudsql-docker/gce-proxy:latest",
"command": ["/cloud_sql_proxy", "-enable_iam_login", "-ip_address_types=PRIVATE", "-instances=<CONNECTION_NAME>=tcp:5432"]}]
}
}' --image=postgres psql-client-proxy
Run in the shell something like: psql -h 127.0.0.1 -p 5432 …
If you are using IAM login: psql -h 127.0.0.1 -p 5432 -u <your-service-account@your-project.iam> -d <your-database>