Logging

To enable logging for your application, you have to

1. deploy your application to GAP
2. create a Systec change request ticket to enable Elastic Common Schema or custom field mappings
   1. in the ticket also ask Systec to add the specific index to their Logstash config (this is required to workaround a kubernetes bug)
3. create the GCP log sinks (routing your logs in the shipping pipeline) and required LAAS2 side config in tooling portal by:
   1. finding the service in the list of services
   2. select the appropriate G_GSuite group from the dropdown
   3. press Create/Update Logsinks button in “GAP Logsinks” column (if it’s green and disabled, then the sinks are already in place)
   4. press the Create/Update button for the LaaS index configuration in the LaaS2 Config column

You can get more info on Emarsys ECS from Systec documentation here.

The logs from your applications are automatically forwarded to Laas (Emarsys VPN is required). The name of the space is Google Cloud, then search for discover on the top bar and then select the index name gap-{name}, where the name is what you set in your gap.yaml.

To view the logs of a specific pod:

kubectl logs -n NAMESPACE POD_NAME

Example:

kubectl logs -n tooling gap-example-project-web-6f85cb4b99-rthgq

To view/stream logs from all pods in the deployment:

kubectl -n NAMESPACE logs -f deployment/DEPLOYMENT_NAME --all-containers=true

Example:

kubectl -n personalization logs -f deployment/personalization-data-v1-web --all-containers=true

Your router logs can be seen in your application’s index. They have the property router_log set to true.

To mark logs as audit log, you have to add the field log_type: audit to the log line. Your audit logs are then located in the index audit in LaaS.

Please see our elastalert setup guide

The default soft quota is 1 GB/day and most of the application do not exceed this amount. For more information about the quotas please visit the Honoring Quotas section of the LaaS Guide.

The container logs can be viewed by going to the Google Kubernetes Engine (GKE) workload page here for staging and here for production, selecting the relevant namespace and then the workload, then clicking on the Container logs on the logs field. This will take you to the Logs Explorer with automatically added filters pertinent to the workload you have selected.