Understanding audit logs

This page describes Cloud Audit Logs log entries in detail: their structure, how to read them, and how to interpret them.

Cloud Audit Logs provides the following audit logs for each Trusted Cloud project, folder, and organization:

• Admin Activity audit logs
• Data Access audit logs
• System Event audit logs
• Policy Denied audit logs

For a general overview of Cloud Audit Logs, see Cloud Audit Logs.

Format of audit log entries

An audit log entry is a type of Cloud Logging log entry. Like all Logging log entries, an audit log entry is stored in a LogEntry object. What distinguishes an audit log entry from other log entries is the protoPayload field. In audit log entries, the log entry's protoPayload field contains an AuditLog object that stores the audit logging data.

In short, every audit log entry is characterized by the following information:

• The project, folder, or organization that owns the log entry.
• The resource to which the log entry applies. This information consists of a resource type from the Monitored resource list and additional values that denote a specific instance. For example, you can view audit log entries from a single Compute Engine VM instance or from all VM instances.
• A timestamp.

• A service: Services are individual Trusted Cloud products, such as Compute Engine, Cloud SQL, or Pub/Sub. Each service is identified by name: Compute Engine is compute.googleapis.com, Cloud SQL is cloudsql.googleapis.com, and so forth. This information is listed in the protoPayload.serviceName field of the audit log entry.

  Resource types belongs to a single service, but a service can have several resource types. For a list of services and resources, go to Mapping services to resources.

• A payload, which is the protoPayload type. The payload of each audit log entry is an object of type AuditLog, which defines a set of fields specific to Cloud Audit Logs, such as serviceName and authenticationInfo. It also has an optional field, metadata, that Trusted Cloud by S3NS services use to list service-specific information in the audit log entry. Some Trusted Cloud by S3NS services still use the older serviceData field to list service-specific information. For a list of services that use the serviceData field, see Service-specific audit data.

• A log name: Audit log entries belong to logs within billing accounts, projects, folders, and organizations. The following table lists log names:

   projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity
   projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access
   projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_event
   projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fpolicy

   folders/FOLDER_ID/logs/cloudaudit.googleapis.com%2Factivity
   folders/FOLDER_ID/logs/cloudaudit.googleapis.com%2Fdata_access
   folders/FOLDER_ID/logs/cloudaudit.googleapis.com%2Fsystem_event
   folders/FOLDER_ID/logs/cloudaudit.googleapis.com%2Fpolicy

   billingAccounts/BILLING_ACCOUNT_ID/logs/cloudaudit.googleapis.com%2Factivity
   billingAccounts/BILLING_ACCOUNT_ID/logs/cloudaudit.googleapis.com%2Fdata_access
   billingAccounts/BILLING_ACCOUNT_ID/logs/cloudaudit.googleapis.com%2Fsystem_event
   billingAccounts/BILLING_ACCOUNT_ID/logs/cloudaudit.googleapis.com%2Fpolicy

   organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com%2Factivity
   organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com%2Fdata_access
   organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com%2Fsystem_event
   organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com%2Fpolicy

Within a billing account, project, folder, or organization, these log names are typically abbreviated as activity, data_access, system_event, and policy.

Sample audit log entry

This section uses a sample audit log entry to explain how to find the most important information in audit log entries.

The following sample is an Admin Activity audit log entry that records a change to an Identity and Access Management (IAM) policy with PROJECT_ID my-gcp-project-id. The serviceName field identifies the service that wrote the audit log. For brevity, some parts of the log entry are omitted, and some fields are highlighted:

    {
      protoPayload: {
        @type: "type.googleapis.com/google.cloud.audit.AuditLog",
        status: {},
        authenticationInfo: {
          principalEmail: "user@example.com"
        },
        serviceName: "appengine.googleapis.com",
        methodName: "SetIamPolicy",
        authorizationInfo: [...],
        serviceData: {
          @type: "type.googleapis.com/google.appengine.legacy.AuditData",
          policyDelta: { bindingDeltas: [
              action: "ADD",
              role: "roles/logging.privateLogViewer",
              member: "user:user@example.com"
          ], }
        },
        request: {
          resource: "my-gcp-project-id",
          policy: { bindings: [...], }
        },
        response: {
          bindings: [
            {
              role: "roles/logging.privateLogViewer",
              members: [ "user:user@example.com" ]
            }
          ],
        }
      },
      insertId: "53179D9A9B559.AD6ACC7.B40604EF",
      resource: {
        type: "gae_app",
        labels: { project_id: "my-gcp-project-id" }
      },
      timestamp: "2019-05-27T16:24:56.135Z",
      severity: "NOTICE",
      logName: "projects/my-gcp-project-id/logs/cloudaudit.googleapis.com%2Factivity",
    }

Here is the query that was used to select the previous audit log entry sample. The query can be used in the Logs Explorer, Logging API, or Google Cloud CLI. The project identifier is in the log's name:

    resource.type = "gae_app"
    logName = "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"

If you are looking for audit logs from a single instance of a resource type, such as gce_instance, add an instance qualifier:

    resource.type = "gce_instance"
    resource.instance_id = "INSTANCE_ID"
    logName = "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"

Interpreting the sample audit log entry

In the previous audit log entry sample, the protoPayload, insertId, resource, timestamp, severity and logName fields shown are part of the LogEntry object. The value of the protoPayload field is an AuditLog object. It encapsulates the audit logging data.

Looking at the audit log entry sample, you might have some questions:

• Is this an audit log entry? It is, which you can tell in two ways:

  • The protoPayload.@type field is type.googleapis.com/google.cloud.audit.AuditLog.

  • The logName field includes the domain cloudaudit.googleapis.com.

• What service wrote the audit log? The log was written by App Engine. This information is listed in the protoPayload.serviceName field of the audit log entry.

• What operation is being audited? SetIamPolicy, as specified in the protoPayload.methodName field, is being audited. More information about the audited operation is in the AuditData object in protoPayload.serviceData.

• What resource is being audited? An application running in App Engine, associated with a Trusted Cloud project my-gcp-project-id, is being audited. You can determine this from the resource field, which specifies the resource type gae_app and the project identifier my-gcp-project-id. In this example, you would find details on the resource type in the monitored resource type list.

For more information, see the LogEntry type, the AuditLog type, and the IAM AuditData type.

Audit logs for long-running operations

APIs that are long-running operations emit two audit logs; one when the API is called and the operation starts, and one when the operation completes.

In this case, the LogEntry object contains an operation field. Log entries for the same operation have the same value for both LogEntry.operation.id and LogEntry.operation.producer. The first log entry that was written has LogEntry.operation.first=true, and the completion log entry has LogEntry.operation.last=true.

In cases where the operation completes immediately or fails, there is only one log entry containing both LogEntry.operation.first=true and LogEntry.operation.last=true.

Some services don't populate the LogEntry.operation field when the operation fails. However, you can determine which operations are long-running operations by referring to the service's audit logging documentation.

These APIs implement the Operations service. This service generally emit audit log entries when called. Depending on which APIs are called, protoPayload.methodName is one of the following:

• google.longrunning.Operations.ListOperations
• google.longrunning.Operations.GetOperation
• google.longrunning.Operations.CancelOperation
• google.longrunning.Operations.WaitOperation
• google.longrunning.Operations.DeleteOperation

LogEntry.operation isn't specified in this case, as this API returns metadata about long-running operations, but is not a long-running operation itself.

See Trusted Cloud by S3NS services with audit logs for details about which APIs are audited, as it can vary per service.

Audit logs for streaming APIs

Similar to long-running operations, streaming APIs emit two audit log entries; one when the API is first called and one when the streaming connection has ended.

In this case, the LogEntry object contains an operation field and log entries for the same operation have the same value for both LogEntry.operation.id and LogEntry.operation.producer. The first log written has LogEntry.operation.first=true, and the completion log will have LogEntry.operation.last=true.

This API may also emit continuation log entries with neither LogEntry.operation.first nor LogEntry.operation.last set to indicate that the stream remains open.

Service-specific audit data

Some services extend the information stored in their AuditLog by placing a supplementary data structure in the audit log entry's serviceData field. The following table lists the services that use serviceData field and provides a link to their AuditData type.

Viewing audit logs

You can query for all audit logs or you can query for logs by their audit log name. The audit log name includes the resource identifier of the Trusted Cloud project, folder, billing account, or organization for which you want to view audit logging information. Your queries can specify indexed LogEntry fields. For more information about querying your logs, see Build queries in the Logs Explorer

Most audit logs can be viewed in Cloud Logging by using the Trusted Cloud console, the Google Cloud CLI, or the Logging API. However, for audit logs related to billing, you can only use the Google Cloud CLI or the Logging API.

gcloud logging read "logName : projects/PROJECT_ID/logs/cloudaudit.googleapis.com" \
    --project=PROJECT_ID

gcloud logging read "logName : folders/FOLDER_ID/logs/cloudaudit.googleapis.com" \
    --folder=FOLDER_ID

gcloud logging read "logName : organizations/ORGANIZATION_ID/logs/cloudaudit.googleapis.com" \
    --organization=ORGANIZATION_ID

gcloud logging read "logName : billingAccounts/BILLING_ACCOUNT_ID/logs/cloudaudit.googleapis.com" \
    --billing-account=BILLING_ACCOUNT_ID