Collate and route organization- and folder-level logs to supported destinations

Console

To create an aggregated sink for your folder or organization, do the following:

1. In the Cloud de Confiance console, go to the Log Router page:

   Go to Log Router

   If you use the search bar to find this page, then select the result whose subheading is Logging.

2. Select an existing folder or organization.

3. Select Create sink.

4. In the Sink details panel, enter the following details:

   • Sink name: Provide an identifier for the sink. After you create the sink, you can't rename it, but you can delete it and create a new one.

   • Sink description (optional): Describe the purpose or use case for the sink.

5. In the Select sink service menu, select the type of destination, and then complete the dialog to specify the destination. You can select an existing destination or create the destination.

   • For an intercepting sink, select Cloud de Confiance project, and then enter the fully qualified name of the destination Cloud de Confiance project:

     logging.googleapis.com/projects/DESTINATION_PROJECT_ID
     

   • For a non-intercepting sink, select the destination, and then enter the fully qualified name of the destination. The following destinations are supported:

     • Cloud de Confiance project

       logging.googleapis.com/projects/DESTINATION_PROJECT_ID
       

     • Logging bucket

       logging.googleapis.com/projects/DESTINATION_PROJECT_ID/locations/LOCATION/buckets/BUCKET_NAME
       

     • Pub/Sub topic

       Don't enter the name of a Pub/Sub topic that enforces in-transit restrictions.

       pubsub.googleapis.com/projects/DESTINATION_PROJECT_ID/topics/TOPIC_ID
       

6. In the Choose logs to include in sink panel, select the resources to include in the sink.

   • For an intercepting sink, select Intercept logs ingested by this organization and all child resources.

   • For a non-intercepting sink, select Include logs ingested by this resource and all child resources.

7. In the Build inclusion filter field, enter a filter expression that matches the log entries that you want to include. If you don't set a filter, all log entries from your selected resource are routed to the destination.

   For example, you might want to build a filter to route all Data Access audit logs to a single Logging bucket. This filter looks like the following:

   LOG_ID("cloudaudit.googleapis.com/data_access") OR LOG_ID("externalaudit.googleapis.com/data_access")
   

   For examples, see Filters for aggregated sinks.

   Note that the length of a filter can't exceed 20,000 characters.

8. Optional: To verify that you entered the correct filter, select Preview logs. This opens the Logs Explorer in a new tab with the filter pre-populated.

9. Optional: In the Choose logs to exclude from the sink panel, do the following:

   1. In the Exclusion filter name field, enter a name.

   2. In the Build an exclusion filter field, enter a filter expression that matches the log entries that you want to exclude. You can also use the sample function to select a portion of the log entries to exclude.

      For example, to exclude the log entries from a specific project from being routed to the destination, add the following exclusion filter:

      logName:projects/PROJECT_ID
      

      To exclude log entries from multiple projects, use the logical-OR operator to join logName clauses.

   You can create up to 50 exclusion filters per sink. Note that the length of a filter can't exceed 20,000 characters.

10. Select Create sink.

11. To complete the configuration of your aggregated sink, grant the service account for the sink the permission to write log entries to your sink's destination. For more information, see Set destination permissions.

gcloud

To create an aggregated sink, use the logging sinks create command:

1. To create a sink, run the gcloud logging sinks create command, and include the --include-children flag.

   Before using the following command, make the following replacements:

   • SINK_NAME: The name of the log sink. You can't change the name of a sink after you create it.
   • SINK_DESTINATION: The service or project to where you want your log entries routed. For information about the format of these destinations, see Destination path formats.
   • INCLUSION_FILTER: The inclusion filter for a sink. For examples, see Filters for aggregated sinks.
   • FOLDER_ID: The ID of the folder. If you want to create a sink at the organization level, then replace --folder=FOLDER_ID with --organization=ORGANIZATION_ID.

   Run the gcloud logging sinks create command:

   gcloud logging sinks create SINK_NAME \
     SINK_DESTINATION  --include-children \
     --folder=FOLDER_ID --log-filter="INCLUSION_FILTER"
   

   To create an intercepting sink, also include the --intercept-children flag.

   For example, if you're creating a folder-level aggregated sink whose destination is a Pub/Sub topic, then your command might look like the following:

   gcloud logging sinks create SINK_NAME \
     pubsub.googleapis.com/projects/PROJECT_ID/topics/TOPIC_ID --include-children \
     --folder=FOLDER_ID --log-filter="logName:activity"

2. Grant the service account for the sink permission to write to your sink destination. For more information, see Set destination permissions.

REST

To create an aggregated sink, use the organizations.sinks.create or folders.sinks.create Logging API method. Prepare the arguments to the method as follows:

1. Set the parent field to be the Cloud de Confiance organization or folder in which to create the sink. The parent must be one of the following:

   • organizations/ORGANIZATION_ID
   • folders/FOLDER_ID

2. In the LogSink object in the method request body, do one of the following:

   • Set includeChildren to true.

   • To create an intercepting sink, also set the interceptChildren field to true.

3. Set the filter field to match the log entries that you want to include.

   For examples, see Filters for aggregated sinks.

   The length of a filter can't exceed 20,000 characters.

4. Set the remaining LogSink fields as you would for any sink. For more information, see Route logs to supported destinations.

5. Call organizations.sinks.create or folders.sinks.create to create the sink.

6. Grant the service account for the sink permission to write to your sink destination. For more information, see Set destination permissions.

Console

1. To get information about the service account for your sink, do the following:

   1. In the Cloud de Confiance console, go to the Log Router page:

      Go to Log Router

      If you use the search bar to find this page, then select the result whose subheading is Logging.

   2. Select more_vert Menu and then select View sink details. The writer identity appears in the Sink details panel.

   3. If the value of the writerIdentity field contains an email address, then proceed to the next step. When the value is None, you don't need to configure destination permissions.

   4. Copy the sink's writer identity into your clipboard. Writer identities might look different depending on what resource your sink belongs to, but are always prefixed by serviceAccount:. The following is an example of a writer identity:

      serviceAccount:service-123456789012@gcp-sa-logging.s3ns-system.iam.gserviceaccount.com
      

2. Grant the principal specified by the sink's writer identity the permission to write log data to the destination:

   1. In the Cloud de Confiance console, go to the IAM page:

      Go to IAM

      If you use the search bar to find this page, then select the result whose subheading is IAM & Admin.

   2. In the toolbar of the Cloud de Confiance console, select the project that stores the destination of the aggregated sink. When the destination is a project, select that project.

   3. Click person_add Grant access.

   4. Enter the principal specified by the sink's writer identity and then grant an IAM role:

      • For all destinations, grant the Logs Writer role (roles/logging.logWriter). Specifically, a principal needs the logging.logEntries.route permission.
      • Grant one of the following roles based on the destination:
        • Log bucket: Grant the Logs Bucket Writer role (roles/logging.bucketWriter).
        • Pub/Sub topic: Grant the Pub/Sub Publisher role (roles/pubsub.publisher).

gcloud

1. Ensure that you have Owner access on the Cloud de Confiance project that contains the destination. If you don't have Owner access to the destination of the sink, then ask a project owner to add the writer identity as a principal.

2. To get information about the service account for your sink, run the gcloud logging sinks describe command.

   Before using the following command, make the following replacements:

   • SINK_NAME: The name of the log sink. You can't change the name of a sink after you create it.

   Run the gcloud logging sinks describe command:

   gcloud logging sinks describe SINK_NAME
   

3. If the sink details contain a field labeled writerIdentity, then proceed to the next step. When the details don't include a writerIdentity field, you don't need to configure destination permissions for the sink.

4. Copy the sink's writer identity into your clipboard. The following illustrates a writer identity:

   serviceAccount:service-123456789012@gcp-sa-logging.s3ns-system.iam.gserviceaccount.com
   

5. Grant the sink's writer identity the permission to write log data to the destination by running the gcloud projects add-iam-policy-binding command.

   Before using the following command, make the following replacements:

   • PROJECT_ID: The identifier of the project. Select the project that stores the destination of the aggregated sink. When the destination is a project, select that project.
   • PRINCIPAL: An identifier for the principal that you want to grant the role to. Principal identifiers usually have the following form: PRINCIPAL-TYPE:ID. For example, principal://iam.googleapis.com/locations/global/workforcePools/my-pool/subject/my-user@example.com. For a full list of the formats that PRINCIPAL can have, see Principal identifiers.

   • ROLE: An IAM role. Grant the sink's writer identity an IAM role based on the destination of the log sink:

     • For all destinations, grant the Logs Writer role (roles/logging.logWriter). Specifically, a principal needs the logging.logEntries.route permission.
     • Grant one of the following roles based on the destination:
       • Log bucket: Grant the Logs Bucket Writer role (roles/logging.bucketWriter).
       • Pub/Sub topic: Grant the Pub/Sub Publisher role (roles/pubsub.publisher).

   Run the gcloud projects add-iam-policy-binding command:

   gcloud projects add-iam-policy-binding PROJECT_ID --member=PRINCIPAL --role=ROLE
   

REST

We recommend that you use the Cloud de Confiance console or the Google Cloud CLI to grant a role to the service account.