Move data between buckets

This page describes how to move the contents of a bucket by copying objects from a source bucket to a destination bucket.

Overview

When you create a bucket, some of the properties you set are permanent and cannot be changed, including the bucket's name and its parent project.

If you want to use a new name for your bucket, you can effectively do so by creating a new bucket with the name you want to use, and then copying your objects from the old bucket to the new bucket. You can then delete the old bucket. Note the following:

Required permissions

Console

In order to complete this guide using the Trusted Cloud console, you must have the proper IAM permissions. If the buckets you want to access exist in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for the Trusted Cloud console.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Command line

In order to complete this guide using a command-line utility, you must have the proper IAM permissions. If the buckets you want to access exist in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for gcloud storage commands.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

REST APIs

JSON API

In order to complete this guide using the JSON API, you must have the proper IAM permissions. If the buckets you want to access exist in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for JSON methods.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Move data from one bucket to another

Console

  1. If you don't have a destination bucket yet, create the bucket.

  2. Copy each object from your source bucket to the destination bucket.

  3. Delete your source bucket and its content.

Command line

  1. If you don't have one yet, create a destination bucket.

  2. To recursively copy the contents of your source bucket to your destination bucket, use the gcloud storage cp command with the --recursive option:

    gcloud storage cp --recursive gs://SOURCE_BUCKET/* gs://DESTINATION_BUCKET

    Where:

    • SOURCE_BUCKET is the name of your original bucket. For example, old-bucket.

    • DESTINATION_BUCKET is the name of the bucket you are moving your data to. For example, my-bucket.

    If your bucket uses managed folders and a flat namespace, you must include the flag --include-managed-folders in the command for the managed folders to be copied.

  3. To recursively delete the contents from the source bucket, along with the source bucket itself, use the gcloud storage rm command with the --recursive option:

    gcloud storage rm --recursive gs://SOURCE_BUCKET

    Where SOURCE_BUCKET is the name of your original bucket. For example, old-bucket.

    Alternatively, to delete the contents from the source bucket without deleting the source bucket itself, use the gcloud storage rm command with the --all-versions flag and ** wildcard:

    gcloud storage rm --all-versions gs://SOURCE_BUCKET/**

    Where SOURCE_BUCKET is the name of your original bucket. For example, old-bucket.

REST APIs

JSON API

  1. If you don't have a destination bucket yet, create the bucket.

  2. Copy each object from your source bucket to the destination bucket.

  3. Delete each object in your source bucket.

  4. Delete your source bucket.

XML API

  1. If you don't have a destination bucket yet, create the bucket.

  2. Copy each object from your source bucket to the destination bucket.

  3. Delete each object in your source bucket.

  4. Delete your source bucket.

What's next