Create a TPU VM instance

This document describes how to create a TPU virtual machine (VM) instance. A TPU VM is also called a single-host TPU slice.

To create a group of single-host slices, see Create a MIG with single-host TPU slices.

Before you begin

  • If you haven't already, set up authentication. Authentication verifies your identity for access to Cloud de Confiance by S3NS services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    1. Install the Google Cloud CLI, and then sign in to the gcloud CLI with your federated identity. After signing in, initialize the Google Cloud CLI by running the following command: 

      gcloud init
    2. Set a default region and zone.

Prerequisites

Before you proceed to create a TPU instance, you must do the following:

  1. Choose the TPU version: select the TPU version that is suitable for your workload. For a list of TPU versions by workload type, see Recommended TPU versions by workload types.

  2. Validate TPU availability in your preferred location: TPUs are available in specific Cloud de Confiance by S3NS regions. To use a TPU version, ensure its availability in your preferred region. For a list of TPU locations, see TPU availability.

  3. Ensure that your project has sufficient TPU quota: if you are creating a TPU instance with on-demand or Spot VMs, you must have sufficient TPU quota available in the region that you want to use. Creating a TPU instance that consumes a TPU reservation doesn't require any TPU quota as quota is used when the reservation was created. For a list of TPU quota names, see TPU quota and for instructions on how to view the quota, see View and manage quotas

  4. Choose a TPU consumption option: select a consumption option that best fits your workload, its duration, and your cost needs. For a list of consumption option availability by TPU versions, see TPU consumption options.

Create a TPU VM instance

The parameters you use to create a TPU VM instance depend on the consumption option you are using: on-demand, Spot, reservation-bound, or flex-start. For more information, see About VM provisioning models .

Create an on-demand TPU VM instance

To create an on-demand TPU VM instance, use the gcloud compute instances create command:

  gcloud compute instances create TPU_NAME \
    --machine-type=MACHINE_TYPE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT \
    --zone=ZONE \
    --maintenance-policy=TERMINATE

Replace the following placeholders:

  • TPU_NAME: A name for your TPU VM.
  • MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
  • IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
  • IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
  • ZONE: The zone for the TPU VM (for example us-central1-b).

Create a TPU Spot VM instance

To create a TPU Spot VM instance, use the gcloud compute instances create command with the --provisioning-model=SPOT flag:

  gcloud compute instances create TPU_NAME \
    --machine-type=MACHINE_TYPE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT \
    --zone=ZONE \
    --provisioning-model=SPOT \
    --instance-termination-action=DELETE \
    --maintenance-policy=TERMINATE

Replace the following placeholders:

  • TPU_NAME: A name for your TPU VM.
  • MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
  • IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
  • IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
  • ZONE: The zone for the TPU VM (for example us-central1-b).

Create a TPU VM instance using a reservation

To create a TPU VM instance using the reservation-bound consumption option, use the gcloud compute instances create command with the --reservation-affinity=specific and --reservation flags:

  gcloud compute instances create TPU_NAME \
    --machine-type=MACHINE_TYPE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT \
    --zone=ZONE \
    --provisioning-model=reservation-bound \
    --reservation-affinity=specific \
    --reservation=RESERVATION_NAME \
    --instance-termination-action=DELETE \
    --maintenance-policy=TERMINATE

Replace the following placeholders:

  • TPU_NAME: A name for your TPU VM.
  • MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
  • IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
  • IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
  • ZONE: The zone for the TPU VM (for example us-central1-b).
  • RESERVATION_NAME: The name of your reservation.

Create a TPU Flex-start VM

To create a TPU Flex-start VM instance, use the gcloud compute instances create command with the --provisioning-model=FLEX_START flag:

  gcloud compute instances create TPU_NAME \
    --machine-type=MACHINE_TYPE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT \
    --zone=ZONE \
    --provisioning-model=FLEX_START \
    --max-run-duration=MAX_RUN_DURATION \
    --request-valid-for-duration=VALID_DURATION \
    --instance-termination-action=DELETE \
    --maintenance-policy=TERMINATE

Replace the following placeholders:

  • TPU_NAME: A name for your TPU VM.
  • MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
  • IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
  • IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
  • ZONE: The zone for the TPU VM (for example us-central1-b).
  • MAX_RUN_DURATION: The maximum run duration for the TPU VM (for example 6h).
  • VALID_DURATION: The maximum duration for which the request is valid.

For more information about creating Compute Engine instances, see Create and start a Compute Engine instance.

Limitations

  • Lifecycle operations: You can't stop, start, resume, or suspend TPU instances. To change configurations that require a restart or to stop incurring charges, you must delete the instances.

  • Update VM details: You can update the properties that only require a refresh (REFRESH) action. If a configuration requires a restart (RESTART), then you must recreate the instance.

What's next