In GKE version 1.32.3-gke.1927002 and later, GKE uses a container-optimized compute platform for the general-purpose Autopilot compute class. This platform improves Pod scheduling latency, especially during autoscaling operations. The container-optimized compute platform provides benefits like faster scaling reaction times and more precise capacity right-sizing. For more information about the general-purpose compute class, see About built-in compute classes in Autopilot clusters.
]]>GKE now provides insights and recommendations that help you to identify and troubleshoot clusters with Custom Resource Definitions that contain an invalid or malformed Certificate Authority bundle, which might disrupt cluster operations. Implementing the recommendation helps you to keep your clusters stable and performant.
]]>In GKE version 1.33 and later, the Compute Engine persistent disk CSI Driver supports provisioning Hyperdisk Balanced High Availability volumes in the ReadWriteOnce, ReadWriteOncePod, and ReadWriteMany access modes. For more information, see Provisioning Hyperdisk Balanced High Availability volumes.
In GKE version 1.32 and later, GKE Sandbox (gVisor) can now be configured with SYS_ADMIN privileges in GKE Autopilot. This lets you use Docker-in-Docker with gVisor in GKE Autopilot.
]]>GKE now provides insights and recommendations that help you identify workloads without resource requests or limits so that you can specify the resource needs for these workloads. Configuring CPU and memory requests and limits for containers is the best practice for improving reliability and performance, and is a necessary prerequisite for understanding and optimizing resource utilization by your workloads and their cost.
]]>Automatic application monitoring is now generally available in GKE versions 1.28 and later. When configured on GKE clusters, this feature automatically collects key metrics with Google Cloud Managed Service for Prometheus and provides out-of-the-box dashboards for monitoring the supported workloads. Automatic application monitoring supports six new AI model servers (NVIDIA Triton, vLLM, TGI, JetStream, TorchServe and TensorFlow Serving). For more information, see Configure automatic application monitoring.
]]>In version 1.32.1-gke.1729000 and later, you can customize specific kubelet and Linux kernel parameters like sysctls and huge pages by using the nodeSystemConfig field in your GKE compute classes. Additionally, you can now specify default values for fields that are omitted in individual rules in a compute class by using the priorityDefaults field. For details, see About custom compute classes.
In GKE version 1.32.2-gke.1652000 and later, new external LoadBalancer Services use zonal Network Endpoint Group (NEG) backends by default. This applies only to new backend service-based external LoadBalancer Services. Existing LoadBalancer Services are not affected. To learn more, see Create a backend service-based external load balancer.
Starting in GKE version 1.32.1-gke.1729000, Autopilot clusters will automatically use the new Performance HPA Profile. This new profile enables faster autoscaling on CPU and Memory metrics for up to 1,000 HorizontalPodAutoscaler objects by routing autoscaling metrics through the gke-metrics-agent Daemonset. If desired, users can revert to the old autoscaling profile by disabling the Peformance HPA Profile.
You can now monitor startup latency of Kubernetes workloads and nodes using the new Startup Latency dashboard available in the Observability tab on the Deployment details and Cluster details pages in the GKE Console. The dashboard is useful for tracking, troubleshooting and optimizing startup latency of your GKE workloads.
GKE now allows you to enable logging of Horizontal Pod Autoscaler decisions starting from GKE version 1.31.5-gke.1090000 or later, or version 1.32.1-gke.1260000 or later. These logs include atomic recommendations (based on individual metrics) and final recommendations (consolidated HPA decisions). The logs are stored in Cloud Logging and offer insights into the decision-making process of the Horizontal Pod Autoscaler.
]]>The europe-north2 region in Stockholm, Sweden is now available. For more information, see the Global Locations.
New recommendations of NODE_SA_MISSING_PERMISSIONS subtype are added to the portfolio of GKE Recommendations. Use the new recommendations to identify clusters with node service accounts missing IAM permissions that are critical for normal cluster operations.
If your organization has a policy to disable automatic role grants to default service accounts, the created default GKE node service account will not get the necessary permissions. Missing critical permissions can degrade your essential cluster operations, such as logging and monitoring.
]]>The GKE Autopilot partner program now lets partners create and manage allowlists that correspond to specific partner workloads. In GKE version 1.32.1-gke.1729000 and later, you can explicitly install allowlists in your clusters to run only the partner solutions that you need.
To learn more, see Run privileged workloads from GKE Autopilot partners.
]]>Three new metrics are added for checking node and node pool status:
kubernetes.io/node/status_condition: The condition of a node from the node status condition field. The Ready field has Unknown status if the node controller has not heard from the node in the last node-monitor-grace-period period. This metric is available for clusters with GKE version 1.32.1-gke.1260000 and later.
kubernetes.io/node_pool/multi_host/available: The multi-host NodePool availability. When all the nodes in the node pool are available, the value is True. If any of the nodes in the node pool are unavailable, the value is False. This metric is available for Multi-host TPU node pools only.
kubernetes.io/node_pool/status: The current status of the node pool from the NodePool instance. Status updates happen after GKE API operations complete. This metric is available for Multi-host TPU node pools only.
GKE automatically adds the following resource labels to node pools:
goog-gke-accelerator-type: The accelerator type used in the node pool.goog-gke-tpu-node-pool-type: The TPU node pool type, which can be single-host or multi-host.goog-gke-node-pool-provisioning-model: The provisioning model of the node pool. The nodes can be on demand, by reservation, or Spot VMs. To learn more, see Automatically applied labels.
GKE Managed NVIDIA Data Center GPU Manager (DCGM) Metrics Package is now generally available for both GKE Standard and Autopilot clusters running version 1.32.0-gke.1764000 and later. You can enable the feature via the Console, gcloud, or Terraform. Starting with cluster version 1.32.1-gke.1357000, GKE Managed NVIDIA DCGM will be default-on for new clusters.
GKE Managed DCGM provides a curated set of metrics for monitoring the utilization, performance, and health of NVIDIA GPUs. These metrics are collected by Google Cloud Managed Service for Prometheus and you can view the metric charts in the Observability Tab on the Kubernetes Clusters page or in Cloud Monitoring. For more information, see Collect and view DCGM metrics.
To learn more, see Collect and view DCGM metrics.
]]>GKE cluster notifications have the following new capabilities:
For more details about the different types of cluster notifications GKE sends and how you can receive them, see Cluster notifications.
]]>User-managed firewall rules for GKE LoadBalancer Services is now generally available on GKE clusters running version 1.31.3-gke.1056000 or later. By allowing user-managed firewall rules for GKE LoadBalancer Services, advanced firewall policies can now be configured to control ingress traffic to your GKE Services exposed with passthrough network load balancers. To learn more, see User-managed firewall rules for GKE LoadBalancer Services.
You can now customize a node system configuration with the following new kubelet and sysctl configuration options:
Kubelet
Sysctl
To learn more, see Kubelet configuration options and Sysctl configuration options.
]]>You can now use A3 Ultra VM powered by NVIDIA H200 Tensor Core GPUs with our new Titanium ML network adapter, which delivers non-blocking 3.2 Tbps of GPU-to-GPU traffic with RDMA over Converged Ethernet (RoCE).
A3 Ultra VMs are generally available in the a3-ultragpu-8g machine type and can be used through both the modes of operation in Google Kubernetes Engine (GKE):
Trillium, our sixth-generation TPU, is now generally available. Support is available for GKE Standard clusters in version 1.31.1-gke.1846000 or later, and Autopilot clusters in version 1.31.2-gke.1384000 or later. You can use TPU Trillium in the us-east5-b, europe-west4-a, us-east1-d, asia-northeast1-b, and us-south1-a zones.
To learn more, see Benefits of using TPU Trillium.
Cloud DNS additive VPC scope is now generally available on GKE clusters running version 1.28.3-gke.1430000 or later. You can now configure your GKE clusters to add GKE headless service entries to your Cloud DNS private zone visible from your VPC networks, on top of using Cloud DNS (cluster scope) as your GKE DNS provider.
To learn more, read Cloud DNS scopes for GKE.
]]>The C4A machine family is generally available in the following versions:
Standard clusters in version 1.28.13-gke.1024000, 1.29.8-gke.1057000, 1.30.4-gke.1213000 or later. To use this family in GKE Standard, you can use the --machine-type flag when creating a cluster or node pool.
Autopilot clusters in 1.28.15-gke.1344000, 1.29.11-gke.1012000, 1.30.7-gke.1136000, 1.31.3-gke.1056000 or later. To use this family in GKE Autopilot, schedule your workloads along with the kubernetes.io/machine-family: c4a node selector. In versions 1.31 or above, the kubernetes.io/arch: arm64 node selector would default to C4A machine family.
Cluster autoscaler and node auto-provisioning are supported in 1.28.15-gke.1344000, 1.29.11-gke.1012000, 1.30.7-gke.1136000, 1.31.3-gke.1056000 or later.
Local SSD support is available for Public Preview from 1.31.1-gke.2008000. Contact your Account Team to participate in the preview.
GKE now provides insights and recommendations that help you identify and amend clusters running a minor version that reached end of standard support, clusters with nodes in violation of version skew policy, and clusters without a maintenance window to achieve reliable operations, up-to-date security posture and supportability.
]]>In GKE version 1.31.1-gke.2105000 or later, you can now configure custom compute classes to consume Compute Engine reservations. Workloads that use those custom compute classes automatically trigger reservation consumption during node creation. This lets you manage reservation consumption more centrally. To learn more, see About custom compute classes.
]]>Cloud TPU Trillium (v6e) machine types are now in public preview for Autopilot clusters running version 1.31.2-gke.1384000 or later. These TPUs are available in the following zones: us-east5-b, europe-west4-a, us-east1-d, asia-northeast1-b, and us-south1-a. To learn more, see Plan TPUs in GKE.
Cluster autoscaler and node auto-provisioning support the C4 machine family in GKE version 1.28.15-gke.1159000, 1.29.10-gke.1227000 or later.
]]>You can now specify a custom resource policy as a compact placement policy with node auto-provisioning in clusters running GKE version 1.31.1-gke.2010000 or later. To learn more, see Use compact placement for node auto-provisioning.
]]>GKE version 1.31 introduces increased scalability, allowing users to create clusters with up to 65,000 nodes. For clusters exceeding 5,000 nodes, a quota increase is required. Contact Google Cloud support to request this increase.
]]>Performance horizontal Pod autoscaling (HPA) profile is now available in Preview for new and existing GKE clusters running version 1.31.2-gke.1138000 or later. This feature speeds up HPA reaction time and enables quick recalculation of up to 1,000 HPA objects. To learn more, see Configuring Performance HPA profile.
]]>DNS-based access for GKE clusters control plane is now generally available. This capability provides each cluster with a unique domain name system (DNS) name or fully-qualified domain name (FQDN). Access to clusters is controlled through IAM policies, eliminating the need for bastion hosts or proxy nodes. Authorized users can connect to the control plane from different cloud networks, on-prem deployments, or from remote locations, without relying on proxies.
To learn more, see About network isolation in GKE.
]]>GKE clusters running version 1.28 or later now support automatic application monitoring in public preview. Enabling this feature automatically deploys PodMonitoring configurations to capture key metrics for supported workloads like Apache Airflow, Istio, and RabbitMQ. These metrics are integrated with Cloud Monitoring dashboards for observability. To learn more, see Configure automatic application monitoring for workloads.
The GKE Volume Populator is generally available on GKE clusters running version 1.31.1-gke.1729000 or later. This feature provides a way to automate data transfer from a Google Cloud Storage bucket source storage to a destination PersistentVolumeClaim backed by a Parallelstore instance. To learn more, see Transfer data from Cloud Storage during dynamic provisioning using GKE Volume Populator.
]]>Cloud TPU v6e machine types are now in public preview for GKE clusters running version 1.30.4-gke.1167000 or later. These TPU VMs (ct6e-standard) are available in the following zones: us-east5-b, europe-west4-a, us-east1-d, asia-northeast1-b, and us-south1-a. To learn more, see Plan TPUs in GKE.
Generally available: In GKE version 1.26 and later, Hyperdisk Balanced volumes can be created in Confidential mode for custom boot disks and persistent volumes and attached to Confidential GKE Nodes.
]]>GKE control plane authority is now generally available with version 1.31.1-gke.1846000 or later. GKE control plane authority provides enhanced visibility, security controls, and customization of the GKE control plane. For more information, see the About GKE control plane authority.
For GKE clusters running version 1.31.1-gke.1146000 or later, Cloud Tensor Processing Unit (TPU) v3 machine types are generally available. These TPU VMs (ct3-hightpu-4t and ct3p-hightpu-4t) are currently available in us-east1-d, europe-west4-a, us-central1-a, us-central1-b, and us-central1-f. To learn more, see TPUs in GKE.