AI Hypercomputer release notes

Generally available: You can troubleshoot workloads with slow performance by using straggler detection metrics and logs.

Stragglers are single-point, non-crashing failures that eventually slow down your entire workload. Large-scale ML workloads are very susceptible to stragglers, and VMs with stragglers are often very difficult to notice and pinpoint without straggler detection.

For more information, see Monitor VMs and Slurm clusters and Troubleshoot slow performance.

Generally available: You can now manage the Collective Communication Analyzer (CoMMA), a library that uses the NVIDIA Collective Communication Library (NCCL) profiler plugin to collect detailed NCCL telemetry for GPU machine types. The collected performance metrics and operational events are used for analyzing and optimizing large-scale AI and ML training workloads.

CoMMA is automatically installed and enabled on A4X, A4 High, and A3 Ultra machine types when using specific images. You can manage this data collection by disabling the plugin, adjusting its data granularity levels, or manually installing it on other GPU machine types. For more information, see Enable, disable, and configure CoMMA.

Preview: You can use future reservations in calendar mode to obtain resources for up to 90 days. By creating a request in calendar mode, you can reserve up to 80 GPU VMs for a future date and time. Then, you can use that capacity to run the following workloads:

• Model pre-training

• Model fine-tuning

• Simulations

• Inference

For more information, see Choose a consumption option.

Generally available: You can apply a workload policy in a managed instance group (MIG) to specify the type of the workload to run on the MIG. Workload policies help improve the workload performance by optimizing the underlying infrastructure. The supported type, high-throughput, is ideal for workloads that require high networking performance. For more information, see Workload policy for MIGs.

Generally available: You can proactively manage upcoming maintenance host events on your reserved blocks of capacity, whether VMs are running on them or not. This approach helps you minimize disruptions and maintain optimal performance. For more information, see Manage host events across reservations.

Preview: You can use the Flex-start consumption option to obtain resources for up to seven days. Flex-start provisions capacity from a secured resource pool. Using this feature increases your chance to obtain high-demand resources like GPUs. For more information, see Choose a consumption option.

Generally available: The A4 accelerator-optimized machine type is now generally available. A4 VMs are powered by NVIDIA B200 GPUs and provide up to 3x performance of previous GPU machine types for most GPU accelerated workloads. A4 is especially recommended for ML training workloads at large scales. A4 is available in the following region and zone:

• Council Bluffs, Iowa: us-central1-b

When provisioning A4 machine types, you can use Hypercompute Cluster to request capacity and create VMs or clusters. To get started see Overview of creating VMs and clusters.

Software stack updates

The following new Docker images are also released to support workloads running on your A4 GKE clusters that are deployed using Hypercompute Cluster.

• NeMo docker image: nemo25.02-gib1.0.5-A4
• MaxText docker image: jax-maxtext-gpu:jax0.5.1-cuda_dl25.02-rev1-maxtext-20150317

For more information, see AI Hypercomputer images.

Generally available: The A3 Ultra accelerator-optimized machine type is now generally available. A3 Ultra VMs are powered by NVIDIA H200 Tensor Core GPUs and support the new Titanium ML network adapter, which delivers non-blocking 3.2 Tbps of GPU-to-GPU traffic with RDMA over Converged Ethernet (RoCE). A3 Ultra VMs are ideal for foundation ML model training and serving. The A3 Ultra machine type is available in the following region and zone:

• St. Ghislain, Belgium, Europe - europe-west1-b

When provisioning A3 Ultra machine types, you must use Hypercompute Cluster to request capacity and create VMs or clusters. To get started see Overview of creating VMs and clusters in the AI Hypercomputer documentation.

Preview: Hypercompute Cluster is now available in preview. With Hypercompute Cluster, you can streamline the provisioning of up to tens of thousands of A3 Ultra accelerator-optimized machines.

With features such as dense co-location of resources, ultra-low latency networking, targeted workload placement, and advanced maintenance controls to minimize workload disruptions, Hypercompute Cluster is built to deliver exceptional performance and resilience, so you can run your most demanding AI, ML, and HPC workloads with confidence.

To get started, review the overview for VM and cluster creation.