Maximum transmission unit

The maximum transmission unit (MTU) is the size, in bytes, of the largest possible IP packet, including IP headers, layer 4 protocol headers, and layer 4 data, that can fit inside an Ethernet frame.

Valid VPC network MTU sizes

Virtual Private Cloud (VPC) networks use a default MTU of 1,460 bytes. You can set a VPC network's MTU to any value between 1,300 bytes and 8,896 bytes (inclusive). Common custom MTU sizes are 1,500 bytes (standard Ethernet) or 8,896 bytes (the maximum possible). We recommend that you configure the MTU for each virtual machine (VM) instance network interface (NIC) to match the MTU of the VPC network to which it is connected. For more information, see VMs and MTU settings.

Communication between Google Cloud VMs within VPC networks

When sending and receiving VMs use the same VPC network or peered VPC networks that have identical MTUs, IP packets up to the MTU size can be sent between the two VMs, if the interfaces for both VMs are configured to use the VPC network MTU.

To avoid MTU mismatch issues, we recommend that you use the same MTU for all of your connected VPC networks. While that is the recommended practice, you are not forced to have identical MTUs among connected VPC networks. For details about how protocols handle situations where there is a MTU mismatch between VPC networks, see Mismatched MTUs, MSS clamping, path MTU discovery.

From the perspective of a sending VM, paths to the following destinations represent VM-to-VM traffic routed within a VPC network:

• A regional internal IPv4 address in a subnet primary IPv4 or subnet secondary IPv4 address range, including private IPv4 address ranges and privately used public IPv4 address ranges, used by these destination resources:
  • The primary internal IPv4 address of a receiving VM's network interface (NIC).
  • An internal IPv4 address in an alias IP range of a receiving VM's NIC.
  • An internal IPv4 address of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
• Internal IPv6 subnet address ranges used by these destination resources:
  • An IPv6 address from the /96 IPv6 address range assigned to a receiving VM's NIC.
  • An IPv6 address from the /96 IPv6 address range of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
• External IPv6 subnet address ranges used by these destination resources when packets are routed using subnet routes or peering subnet routes within the VPC network:
  • An IPv6 address from the /96 IPv6 address range assigned to a receiving VM's NIC.
  • An IPv6 address from the /96 IPv6 address range of an external forwarding rule for either protocol forwarding or for an external passthrough Network Load Balancer.

The following VM-to-VM paths are treated in the same way as Communication to destinations outside of a VPC network:

• If the packet destination is an external IPv4 address of a receiving Google Cloud VM's NIC.
• If the packet destination is an external IPv4 address of a external passthrough Network Load Balancer.
• If the packet destination is an external IPv4 address of a forwarding rule for protocol forwarding
• If the packet destination is an external IPv6 address of a Google Cloud VM's NIC, external passthrough Network Load Balancer, or forwarding rule for external protocol forwarding and the applicable route in the VPC network uses a default internet gateway next hop. In this scenario, receiving VMs are neither in the same VPC network as the sending VM nor in a VPC network connected to the sending VM's VPC network using VPC Network Peering.

Communication to destinations outside of a VPC network

Google Cloud processes packets sent to destinations outside of the sending VM's VPC network as shown in the following table. Destinations outside of a sending VM's VPC network include publicly routable IP addresses for resources outside of Google Cloud and customer-usable external IP addresses within Google Cloud.

Because the internet generally uses an MTU of 1,500 bytes, keeping IP packet size at 1,500 bytes or less usually avoids MTU-related packet loss.

Communication to Google APIs and services

VMs using any valid VPC network MTU size can send packets to Google APIs and services, including using Private Google Access and Private Service Connect for Google APIs. The details in this section also apply to on-premises resources that send packets to Google APIs and services using Private Google Access for on-premises hosts.

The traffic path to Google APIs and services described in this section is implemented by Google Front Ends (GFEs). These GFEs use non-configurable, fixed MTUs. Traffic from Google Cloud to Google APIs and services always uses the TCP protocol: If a VM connects to Google APIs and services from a VPC network whose MTU does not match the MTU of the GFE, the segment size is negotiated by using TCP MSS advertisement as described in Mismatched MTUs, MSS clamping, path MTU discovery.

Communication through Cloud VPN tunnels

Cloud VPN has both a gateway MTU for encapsulated packets and a payload MTU for packets before and after encapsulation.

For precise payload MTU values and other Cloud VPN MTU information, see MTU considerations in the Cloud VPN documentation.

Communication through Cloud Interconnect (VLAN) attachments

We recommend that you use the same MTU for all VLAN attachments that are connected to the same VPC network, and that you set the MTU of the VPC network to the same value. For details about Cloud Interconnect VLAN attachment MTUs, see Cloud Interconnect MTU.

Jumbo frame support

The following table summarizes jumbo frame support among Google Cloud products and features:

VMs and MTU settings

As a best practice, match a VM's NIC MTU to the MTU of the VPC network to which the NIC is connected:

• Each NIC MTU for a Linux VM based on a public OS image is automatically set to the respective VPC network MTU by using DHCP Option 26.

• Each NIC MTU for a Windows VM based on a public OS image is configured with a fixed MTU of 1,460 bytes. If you change the MTU of a VPC network that contains Windows VMs based on public OS images, you must change the MTU setting of a Windows VM.

• If you use custom guest OS images, you must configure NIC MTUs or verify that the guest OS accepts the VPC network MTU by using DHCP Option 26.

• If a VM has multiple network interfaces, set each NIC MTU to the respective VPC network MTU.

• If a NIC MTU must differ from the VPC network MTU, set the NIC MTU to less than the VPC network MTU. Forcibly decreasing the NIC MTU is advantageous for some advanced networking scenarios.

Changing the MTU of a VPC network

If you change the MTU of a VPC network with running VMs, keep the following considerations in mind:

• If you reduce the VPC network MTU, you must stop and start each VM. Rebooting a VM from within its guest operating system does not update its MTU.

• If you increase the VPC network MTU, running VMs using the VPC network won't take advantage of the increased VPC network MTU until the VMs have been stopped and started. Until each VM has been stopped and restarted, the VM continues to use the previous (lower) MTU value.

For instructions, see Change the MTU setting of a VPC network.

GKE and MTU settings

The MTU selected for a Pod interface is dependent on the Container Network Interface (CNI) used by the cluster Nodes and the underlying VPC MTU setting. For more information, see Pods.

The Pod interface MTU value is either 1460 or inherited from the primary interface of the Node.

Mismatched MTUs, MSS clamping, path MTU discovery

TCP protocol

The TCP protocol handles MTU mismatches automatically. Both the client and server individually calculate their own effective TCP maximum segment size (MSS) values each time a TCP connection is opened. The client and server don't have to agree on an identical effective MSS value.

• Effective TCP maximum segment size (MSS) of the client: The largest amount of transmissible data in a TCP segment sent from a client to a server is the minimum of the following two values:

  • The value of the MSS field in the SYN-ACK packet received by the client from the server during TCP connection establishment.

  • The MTU of the client's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.

• Effective TCP maximum segment size (MSS) of the server: The largest amount of transmissible data in a TCP segment sent from a server to a client is the minimum of the following two values:

  • The value of the MSS field in the SYN packet received by the server from the client during TCP connection establishment.

  • The MTU of the server's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.

TCP MSS clamping

TCP MSS clamping is a process where a network device between a client and server changes the MSS values in SYN and SYN-ACK packets as it routes the packets between the client and server. Google Cloud uses MSS clamping whenever it sends packets to destinations outside of a VPC network.

Cloud VPN tunnels and Cloud Interconnect VLAN attachments also use MSS clamping. For more information, see Packet encapsulation and processing in the Cloud VPN documentation and Cloud Interconnect MTU.

VPC networks don't perform MSS clamping for packets routed by the next hops within a VPC network because the TCP protocol itself is sufficient.

Non-TCP protocols

Other protocols, such as UDP, require special care when two different VPC network MTUs are involved. It is the responsibility of a sending system to emit packets that fit within its network interface MTU, the MTU of the receiving system's network interface, and the MTU of all networks in between. Google Cloud does not perform IP fragmentation for packets routed by the next hops within a VPC network.

When an IP packet is too large to be delivered—for example, when the packet exceeds the MTU of the VPC network where the receiving VM's NIC is located— Google Cloud drops the packet. If the packet has the DF bit set, Google Cloud also sends a Fragmentation Needed (ICMP over IPv4) or Packet Too Big (ICMPv6) message back to the sender.

Google Cloud sends a Fragmentation Needed or Packet Too Big message in the following circumstances, even when a DF bit is off:

• If the VPC network MTU is less than 1,600 bytes, and the packet being sent exceeds the VPC network MTU.
• If the VPC network MTU is 1,600 bytes or more, and the packet being sent exceeds 1,600 bytes.

ICMP Fragmentation Needed or Packet Too Big messages are necessary for a VM that is sending packets to use Path MTU discovery (PMTUD). To illustrate how PMTUD works, consider the following example with two VMs in different VPC networks that are connected by using VPC Network Peering:

• The sending VM has a NIC in a VPC network whose MTU is 8,896 bytes.
• The receiving VM has a NIC in a VPC network whose MTU is 1,460 bytes.
• The sending VM emits an 8,000-byte IP packet whose Don't Fragment (DF) bit is set. Because the packet is too large to be delivered to the receiving VM, Google Cloud sends a Fragmentation Required or Packet Too Big message to the sending VM. This message indicates the largest possible IP packet size that the sender can use when it attempts to re-transmit packets for the connection.
• The sending VM's operating system uses this information to lower the IP packet size when sending subsequent packets to the receiving VM.

PMTUD has the following additional requirements because PMTUD-generated Fragmentation Needed or Packet Too Big packets use the ICMP protocol and have sources that match an original packet's destination:

• You must configure ingress allow VPC firewall rules or rules in firewall policies such that ICMP (for IPv4) or ICMPv6 (for IPv6) are allowed from sources that match the original packet destinations. To simplify firewall configuration, consider allowing ICMP and ICMPv6 from all sources.
• Forwarding rules for internal passthrough Network Load Balancer and internal protocol forwarding must use the L3_DEFAULT protocol so that they process both ICMP for PMTUD and the protocol used by the original packet.

What's next

• To see a different MTU working, see Create and verify a jumbo frame MTU network.
• Create a VPC network with a specified MTU.
• Change the MTU setting of a VPC network.