Version 1.11. This version is no longer supported. For information about how to upgrade to version 1.12, see Upgrading Anthos on bare metal in the 1.12 documentation. For more information about supported and unsupported versions, see the Version history page in the latest documentation.

Implement flat IPv4 mode network model

Overview

In a flat-mode network model, pods have unique IP addresses across clusters. Ensure that the pod CIDRs assigned are unique and don't overlap with any other subnets. For example, IP addresses can't overlap with IP addresses used for the nodes or the other pod CIDRs in other clusters. These IP addresses can be accessed externally and hence pods on any node can communicate with all pods on all other nodes. Communication from the pod to any external IP address doesn't require network address translation (NAT). For more information about the flat mode network model and how it compares with the default, island network model, see Flat vs island mode network models.

Flat-mode network models are of two types━static mode network and dynamic mode network (using Border Gateway Protocol). Static flat-mode can be used when nodes span a single Layer 2 domain. For nodes spanning across multiple Layer 2 domains, use flat IP mode with BGP.

Use a flat-mode network model when you have a large IP address space and you can assign a unique pod CIDR for a cluster. You can configure the pod CIDRs using the ClusterCIDRConfigs dynamically. You can add or delete ClusterCIDRConfigs after the cluster is created.

For more information on flat-mode with BGP, see Implement flat-mode network model with BGP support.

Understanding the pod IP address reachability

In static flat network mode for IPv4, Pod IP address reachability is based on Address Resolution Protocol (ARP) packets. Therefore, Pods IP addresses are reachable only when the Pods are in the same Layer 2 domain. The nodes must belong to the same Layer 2 domain. The IP addresses you specify for your Pods (using ClusterCIDRConfigs) must be in the same subnet as the cluster nodes. Pods CIDRs configured must be from the nodes' subnet. For example, 222.1.0.0/16 subnet is used by the nodes in a cluster then select a smaller subnet within the subnet for the pods, 222.1.2.0/24. Ensure that no other resource in your cluster is using an IP address from the range allocated for your pods.

Following section describes the configuration for flat-mode networks for IPv4.

How to implement a static flat-mode network

By default, Google Distributed Cloud cluster is created in island-mode networking. This section describes how to set up flat-mode networking for your cluster.

To deploy a cluster with a flat-mode network model, make the following changes to the cluster configuration file:

Flat-mode networking can be enabled for a cluster during cluster creation only. To create a new cluster with flat-mode networking, use the following steps:

Edit the cluster configuration file to add clusterNetwork.flatIPv4 and set it to true.

When you enable flat-mode networking, the pod CIDR specified in the cluster configuration file (clusterNetwork.pods.cidrBlocks) is ignored.
Append a ClusterCIDRConfig manifest to the cluster configuration file.

In the ClusterCIDRConfig manifest, include the following information:
- metadata.namespace: the namespace of your cluster.
- spec.ipv4.cidr: the range of IP addresses in CIDR block format to use for Pods in your cluster. This range must come from the same subnet as the cluster nodes.
- perNodeMaskSize: Cluster creation preflight checks verify that the perNodeMaskSize value is sufficient to provision the number of pods specified in maxPodsPerNode.
- nodeSelector: If no node labels match the nodeSelector value, the node reconciliation remains pending and cluster creation doesn't complete.

The following excerpt of a cluster configuration file shows how to implement flat-mode networking without BGP support. The CIDRs that appear in this excerpt are only examples and you will need to replace them with your own CIDRs. When replacing the CIDRs with your own, ensure that they satisfy the criteria for pod reachability as specified in Understanding the pod IP address reachability.

---
apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: flat-mode
  namespace: cluster-flat-mode
spec:
... (other cluster config omitted)

...
  # Cluster networking configuration
  clusterNetwork:
    flatIPv4: true
    services:
      cidrBlocks:
      - 10.96.0.0/12
... (other cluster config omitted)

...
---
apiVersion: baremetal.cluster.gke.io/v1alpha1
kind: ClusterCIDRConfig
metadata:
  name: cluster-wide-1
  namespace: cluster-flat-mode
spec:
  ipv4:
    cidr: "222.1.0.0/16"
    perNodeMaskSize: 24

Limitations

The static flat-mode network for Google Distributed Cloud comes with the following limitations:

Pods using flat-mode networks would be reachable within the single Layer 2 domain. Any other machine which is not in the cluster, but in the same Layer 2 domain can also reach the Pods. This limitation exists for IPv6 as well when dualstack clusters are created and when IPv6 is in flat-mode without BGP. For more information, see Understanding the pod IP address reachability.
The Google Distributed Cloud IPAM controller tracks the IP address availability within the configured pod CIDRs. It does not track the IPs already in use by other devices. Hence, any other IPs in the Layer 2 domain must not interfere with the POD CIDRs. For more information, see Understanding the pod IP address reachability.
IP address masquerading is disabled when you use flat IPv4 mode. Because the egress NAT gateway feature depends on IP masquerading, you can't use egress NAT gateway in conjunction with flat IPv4 mode.