Network CIDR, Network IP Block and Reserved IP Blocks in Enterprise PKS

Before you install the VMware Enterprise PKS on vSphere with NSX-T, you should plan the CIDRs and IP blocks that you will be using in your deployment. It’s always confusing when you have so many CIDR’s and Blocks to be prepared before installing PKS. In this blog post, I will discuss about the Network CIDR’s, IP Blocks and Reserved IP blocks in PKS that you should consider while deploying/configuring your VMware Enterprise PKS environment.

Network CIDRs :

Before installing Enterprise PKS, you should plan and create the following CIDR’s which you will use in the configuration on PKS.

1) VTEP CIDR :

This is the network that will host the GENEVE tunnel endpoints on the NSX-T transport nodes. Make sure you have enough number of network IP available to configure all your ESXi and Edge transport nodes.

2)PKS MANAGEMENT CIDR :

This network will be used for the VMware Enterprise PKS management virtual machines like operations manager, bosh director, PKS vm and Harbor virtual machine. This can be a small network segment as there will be only a few number of management vm’s that we will deploy.

3)PKS LB CIDR :

PKS LB CIDR is nothing but your floating IP pool that you will be using to configure the load balancing for address space for the Kubernetes clusters which will be created by Enterprise PKS. The Kubernetes API access and exposed service gets the IP address from this pool.

IP Blocks :

Before starting with configuring the Enterprise PKS, you should be prepared with the below mentioned IP blocks.

1)Pod IP Block :

Whenever a namespace is created in a Kubernetes cluster, a subnet of IP is allocated from the POD IP BLOCK. This block will be used by the NSX-T Container Plug-in (NCP) who will assign the address space to the pods created by the Kubernetes. It is recommended to using /16 network for the pod ip block so that there will be enough IP address available for all the pods created.

2)Node IP Block :

Node IP Block is used by Enterprise PKS to provide IP address to the nodes(master and worker) in the Kubernetes clusters. For Node IP block also the recommended size is a /16 network so that there is enough IP address available for the nodes

Reserved IP Blocks :

There are certain set of IP CIDR range which are reserved and should not be used while configuring the above mentioned IP Blocks or Network CIDR’s. The below are the list of IP CIDR range that should NOT be used.

Docker daemon :

The below CIDR ranges are used by docker daemon in the Kubernetes worker nodes and hence they are reserved and should not be used.

• 172.17.0.1/16
• 172.18.0.1/16
• 172.19.0.1/16
• 172.20.0.1/16
• 172.21.0.1/16
• 172.22.0.1/16

Harbor :

If customer has plans of installing VMware Harbor which is an open source cloud native registry that stores, signs, and scans container images for vulnerabilities, the below mentioned IP CIDR’s should not be used.

• 172.18.0.0/16
• 172.19.0.0/16
• 172.20.0.0/16
• 172.21.0.0/16
• 172.22.0.0/16

Kubernetes service :

The below subnet will be used by Kubernetes cluster for Kubernetes services and hence it should not be used which configuring IP blocks.

• 10.100.200.0/24

In addition to the above mentioned IP ranges, 172.17.0.0/16 subnet should not be used by any of the PKS management virtual machines like OpsMan, BOSH Director,PKS control plane VM or Harbor vm.

NOTE: If you create Kubernetes clusters with any of the blocks listed above, the Kubernetes worker nodes will not be able to reach Harbor or internal Kubernetes services