问题:bridge 网络的容器无法跨多个宿主机进行通信
环境信息:kubeadm安装v1.22.2,使用的cni插件为flannel,容器为containerd
解决方法:
删除containerd的网络
ls -la /etc/cni/net.d/
drwxr-xr-x. 2 1001 116 67 Jun 25 14:39 .
drwxr-xr-x. 3 1001 116 19 Jul 30 2021 ..
-rw-r--r-- 1 1001 116 604 Jun 25 12:00 10-containerd-net.conflist
-rw-r--r-- 1 root root 292 Jun 25 14:39 10-flannel.conflistmv /etc/cni/net.d/10-containerd-net.conflist /etc/cni/net.d/10-containerd-net.conflist.bak
ifconfig cni0 down && ip link delete cni0
systemctl daemon-reload
systemctl restart containerd kubelet
原因分析:
集群初始化的网络
10.244.0.0/16 为svc网络,10.96.0.0/16 为pod网络,
10.88.0.0/16为容器的网络并非cni插件flannel和k8s指定的网络,使用ip add查看网卡信息,确定10.88.0.0/16网络的网卡,可以查看到cni0的网络。
这也就造成了创建出来的pod,不能在节点之间的互相通信。
# /etc/cni/net.d是Kubernetes中CNI(Container Network Interface)插件的配置目录[root@k8s-node1 kubernetes]# cat /etc/cni/net.d/10-containerd-net.conflist
{"cniVersion": "0.4.0","name": "containerd-net","plugins": [{"type": "bridge","bridge": "cni0","isGateway": true,"ipMasq": true,"promiscMode": true,"ipam": {"type": "host-local","ranges": [[{"subnet": "10.88.0.0/16"}],[{"subnet": "2001:4860:4860::/64"}]],"routes": [{ "dst": "0.0.0.0/0" },{ "dst": "::/0" }]}},{"type": "portmap","capabilities": {"portMappings": true}}]
}