在检查k8S节点系统日志时发现,有大量的kubelet组件报错,报错内容如下:
Jun 16 09:51:37 master kubelet: W0616 09:51:37.076817 6997 container.go:549] Failed to update stats for container "/kubepods/besteffort/pod022cc533-14a1-43a8-932d-970d30a65f09/c786acf49e946d3a38d5ee1a095791bc5a76db1c23ecc10e20f68c9021b22faa": unable to determine device info for dir: /var/lib/docker/overlay2/837c03201884a6a631ae2137a099f6c217142857ac6e02dd6065de43f0b61342/diff: could not find device with major: 252, minor: 0 in cached partitions map, continuing to push stats
之前换过一次docker的数据目录,可能存在部分容器无法更新状态的情况,解决方法如下:
通过docker system prune清楚无法更新状态的容器,然后重启kubelet与docker组件。
# docker system prune
WARNING! This will remove:- all stopped containers- all networks not used by at least one container- all dangling images- all dangling build cacheAre you sure you want to continue? [y/N] ysystemctl restart kubelet
systemctl restart docker