Kubernetes常见运维

[wangjc0216]

去除节点

//用于清除kubeadm init or kubeadm join 等配置，防止服务器重启后会重新加入集群
kubeadm reset

// cordon 警戒线的意思，它可以防止新的pod调度到这台node上
kubectl cordon <nodename>

//drain 驱逐
kubectl drain <nodename> --ignore-daemonsets --delete-local-data 

//从集群中删除节点
kubectl delete node <nodename>

[wangjc0216]

更换证书

kubeadm certs check-expiration --v=10
kubeadm certs renew all --v=10

[root@test-master1 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jun 26, 2024 05:41 UTC   210d            ca                      no
apiserver                  Jun 26, 2024 05:41 UTC   210d            ca                      no
apiserver-etcd-client      Jun 26, 2024 05:41 UTC   210d            etcd-ca                 no
apiserver-kubelet-client   Jun 26, 2024 05:41 UTC   210d            ca                      no
controller-manager.conf    Jun 26, 2024 05:41 UTC   210d            ca                      no
etcd-healthcheck-client    Jun 26, 2024 05:41 UTC   210d            etcd-ca                 no
etcd-peer                  Jun 26, 2024 05:41 UTC   210d            etcd-ca                 no
etcd-server                Jun 26, 2024 05:41 UTC   210d            etcd-ca                 no
front-proxy-client         Jun 26, 2024 05:41 UTC   210d            front-proxy-ca          no
scheduler.conf             Jun 26, 2024 05:41 UTC   210d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jun 24, 2033 05:41 UTC   9y              no
etcd-ca                 Jun 24, 2033 05:41 UTC   9y              no
front-proxy-ca          Jun 24, 2033 05:41 UTC   9y              no
[root@test-master1 ~]# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.

重启kube-apiserver、kube-controller-manager、kube-scheduler and etcd

通过移动配置文件位置来进行服务的重启： https://stackoverflow.com/questions/73229958/what-should-i-do-to-restart-the-kubernetes-controller-panel-after-renew-the-cert

mkdir manifests-backup
mv /etc/kubernetes/manifests/* manifests-backup/
mv manifests-backup/* /etc/kubernetes/manifests/

[wangjc0216]

Kubernetes集群升级

控制面：https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/#call-kubeadm-upgrade
数据面：https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/upgrading-linux-nodes/

控制面

kubeadm upgrade plan (v1.24.17)

kubeadm upgrade apply
kubeadm upgrade node
(控制面第一台使用apply，其他控制面和数据面使用node)

yum upgrade kubelet-1.27.6-0.x86_64
kubectl drain test-master1 --ignore-daemonsets
systemctl daemon-reload
systemctl restart kubelet
kubectl uncordon test-master1

在升级期间， kubeadm 会备份数据以用来做数据恢复：
/etc/kubernetes/tmp

数据面

kubeadm upgrade node
kubectl -n kube-system get cm kubeadm-config -o yaml

kubectl drain --ignore-daemonsets
sudo systemctl daemon-reload
sudo systemctl restart kubelet
kubectl uncordon

注意：
需要提前使用apt or yum 安装好指定版本的 kubeadm和kubelet，在控制面选择好kubectl(这个影响不大)。

[wangjc0216]

不同版本的kubelet可能会有差异。 比如 从 v1.24.x 升级到 1.27.x时，--container-runtime=remote 参数就被去掉了，升级后会导致kubelet启动失败，需要在/var/lib/kubelet/kubeadm-flags.env 文件中对参数进行修改。