kubernetes v1.36.3升级到v 1.37.0一、更新数据源编辑数据源将版本从1.36 改为1.37rootops-test-021:~# sed -i s#core:/stable:/v1.36#core:/stable:/v1.37#g /etc/apt/sources.list.d/kubernetes.list rootops-test-021:~# apt-get update Get:1 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB] Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease Hit:3 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble InRelease Get:5 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates InRelease [126 kB] Get:6 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [971 kB] Get:7 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports InRelease [126 kB] Get:8 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/main amd64 Packages [1229 kB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb InRelease [1227 B] Get:9 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb Packages [2701 B] Get:10 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [1203 kB] Get:11 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/main amd64 Components [181 kB] Get:12 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe amd64 Packages [1689 kB] Get:13 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe Translation-en [338 kB] Get:14 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe amd64 Components [388 kB] Get:15 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/multiverse amd64 Components [940 B] Get:16 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports/main amd64 Components [5748 B] Get:17 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports/universe amd64 Components [12.6 kB] Fetched 6400 kB in 11s (574 kB/s) Reading package lists... Done rootops-test-021:~# apt-cache madison kubeadm kubeadm | 1.37.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.37/deb Packages rootops-test-021:~#二、升级控制面kubeadmrootops-test-021:~# apt-mark unhold kubeadm Canceled hold on kubeadm. rootops-test-021:~# apt-get install -y kubeadm1.37.0-1.1 Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: cri-tools Use apt autoremove to remove it. The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 45 not upgraded. Need to get 12.6 MB of archives. After this operation, 1552 kB of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubeadm 1.37.0-1.1 [12.6 MB] Fetched 12.6 MB in 8s (1586 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 142192 files and directories currently installed.) Preparing to unpack .../kubeadm_1.37.0-1.1_amd64.deb ... Unpacking kubeadm (1.37.0-1.1) over (1.36.3-1.1) ... Setting up kubeadm (1.37.0-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 7.0.0-28-generic Diagnostics: The currently running kernel version is not the expected kernel version 7.0.0-30-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... Service restarts being deferred: systemctl restart systemd-logind.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. rootops-test-021:~# apt-mark hold kubeadm kubeadm set on hold. rootops-test-021:~# kubeadm version kubeadm version: version.Info{Major:1, Minor:37, EmulationMajor:, EmulationMinor:, MinCompatibilityMajor:, MinCompatibilityMinor:, GitVersion:v1.37.0, GitCommit:f54c212e3a2f75d674b717a9b29052b20b60aefc, GitTreeState:clean, BuildDate:2026-08-26T10:47:17Z, GoVersion:go1.26.6, Compiler:gc, Platform:linux/amd64} rootops-test-021:~#查看升级planrootops-test-021:~# kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the kubeadm-config ConfigMap in namespace kube-system... [upgrade/config] Use kubeadm init phase upload-config kubeadm --config your-config-file to re-upload it. W0827 08:45:55.816103 185218 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: 1.36.0 [upgrade/versions] kubeadm version: v1.37.0 [upgrade/versions] Target version: v1.37.0 [upgrade/versions] Latest version in the v1.36 series: v1.36.4 W0827 08:46:02.854783 185218 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found Components that must be upgraded manually after you have upgraded the control plane with kubeadm upgrade apply: COMPONENT NODE CURRENT TARGET kubelet ops-test-021 v1.36.3 v1.36.4 kubelet ops-test-022 v1.36.3 v1.36.4 kubelet ops-test-023 v1.36.3 v1.36.4 kubelet ops-test-024 v1.36.3 v1.36.4 kubelet ops-test-025 v1.36.3 v1.36.4 kubelet ops-test-026 v1.36.3 v1.36.4 kubelet ops-test-027 v1.36.3 v1.36.4 Upgrade to the latest version in the v1.36 series: COMPONENT NODE CURRENT TARGET kube-apiserver ops-test-021 v1.36.0 v1.36.4 kube-controller-manager ops-test-021 v1.36.0 v1.36.4 kube-scheduler ops-test-021 v1.36.0 v1.36.4 kube-proxy 1.36.0 v1.36.4 CoreDNS v1.14.2 v1.14.6 etcd ops-test-021 3.6.8-0 3.7.0-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.36.4 _____________________________________________________________________ Components that must be upgraded manually after you have upgraded the control plane with kubeadm upgrade apply: COMPONENT NODE CURRENT TARGET kubelet ops-test-021 v1.36.3 v1.37.0 kubelet ops-test-022 v1.36.3 v1.37.0 kubelet ops-test-023 v1.36.3 v1.37.0 kubelet ops-test-024 v1.36.3 v1.37.0 kubelet ops-test-025 v1.36.3 v1.37.0 kubelet ops-test-026 v1.36.3 v1.37.0 kubelet ops-test-027 v1.36.3 v1.37.0 Upgrade to the latest stable version: COMPONENT NODE CURRENT TARGET kube-apiserver ops-test-021 v1.36.0 v1.37.0 kube-controller-manager ops-test-021 v1.36.0 v1.37.0 kube-scheduler ops-test-021 v1.36.0 v1.37.0 kube-proxy 1.36.0 v1.37.0 CoreDNS v1.14.2 v1.14.6 etcd ops-test-021 3.6.8-0 3.7.0-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.37.0 _____________________________________________________________________ The table below shows the current state of component configs as understood by this version of kubeadm. Configs that have a yes mark in the MANUAL UPGRADE REQUIRED column require manual config upgrade or resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually upgrade to is denoted in the PREFERRED VERSION column. API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED kubeproxy.config.k8s.io - v1alpha1 no kubelet.config.k8s.io v1beta1 v1beta1 no _____________________________________________________________________ rootops-test-021:~#检查cilium的状态rootops-test-021:~# cilium config view agent-not-ready-taint-key node.cilium.io/agent-not-ready auto-direct-node-routes true bpf-distributed-lru false bpf-events-drop-enabled true bpf-events-policy-verdict-enabled true bpf-events-trace-enabled true bpf-lb-acceleration disabled bpf-lb-algorithm-annotation false bpf-lb-external-clusterip false bpf-lb-map-max 65536 bpf-lb-mode snat bpf-lb-mode-annotation false bpf-lb-sock false bpf-lb-source-range-all-types false bpf-map-dynamic-size-ratio 0.0025 bpf-policy-map-max 16384 bpf-policy-stats-map-max 65536 bpf-root /sys/fs/bpf cgroup-root /run/cilium/cgroupv2 cilium-endpoint-gc-interval 5m0s cluster-id 0 cluster-name kubernetes clustermesh-cache-ttl 0s clustermesh-enable-endpoint-sync false clustermesh-enable-mcs-api false clustermesh-mcs-api-install-crds true cni-exclusive true cni-log-file /var/run/cilium/cilium-cni.log controller-group-metrics write-cni-file sync-host-ips sync-lb-maps-with-k8s-services custom-cni-conf false datapath-mode veth debug false debug-verbose default-lb-service-ipam lbipam direct-routing-skip-unreachable false dnsproxy-enable-transparent-mode true dnsproxy-socket-linger-timeout 10 egress-gateway-reconciliation-trigger-interval 1s enable-auto-protect-node-port-range true enable-bpf-clock-probe false enable-endpoint-health-checking true enable-endpoint-lockdown-on-policy-overflow false enable-envoy-config true enable-gateway-api true enable-gateway-api-alpn false enable-gateway-api-app-protocol false enable-gateway-api-proxy-protocol false enable-gateway-api-secrets-sync true enable-health-check-loadbalancer-ip false enable-health-check-nodeport true enable-health-checking true enable-hubble true enable-ipv4 true enable-ipv4-big-tcp false enable-ipv4-masquerade true enable-ipv6 false enable-ipv6-big-tcp false enable-ipv6-masquerade true enable-k8s-networkpolicy true enable-l2-announcements true enable-l2-neigh-discovery false enable-l7-proxy true enable-lb-ipam true enable-masquerade-to-route-source false enable-metrics true enable-no-service-endpoints-routable true enable-node-selector-labels false enable-non-default-deny-policies true enable-policy default enable-policy-secrets-sync true enable-sctp false enable-source-ip-verification true enable-tcx true enable-tunnel-big-tcp false enable-vtep false enable-well-known-identities false enable-xt-socket-fallback true envoy-access-log-buffer-size 4096 envoy-base-id 0 envoy-config-retry-interval 15s envoy-keep-cap-netbindservice false external-envoy-proxy true gateway-api-hostnetwork-enabled false gateway-api-hostnetwork-nodelabelselector gateway-api-secrets-namespace cilium-secrets gateway-api-service-externaltrafficpolicy Cluster gateway-api-xff-num-trusted-hops 0 health-check-icmp-failure-threshold 3 http-retry-count 3 http-stream-idle-timeout 300 hubble-disable-tls false hubble-listen-address :4244 hubble-network-policy-correlation-enabled true hubble-socket-path /var/run/cilium/hubble.sock hubble-tls-cert-file /var/lib/cilium/tls/hubble/server.crt hubble-tls-client-ca-files /var/lib/cilium/tls/hubble/client-ca.crt hubble-tls-key-file /var/lib/cilium/tls/hubble/server.key identity-allocation-mode crd identity-gc-interval 15m0s identity-heartbeat-timeout 30m0s identity-management-mode agent install-no-conntrack-iptables-rules false ipam kubernetes ipam-cilium-node-update-rate 15s iptables-random-fully false ipv4-native-routing-cidr 10.244.0.0/16 k8s-require-ipv4-pod-cidr false k8s-require-ipv6-pod-cidr false kube-proxy-replacement true kube-proxy-replacement-healthz-bind-address max-connected-clusters 255 mesh-auth-enabled false mesh-auth-gc-interval 5m0s mesh-auth-queue-size 1024 mesh-auth-rotated-identities-queue-size 1024 metrics-sampling-interval 5m monitor-aggregation medium monitor-aggregation-flags all monitor-aggregation-interval 5s nat-map-stats-entries 32 nat-map-stats-interval 30s node-port-bind-protection true nodeport-addresses nodes-gc-interval 5m0s operator-api-serve-addr 127.0.0.1:9234 operator-prometheus-serve-addr :9963 packetization-layer-pmtud-mode blackhole policy-cidr-match-mode policy-default-local-cluster true policy-deny-response none policy-secrets-namespace cilium-secrets policy-secrets-only-from-secrets-namespace true preallocate-bpf-maps false procfs /host/proc prometheus-serve-addr :9962 proxy-cluster-max-connections 1024 proxy-cluster-max-requests 1024 proxy-connect-timeout 2 proxy-idle-timeout-seconds 60 proxy-initial-fetch-timeout 30 proxy-max-active-downstream-connections 50000 proxy-max-concurrent-retries 128 proxy-max-connection-duration-seconds 0 proxy-max-requests-per-connection 0 proxy-use-original-source-address true proxy-xff-num-trusted-hops-egress 0 proxy-xff-num-trusted-hops-ingress 0 remove-cilium-node-taints true routing-mode native service-no-backend-response reject set-cilium-is-up-condition true set-cilium-node-taints true synchronize-k8s-nodes true tofqdns-dns-reject-response-code refused tofqdns-enable-dns-compression true tofqdns-endpoint-max-ip-per-hostname 1000 tofqdns-idle-connection-grace-period 0s tofqdns-max-deferred-connection-deletes 10000 tofqdns-preallocate-identities true tofqdns-proxy-response-max-delay 100ms tunnel-protocol vxlan tunnel-source-port-range 0-0 unmanaged-pod-watcher-interval 15s vtep-cidr vtep-endpoint vtep-mac vtep-mask write-cni-conf-when-ready /host/etc/cni/net.d/05-cilium.conflist rootops-test-021:~#三、先升级控制面rootops-test-021:~# kubeadm upgrade apply v1.37.0 [upgrade] Reading configuration from the kubeadm-config ConfigMap in namespace kube-system... [upgrade] Use kubeadm init phase upload-config kubeadm --config your-config-file to re-upload it. W0827 09:02:31.344932 185650 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found [upgrade/preflight] Running preflight checks [upgrade] Running cluster health checks [upgrade/preflight] You have chosen to upgrade the cluster version to v1.37.0 [upgrade/versions] Cluster version: v1.36.0 [upgrade/versions] kubeadm version: v1.37.0 [upgrade] Are you sure you want to proceed? [y/N]: y [upgrade/preflight] Pulling images required for setting up a Kubernetes cluster [upgrade/preflight] This might take a minute or two, depending on the speed of your internet connection [upgrade/preflight] You can also perform this action beforehand using kubeadm config images pull [upgrade/control-plane] Upgrading your static Pod-hosted control plane to version v1.37.0 (timeout: 5m0s)... [upgrade/staticpods] Writing new Static Pod manifests to /etc/kubernetes/tmp/kubeadm-upgraded-manifests361672500 [upgrade/staticpods] Preparing for etcd upgrade [upgrade/staticpods] Renewing etcd-server certificate [upgrade/staticpods] Renewing etcd-peer certificate [upgrade/staticpods] Renewing etcd-healthcheck-client certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/etcd.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/etcd.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentetcd [upgrade/staticpods] Component etcd upgraded successfully! [upgrade/etcd] Waiting for etcd to become available [upgrade/staticpods] Preparing for kube-apiserver upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-apiserver.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-apiserver.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-apiserver [upgrade/staticpods] Component kube-apiserver upgraded successfully! [upgrade/staticpods] Preparing for kube-controller-manager upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-controller-manager.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-controller-manager.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-controller-manager [upgrade/staticpods] Component kube-controller-manager upgraded successfully! [upgrade/staticpods] Preparing for kube-scheduler upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-scheduler.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-scheduler.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-scheduler [upgrade/staticpods] Component kube-scheduler upgraded successfully! [upgrade/control-plane] The control plane instance for this node was successfully upgraded! [upload-config] Storing the configuration used in ConfigMap kubeadm-config in the kube-system Namespace [kubelet] Creating a ConfigMap kubelet-config in namespace kube-system with the configuration for the kubelets in the cluster [upgrade/kubeconfig] The kubeconfig files for this node were successfully upgraded! W0827 09:07:00.323635 185650 postupgrade.go:105] Using temporary directory /etc/kubernetes/tmp/kubeadm-kubelet-config-2026-08-27-09-07-00 for kubelet config. To override it set the environment variable KUBEADM_UPGRADE_DRYRUN_DIR [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config-2026-08-27-09-07-00/config.yaml [patches] Applied patch of type application/strategic-merge-patchjson to target kubeletconfiguration [kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml [upgrade/kubelet-config] The kubelet configuration for this node was successfully upgraded! [upgrade/bootstrap-token] Configuring bootstrap token and cluster-info RBAC rules [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API [addons] Applied essential addon: CoreDNS [upgrade/addon] Skipping the addon/kube-proxy phase. The addon is disabled. [upgrade] SUCCESS! A control plane node of your cluster was upgraded to v1.37.0. [upgrade] Now please proceed with upgrading the rest of the nodes by following the right order. rootops-test-021:~#检查集群当前的状态kubectl get nodes -o wide kubectl get pods -A kubectl get pods -n kube-system -o wide cilium status kubectl version四、升级master 升级 021 的 kubelet kubectlrootops-test-021:~# apt-mark unhold kubelet kubectl Canceled hold on kubelet. Canceled hold on kubectl. rootops-test-021:~# apt-get install -y kubelet1.37.0-1.1 kubectl1.37.0-1.1 Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: cri-tools Use apt autoremove to remove it. The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 43 not upgraded. Need to get 25.5 MB of archives. After this operation, 3428 kB of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubectl 1.37.0-1.1 [12.1 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubelet 1.37.0-1.1 [13.5 MB] Fetched 25.5 MB in 14s (1791 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 142192 files and directories currently installed.) Preparing to unpack .../kubectl_1.37.0-1.1_amd64.deb ... Unpacking kubectl (1.37.0-1.1) over (1.36.3-1.1) ... Preparing to unpack .../kubelet_1.37.0-1.1_amd64.deb ... Unpacking kubelet (1.37.0-1.1) over (1.36.3-1.1) ... Setting up kubectl (1.37.0-1.1) ... Setting up kubelet (1.37.0-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 7.0.0-28-generic Diagnostics: The currently running kernel version is not the expected kernel version 7.0.0-30-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service Service restarts being deferred: systemctl restart systemd-logind.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. rootops-test-021:~# apt-mark hold kubelet kubectl kubelet set on hold. kubectl set on hold. rootops-test-021:~# systemctl daemon-reload rootops-test-021:~# systemctl restart kubelet rootops-test-021:~#查看master状态rootops-test-021:~# kubectl get node ops-test-021 -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ops-test-021 Ready control-plane 112d v1.37.0 192.168.2.21 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 rootops-test-021:~# kubectl get nodes NAME STATUS ROLES AGE VERSION ops-test-021 Ready control-plane 112d v1.37.0 ops-test-022 Ready none 112d v1.36.3 ops-test-023 Ready none 112d v1.36.3 ops-test-024 Ready none 11d v1.36.3 ops-test-025 Ready none 26d v1.36.3 ops-test-026 Ready none 15d v1.36.3 ops-test-027 Ready none 12d v1.36.3 rootops-test-021:~# 查看cllium状态然后再逐台升级 Worker检查源cat /etc/apt/sources.list.d/kubernetes.list如果是 v1.36sed -i s#core:/stable:/v1.36#core:/stable:/v1.37#g /etc/apt/sources.list.d/kubernetes.list apt-get update升级apt-mark unhold kubelet apt-get install -y kubelet1.37.0-1.1 apt-mark hold kubelet systemctl daemon-reload systemctl restart kubelet查看集群状态rootops-test-021:~# kubectl get nodes NAME STATUS ROLES AGE VERSION ops-test-021 Ready control-plane 112d v1.37.0 ops-test-022 Ready none 112d v1.37.0 ops-test-023 Ready none 112d v1.37.0 ops-test-024 Ready none 11d v1.37.0 ops-test-025 Ready none 26d v1.37.0 ops-test-026 Ready none 15d v1.37.0 ops-test-027 Ready none 12d v1.37.0 rootops-test-021:~# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ops-test-021 Ready control-plane 112d v1.37.0 192.168.2.21 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 ops-test-022 Ready none 112d v1.37.0 192.168.2.22 none Ubuntu 24.04.4 LTS 6.17.0-29-generic (amd64) containerd://2.2.3 ops-test-023 Ready none 112d v1.37.0 192.168.2.23 none Ubuntu 24.04.4 LTS 6.17.0-29-generic (amd64) containerd://2.2.3 ops-test-024 Ready none 12d v1.37.0 192.168.2.24 none Ubuntu 24.04.2 LTS 6.11.0-24-generic (amd64) containerd://2.2.3 ops-test-025 Ready none 26d v1.37.0 192.168.2.25 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 ops-test-026 Ready none 15d v1.37.0 192.168.2.26 none Ubuntu 24.04.3 LTS 7.0.0-28-generic (amd64) containerd://2.0.2 ops-test-027 Ready none 12d v1.37.0 192.168.2.27 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 rootops-test-021:~# rootops-test-021:~# cilium status /¯¯\ /¯¯\__/¯¯\ Cilium: OK \__/¯¯\__/ Operator: OK /¯¯\__/¯¯\ Envoy DaemonSet: OK \__/¯¯\__/ Hubble Relay: OK \__/ ClusterMesh: disabled DaemonSet cilium Desired: 7, Ready: 7/7, Available: 7/7 DaemonSet cilium-envoy Desired: 7, Ready: 7/7, Available: 7/7 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1 Containers: cilium Running: 7 cilium-envoy Running: 7 cilium-operator Running: 1 clustermesh-apiserver hubble-relay Running: 1 hubble-ui Running: 1 Cluster Pods: 4/4 managed by Cilium Helm chart version: 1.19.1 Image versions cilium quay.io/cilium/cilium:v1.19.1sha256:41f1f74a0000de8656f1de4088ea00c8f2d49d6edea579034c73c5fd5fe01792: 7 cilium-envoy quay.io/cilium/cilium-envoy:v1.35.9-1770979049-232ed4a26881e4ab4f766f251f258ed424fff663sha256:8188114a2768b5f49d6ce58e168b20d765e0fbc64eee0d83241aa2b150ccd788: 7 cilium-operator quay.io/cilium/operator-generic:v1.19.1sha256:e7278d763e448bf6c184b0682cf98cdca078d58a27e1b2f3c906792670aa211a: 1 hubble-relay quay.io/cilium/hubble-relay:v1.19.1sha256:d8c4e13bc36a56179292bb52bc6255379cb94cb873700d316ea3139b1bdb8165: 1 hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.3sha256:db1454e45dc39ca41fbf7cad31eec95d99e5b9949c39daaad0fa81ef29d56953: 1 hubble-ui quay.io/cilium/hubble-ui:v0.13.3sha256:661d5de7050182d495c6497ff0b007a7a1e379648e60830dd68c4d78ae21761d: 1 rootops-test-021:~# kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system cilium-42jkw 1/1 Running 1 (84d ago) 111d kube-system cilium-8zjd6 1/1 Running 1 (90d ago) 111d kube-system cilium-envoy-4s6bn 1/1 Running 5 (11d ago) 111d kube-system cilium-envoy-7vcdl 1/1 Running 0 12d kube-system cilium-envoy-9tr8g 1/1 Running 1 (90d ago) 111d kube-system cilium-envoy-jw8vw 1/1 Running 1 (12d ago) 15d kube-system cilium-envoy-mb5sc 1/1 Running 0 26d kube-system cilium-envoy-nczxb 1/1 Running 1 (84d ago) 111d kube-system cilium-envoy-xcpn7 1/1 Running 0 12d kube-system cilium-ght94 1/1 Running 0 12d kube-system cilium-hbv5q 1/1 Running 1 (12d ago) 15d kube-system cilium-hc66j 1/1 Running 1 (11d ago) 12d kube-system cilium-operator-845ffffd57-2hlnl 1/1 Running 6 (19m ago) 90d kube-system cilium-rxdqg 1/1 Running 0 26d kube-system cilium-s7b6n 1/1 Running 5 (11d ago) 111d kube-system coredns-6cf5fbd489-cbdcz 1/1 Running 0 18m kube-system coredns-6cf5fbd489-qmlrn 1/1 Running 0 18m kube-system etcd-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system hubble-relay-5d96b56dc8-xtz9s 1/1 Running 0 90d kube-system hubble-ui-6b4f8fcff9-jrxxm 2/2 Running 0 90d kube-system kube-apiserver-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system kube-controller-manager-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system kube-scheduler-ops-test-021 1/1 Running 1 (11m ago) 11m rootops-test-021:~#升级完成
kubernetes v1.36.3升级到v 1.37.0
kubernetes v1.36.3升级到v 1.37.0一、更新数据源编辑数据源将版本从1.36 改为1.37rootops-test-021:~# sed -i s#core:/stable:/v1.36#core:/stable:/v1.37#g /etc/apt/sources.list.d/kubernetes.list rootops-test-021:~# apt-get update Get:1 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB] Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease Hit:3 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble InRelease Get:5 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates InRelease [126 kB] Get:6 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [971 kB] Get:7 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports InRelease [126 kB] Get:8 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/main amd64 Packages [1229 kB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb InRelease [1227 B] Get:9 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb Packages [2701 B] Get:10 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [1203 kB] Get:11 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/main amd64 Components [181 kB] Get:12 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe amd64 Packages [1689 kB] Get:13 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe Translation-en [338 kB] Get:14 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/universe amd64 Components [388 kB] Get:15 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-updates/multiverse amd64 Components [940 B] Get:16 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports/main amd64 Components [5748 B] Get:17 http://mirrors.tuna.tsinghua.edu.cn/ubuntu noble-backports/universe amd64 Components [12.6 kB] Fetched 6400 kB in 11s (574 kB/s) Reading package lists... Done rootops-test-021:~# apt-cache madison kubeadm kubeadm | 1.37.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.37/deb Packages rootops-test-021:~#二、升级控制面kubeadmrootops-test-021:~# apt-mark unhold kubeadm Canceled hold on kubeadm. rootops-test-021:~# apt-get install -y kubeadm1.37.0-1.1 Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: cri-tools Use apt autoremove to remove it. The following packages will be upgraded: kubeadm 1 upgraded, 0 newly installed, 0 to remove and 45 not upgraded. Need to get 12.6 MB of archives. After this operation, 1552 kB of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubeadm 1.37.0-1.1 [12.6 MB] Fetched 12.6 MB in 8s (1586 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 142192 files and directories currently installed.) Preparing to unpack .../kubeadm_1.37.0-1.1_amd64.deb ... Unpacking kubeadm (1.37.0-1.1) over (1.36.3-1.1) ... Setting up kubeadm (1.37.0-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 7.0.0-28-generic Diagnostics: The currently running kernel version is not the expected kernel version 7.0.0-30-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... Service restarts being deferred: systemctl restart systemd-logind.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. rootops-test-021:~# apt-mark hold kubeadm kubeadm set on hold. rootops-test-021:~# kubeadm version kubeadm version: version.Info{Major:1, Minor:37, EmulationMajor:, EmulationMinor:, MinCompatibilityMajor:, MinCompatibilityMinor:, GitVersion:v1.37.0, GitCommit:f54c212e3a2f75d674b717a9b29052b20b60aefc, GitTreeState:clean, BuildDate:2026-08-26T10:47:17Z, GoVersion:go1.26.6, Compiler:gc, Platform:linux/amd64} rootops-test-021:~#查看升级planrootops-test-021:~# kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the kubeadm-config ConfigMap in namespace kube-system... [upgrade/config] Use kubeadm init phase upload-config kubeadm --config your-config-file to re-upload it. W0827 08:45:55.816103 185218 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: 1.36.0 [upgrade/versions] kubeadm version: v1.37.0 [upgrade/versions] Target version: v1.37.0 [upgrade/versions] Latest version in the v1.36 series: v1.36.4 W0827 08:46:02.854783 185218 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found Components that must be upgraded manually after you have upgraded the control plane with kubeadm upgrade apply: COMPONENT NODE CURRENT TARGET kubelet ops-test-021 v1.36.3 v1.36.4 kubelet ops-test-022 v1.36.3 v1.36.4 kubelet ops-test-023 v1.36.3 v1.36.4 kubelet ops-test-024 v1.36.3 v1.36.4 kubelet ops-test-025 v1.36.3 v1.36.4 kubelet ops-test-026 v1.36.3 v1.36.4 kubelet ops-test-027 v1.36.3 v1.36.4 Upgrade to the latest version in the v1.36 series: COMPONENT NODE CURRENT TARGET kube-apiserver ops-test-021 v1.36.0 v1.36.4 kube-controller-manager ops-test-021 v1.36.0 v1.36.4 kube-scheduler ops-test-021 v1.36.0 v1.36.4 kube-proxy 1.36.0 v1.36.4 CoreDNS v1.14.2 v1.14.6 etcd ops-test-021 3.6.8-0 3.7.0-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.36.4 _____________________________________________________________________ Components that must be upgraded manually after you have upgraded the control plane with kubeadm upgrade apply: COMPONENT NODE CURRENT TARGET kubelet ops-test-021 v1.36.3 v1.37.0 kubelet ops-test-022 v1.36.3 v1.37.0 kubelet ops-test-023 v1.36.3 v1.37.0 kubelet ops-test-024 v1.36.3 v1.37.0 kubelet ops-test-025 v1.36.3 v1.37.0 kubelet ops-test-026 v1.36.3 v1.37.0 kubelet ops-test-027 v1.36.3 v1.37.0 Upgrade to the latest stable version: COMPONENT NODE CURRENT TARGET kube-apiserver ops-test-021 v1.36.0 v1.37.0 kube-controller-manager ops-test-021 v1.36.0 v1.37.0 kube-scheduler ops-test-021 v1.36.0 v1.37.0 kube-proxy 1.36.0 v1.37.0 CoreDNS v1.14.2 v1.14.6 etcd ops-test-021 3.6.8-0 3.7.0-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.37.0 _____________________________________________________________________ The table below shows the current state of component configs as understood by this version of kubeadm. Configs that have a yes mark in the MANUAL UPGRADE REQUIRED column require manual config upgrade or resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually upgrade to is denoted in the PREFERRED VERSION column. API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED kubeproxy.config.k8s.io - v1alpha1 no kubelet.config.k8s.io v1beta1 v1beta1 no _____________________________________________________________________ rootops-test-021:~#检查cilium的状态rootops-test-021:~# cilium config view agent-not-ready-taint-key node.cilium.io/agent-not-ready auto-direct-node-routes true bpf-distributed-lru false bpf-events-drop-enabled true bpf-events-policy-verdict-enabled true bpf-events-trace-enabled true bpf-lb-acceleration disabled bpf-lb-algorithm-annotation false bpf-lb-external-clusterip false bpf-lb-map-max 65536 bpf-lb-mode snat bpf-lb-mode-annotation false bpf-lb-sock false bpf-lb-source-range-all-types false bpf-map-dynamic-size-ratio 0.0025 bpf-policy-map-max 16384 bpf-policy-stats-map-max 65536 bpf-root /sys/fs/bpf cgroup-root /run/cilium/cgroupv2 cilium-endpoint-gc-interval 5m0s cluster-id 0 cluster-name kubernetes clustermesh-cache-ttl 0s clustermesh-enable-endpoint-sync false clustermesh-enable-mcs-api false clustermesh-mcs-api-install-crds true cni-exclusive true cni-log-file /var/run/cilium/cilium-cni.log controller-group-metrics write-cni-file sync-host-ips sync-lb-maps-with-k8s-services custom-cni-conf false datapath-mode veth debug false debug-verbose default-lb-service-ipam lbipam direct-routing-skip-unreachable false dnsproxy-enable-transparent-mode true dnsproxy-socket-linger-timeout 10 egress-gateway-reconciliation-trigger-interval 1s enable-auto-protect-node-port-range true enable-bpf-clock-probe false enable-endpoint-health-checking true enable-endpoint-lockdown-on-policy-overflow false enable-envoy-config true enable-gateway-api true enable-gateway-api-alpn false enable-gateway-api-app-protocol false enable-gateway-api-proxy-protocol false enable-gateway-api-secrets-sync true enable-health-check-loadbalancer-ip false enable-health-check-nodeport true enable-health-checking true enable-hubble true enable-ipv4 true enable-ipv4-big-tcp false enable-ipv4-masquerade true enable-ipv6 false enable-ipv6-big-tcp false enable-ipv6-masquerade true enable-k8s-networkpolicy true enable-l2-announcements true enable-l2-neigh-discovery false enable-l7-proxy true enable-lb-ipam true enable-masquerade-to-route-source false enable-metrics true enable-no-service-endpoints-routable true enable-node-selector-labels false enable-non-default-deny-policies true enable-policy default enable-policy-secrets-sync true enable-sctp false enable-source-ip-verification true enable-tcx true enable-tunnel-big-tcp false enable-vtep false enable-well-known-identities false enable-xt-socket-fallback true envoy-access-log-buffer-size 4096 envoy-base-id 0 envoy-config-retry-interval 15s envoy-keep-cap-netbindservice false external-envoy-proxy true gateway-api-hostnetwork-enabled false gateway-api-hostnetwork-nodelabelselector gateway-api-secrets-namespace cilium-secrets gateway-api-service-externaltrafficpolicy Cluster gateway-api-xff-num-trusted-hops 0 health-check-icmp-failure-threshold 3 http-retry-count 3 http-stream-idle-timeout 300 hubble-disable-tls false hubble-listen-address :4244 hubble-network-policy-correlation-enabled true hubble-socket-path /var/run/cilium/hubble.sock hubble-tls-cert-file /var/lib/cilium/tls/hubble/server.crt hubble-tls-client-ca-files /var/lib/cilium/tls/hubble/client-ca.crt hubble-tls-key-file /var/lib/cilium/tls/hubble/server.key identity-allocation-mode crd identity-gc-interval 15m0s identity-heartbeat-timeout 30m0s identity-management-mode agent install-no-conntrack-iptables-rules false ipam kubernetes ipam-cilium-node-update-rate 15s iptables-random-fully false ipv4-native-routing-cidr 10.244.0.0/16 k8s-require-ipv4-pod-cidr false k8s-require-ipv6-pod-cidr false kube-proxy-replacement true kube-proxy-replacement-healthz-bind-address max-connected-clusters 255 mesh-auth-enabled false mesh-auth-gc-interval 5m0s mesh-auth-queue-size 1024 mesh-auth-rotated-identities-queue-size 1024 metrics-sampling-interval 5m monitor-aggregation medium monitor-aggregation-flags all monitor-aggregation-interval 5s nat-map-stats-entries 32 nat-map-stats-interval 30s node-port-bind-protection true nodeport-addresses nodes-gc-interval 5m0s operator-api-serve-addr 127.0.0.1:9234 operator-prometheus-serve-addr :9963 packetization-layer-pmtud-mode blackhole policy-cidr-match-mode policy-default-local-cluster true policy-deny-response none policy-secrets-namespace cilium-secrets policy-secrets-only-from-secrets-namespace true preallocate-bpf-maps false procfs /host/proc prometheus-serve-addr :9962 proxy-cluster-max-connections 1024 proxy-cluster-max-requests 1024 proxy-connect-timeout 2 proxy-idle-timeout-seconds 60 proxy-initial-fetch-timeout 30 proxy-max-active-downstream-connections 50000 proxy-max-concurrent-retries 128 proxy-max-connection-duration-seconds 0 proxy-max-requests-per-connection 0 proxy-use-original-source-address true proxy-xff-num-trusted-hops-egress 0 proxy-xff-num-trusted-hops-ingress 0 remove-cilium-node-taints true routing-mode native service-no-backend-response reject set-cilium-is-up-condition true set-cilium-node-taints true synchronize-k8s-nodes true tofqdns-dns-reject-response-code refused tofqdns-enable-dns-compression true tofqdns-endpoint-max-ip-per-hostname 1000 tofqdns-idle-connection-grace-period 0s tofqdns-max-deferred-connection-deletes 10000 tofqdns-preallocate-identities true tofqdns-proxy-response-max-delay 100ms tunnel-protocol vxlan tunnel-source-port-range 0-0 unmanaged-pod-watcher-interval 15s vtep-cidr vtep-endpoint vtep-mac vtep-mask write-cni-conf-when-ready /host/etc/cni/net.d/05-cilium.conflist rootops-test-021:~#三、先升级控制面rootops-test-021:~# kubeadm upgrade apply v1.37.0 [upgrade] Reading configuration from the kubeadm-config ConfigMap in namespace kube-system... [upgrade] Use kubeadm init phase upload-config kubeadm --config your-config-file to re-upload it. W0827 09:02:31.344932 185650 configset.go:77] Warning: No kubeproxy.config.k8s.io/v1alpha1 config is loaded. Continuing without it: configmaps kube-proxy not found [upgrade/preflight] Running preflight checks [upgrade] Running cluster health checks [upgrade/preflight] You have chosen to upgrade the cluster version to v1.37.0 [upgrade/versions] Cluster version: v1.36.0 [upgrade/versions] kubeadm version: v1.37.0 [upgrade] Are you sure you want to proceed? [y/N]: y [upgrade/preflight] Pulling images required for setting up a Kubernetes cluster [upgrade/preflight] This might take a minute or two, depending on the speed of your internet connection [upgrade/preflight] You can also perform this action beforehand using kubeadm config images pull [upgrade/control-plane] Upgrading your static Pod-hosted control plane to version v1.37.0 (timeout: 5m0s)... [upgrade/staticpods] Writing new Static Pod manifests to /etc/kubernetes/tmp/kubeadm-upgraded-manifests361672500 [upgrade/staticpods] Preparing for etcd upgrade [upgrade/staticpods] Renewing etcd-server certificate [upgrade/staticpods] Renewing etcd-peer certificate [upgrade/staticpods] Renewing etcd-healthcheck-client certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/etcd.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/etcd.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentetcd [upgrade/staticpods] Component etcd upgraded successfully! [upgrade/etcd] Waiting for etcd to become available [upgrade/staticpods] Preparing for kube-apiserver upgrade [upgrade/staticpods] Renewing apiserver certificate [upgrade/staticpods] Renewing apiserver-kubelet-client certificate [upgrade/staticpods] Renewing front-proxy-client certificate [upgrade/staticpods] Renewing apiserver-etcd-client certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-apiserver.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-apiserver.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-apiserver [upgrade/staticpods] Component kube-apiserver upgraded successfully! [upgrade/staticpods] Preparing for kube-controller-manager upgrade [upgrade/staticpods] Renewing controller-manager.conf certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-controller-manager.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-controller-manager.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-controller-manager [upgrade/staticpods] Component kube-controller-manager upgraded successfully! [upgrade/staticpods] Preparing for kube-scheduler upgrade [upgrade/staticpods] Renewing scheduler.conf certificate [upgrade/staticpods] Moving new manifest to /etc/kubernetes/manifests/kube-scheduler.yaml and backing up old manifest to /etc/kubernetes/tmp/kubeadm-backup-manifests-2026-08-27-09-04-12/kube-scheduler.yaml [upgrade/staticpods] Waiting for the kubelet to restart the component [upgrade/staticpods] This can take up to 5m0s [apiclient] Found 1 Pods for label selector componentkube-scheduler [upgrade/staticpods] Component kube-scheduler upgraded successfully! [upgrade/control-plane] The control plane instance for this node was successfully upgraded! [upload-config] Storing the configuration used in ConfigMap kubeadm-config in the kube-system Namespace [kubelet] Creating a ConfigMap kubelet-config in namespace kube-system with the configuration for the kubelets in the cluster [upgrade/kubeconfig] The kubeconfig files for this node were successfully upgraded! W0827 09:07:00.323635 185650 postupgrade.go:105] Using temporary directory /etc/kubernetes/tmp/kubeadm-kubelet-config-2026-08-27-09-07-00 for kubelet config. To override it set the environment variable KUBEADM_UPGRADE_DRYRUN_DIR [upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config-2026-08-27-09-07-00/config.yaml [patches] Applied patch of type application/strategic-merge-patchjson to target kubeletconfiguration [kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml [upgrade/kubelet-config] The kubelet configuration for this node was successfully upgraded! [upgrade/bootstrap-token] Configuring bootstrap token and cluster-info RBAC rules [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API [addons] Applied essential addon: CoreDNS [upgrade/addon] Skipping the addon/kube-proxy phase. The addon is disabled. [upgrade] SUCCESS! A control plane node of your cluster was upgraded to v1.37.0. [upgrade] Now please proceed with upgrading the rest of the nodes by following the right order. rootops-test-021:~#检查集群当前的状态kubectl get nodes -o wide kubectl get pods -A kubectl get pods -n kube-system -o wide cilium status kubectl version四、升级master 升级 021 的 kubelet kubectlrootops-test-021:~# apt-mark unhold kubelet kubectl Canceled hold on kubelet. Canceled hold on kubectl. rootops-test-021:~# apt-get install -y kubelet1.37.0-1.1 kubectl1.37.0-1.1 Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: cri-tools Use apt autoremove to remove it. The following packages will be upgraded: kubectl kubelet 2 upgraded, 0 newly installed, 0 to remove and 43 not upgraded. Need to get 25.5 MB of archives. After this operation, 3428 kB of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubectl 1.37.0-1.1 [12.1 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.37/deb kubelet 1.37.0-1.1 [13.5 MB] Fetched 25.5 MB in 14s (1791 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 142192 files and directories currently installed.) Preparing to unpack .../kubectl_1.37.0-1.1_amd64.deb ... Unpacking kubectl (1.37.0-1.1) over (1.36.3-1.1) ... Preparing to unpack .../kubelet_1.37.0-1.1_amd64.deb ... Unpacking kubelet (1.37.0-1.1) over (1.36.3-1.1) ... Setting up kubectl (1.37.0-1.1) ... Setting up kubelet (1.37.0-1.1) ... Scanning processes... Scanning candidates... Scanning linux images... Pending kernel upgrade! Running kernel version: 7.0.0-28-generic Diagnostics: The currently running kernel version is not the expected kernel version 7.0.0-30-generic. Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. Restarting services... systemctl restart kubelet.service Service restarts being deferred: systemctl restart systemd-logind.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. rootops-test-021:~# apt-mark hold kubelet kubectl kubelet set on hold. kubectl set on hold. rootops-test-021:~# systemctl daemon-reload rootops-test-021:~# systemctl restart kubelet rootops-test-021:~#查看master状态rootops-test-021:~# kubectl get node ops-test-021 -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ops-test-021 Ready control-plane 112d v1.37.0 192.168.2.21 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 rootops-test-021:~# kubectl get nodes NAME STATUS ROLES AGE VERSION ops-test-021 Ready control-plane 112d v1.37.0 ops-test-022 Ready none 112d v1.36.3 ops-test-023 Ready none 112d v1.36.3 ops-test-024 Ready none 11d v1.36.3 ops-test-025 Ready none 26d v1.36.3 ops-test-026 Ready none 15d v1.36.3 ops-test-027 Ready none 12d v1.36.3 rootops-test-021:~# 查看cllium状态然后再逐台升级 Worker检查源cat /etc/apt/sources.list.d/kubernetes.list如果是 v1.36sed -i s#core:/stable:/v1.36#core:/stable:/v1.37#g /etc/apt/sources.list.d/kubernetes.list apt-get update升级apt-mark unhold kubelet apt-get install -y kubelet1.37.0-1.1 apt-mark hold kubelet systemctl daemon-reload systemctl restart kubelet查看集群状态rootops-test-021:~# kubectl get nodes NAME STATUS ROLES AGE VERSION ops-test-021 Ready control-plane 112d v1.37.0 ops-test-022 Ready none 112d v1.37.0 ops-test-023 Ready none 112d v1.37.0 ops-test-024 Ready none 11d v1.37.0 ops-test-025 Ready none 26d v1.37.0 ops-test-026 Ready none 15d v1.37.0 ops-test-027 Ready none 12d v1.37.0 rootops-test-021:~# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ops-test-021 Ready control-plane 112d v1.37.0 192.168.2.21 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 ops-test-022 Ready none 112d v1.37.0 192.168.2.22 none Ubuntu 24.04.4 LTS 6.17.0-29-generic (amd64) containerd://2.2.3 ops-test-023 Ready none 112d v1.37.0 192.168.2.23 none Ubuntu 24.04.4 LTS 6.17.0-29-generic (amd64) containerd://2.2.3 ops-test-024 Ready none 12d v1.37.0 192.168.2.24 none Ubuntu 24.04.2 LTS 6.11.0-24-generic (amd64) containerd://2.2.3 ops-test-025 Ready none 26d v1.37.0 192.168.2.25 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 ops-test-026 Ready none 15d v1.37.0 192.168.2.26 none Ubuntu 24.04.3 LTS 7.0.0-28-generic (amd64) containerd://2.0.2 ops-test-027 Ready none 12d v1.37.0 192.168.2.27 none Ubuntu 24.04.4 LTS 7.0.0-28-generic (amd64) containerd://2.2.3 rootops-test-021:~# rootops-test-021:~# cilium status /¯¯\ /¯¯\__/¯¯\ Cilium: OK \__/¯¯\__/ Operator: OK /¯¯\__/¯¯\ Envoy DaemonSet: OK \__/¯¯\__/ Hubble Relay: OK \__/ ClusterMesh: disabled DaemonSet cilium Desired: 7, Ready: 7/7, Available: 7/7 DaemonSet cilium-envoy Desired: 7, Ready: 7/7, Available: 7/7 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1 Containers: cilium Running: 7 cilium-envoy Running: 7 cilium-operator Running: 1 clustermesh-apiserver hubble-relay Running: 1 hubble-ui Running: 1 Cluster Pods: 4/4 managed by Cilium Helm chart version: 1.19.1 Image versions cilium quay.io/cilium/cilium:v1.19.1sha256:41f1f74a0000de8656f1de4088ea00c8f2d49d6edea579034c73c5fd5fe01792: 7 cilium-envoy quay.io/cilium/cilium-envoy:v1.35.9-1770979049-232ed4a26881e4ab4f766f251f258ed424fff663sha256:8188114a2768b5f49d6ce58e168b20d765e0fbc64eee0d83241aa2b150ccd788: 7 cilium-operator quay.io/cilium/operator-generic:v1.19.1sha256:e7278d763e448bf6c184b0682cf98cdca078d58a27e1b2f3c906792670aa211a: 1 hubble-relay quay.io/cilium/hubble-relay:v1.19.1sha256:d8c4e13bc36a56179292bb52bc6255379cb94cb873700d316ea3139b1bdb8165: 1 hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.3sha256:db1454e45dc39ca41fbf7cad31eec95d99e5b9949c39daaad0fa81ef29d56953: 1 hubble-ui quay.io/cilium/hubble-ui:v0.13.3sha256:661d5de7050182d495c6497ff0b007a7a1e379648e60830dd68c4d78ae21761d: 1 rootops-test-021:~# kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system cilium-42jkw 1/1 Running 1 (84d ago) 111d kube-system cilium-8zjd6 1/1 Running 1 (90d ago) 111d kube-system cilium-envoy-4s6bn 1/1 Running 5 (11d ago) 111d kube-system cilium-envoy-7vcdl 1/1 Running 0 12d kube-system cilium-envoy-9tr8g 1/1 Running 1 (90d ago) 111d kube-system cilium-envoy-jw8vw 1/1 Running 1 (12d ago) 15d kube-system cilium-envoy-mb5sc 1/1 Running 0 26d kube-system cilium-envoy-nczxb 1/1 Running 1 (84d ago) 111d kube-system cilium-envoy-xcpn7 1/1 Running 0 12d kube-system cilium-ght94 1/1 Running 0 12d kube-system cilium-hbv5q 1/1 Running 1 (12d ago) 15d kube-system cilium-hc66j 1/1 Running 1 (11d ago) 12d kube-system cilium-operator-845ffffd57-2hlnl 1/1 Running 6 (19m ago) 90d kube-system cilium-rxdqg 1/1 Running 0 26d kube-system cilium-s7b6n 1/1 Running 5 (11d ago) 111d kube-system coredns-6cf5fbd489-cbdcz 1/1 Running 0 18m kube-system coredns-6cf5fbd489-qmlrn 1/1 Running 0 18m kube-system etcd-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system hubble-relay-5d96b56dc8-xtz9s 1/1 Running 0 90d kube-system hubble-ui-6b4f8fcff9-jrxxm 2/2 Running 0 90d kube-system kube-apiserver-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system kube-controller-manager-ops-test-021 1/1 Running 1 (11m ago) 11m kube-system kube-scheduler-ops-test-021 1/1 Running 1 (11m ago) 11m rootops-test-021:~#升级完成