K3s failed to get CA certs

Mar 11, 2025

Issue:

node$ journalctl -xu k3s-agent -r | less
Mar 11 10:09:35 node-001 k3s[1829]: time="2025-03-11T10:09:35Z" level=error msg="failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:60234->127.0.0.1:6444: read: connection reset by peer"

node$ curl https://master:6443/ -k -vv
No route to host

local$ curl https://master:6443/ -k -vv
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to master:6443

master$ journalctl -xu k3s -r | less
Mar 11 12:45:22 master k3s[214073]: time="2025-03-11T12:45:22Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:6443/v1-k3s/readyz: 500 Internal Server Error"

Solution:

master$ sudo iptables-save > iptables.rules
$ sudo systemctl status iptables

$ sudo iptables -F
$ sudo iptables -X
$ sudo iptables -Z
$ sudo iptables -L
$ sudo systemctl restart k3s

$ sudo systemctl status k3s
$ journalctl -xefu k3s -n 100
$ sudo kubectl get node
NAME       STATUS   ROLES                  AGE      VERSION
node-001   Ready    <none>                 2y114d   v1.25.3+k3s1
master     Ready    control-plane,master   2y114d   v1.28.3+k3s2
node-002   Ready    <none>                 2y114d   v1.28.3+k3s2

[back]