We have a local k3s cluster for our staging environment to reproduce something like our production environment. Today, our single node reached its limit, so we decided to add a new node.
I bought a new physical server and just installed Ubuntu Server 24.04.1 LTS. The next step was to install k3s agent to make it join the existing cluster. I followed the online documentation:
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.1:6443 K3S_TOKEN=<my master token> sh -
Then, I checked that everything was ready with kubectl get nodes:
NAME STATUS ROLES AGE VERSION
serv1 Ready control-plane,master 382d v1.28.5+k3s1
serv2 Ready <none> 117s v1.31.4+k3s1
But when the first pod was assigned to this new node, it gets the status CreateContainerConfigError.
Describing the pod with kubectl describe pod, I can this this error:
Warning Failed 12s (x2 over 13s) kubelet Error: services have not yet been read at least once, cannot construct envvars
I found a few things online about this error. It seems that there is a problem between our two servers and that for some reason they cannot communicate properly. But as the new node is marked Ready, I don't understand where is the problem...
I also found this exact same case here, but it seems no real solution is shared.
Does anyone have an idea about the cause of this problem?