0

The issue:

I'd like to access an application hosted behind a Cilium ingress in my bare metal cluster. However, I can only access it from the node and not an external system. I am not sure if everything is set up correctly or which IP:PORT combination I should use.

The setup:

I have a two node Kubernetes cluster setup using Multipass (two Ubuntu 24.04 machines). I have bridged the network, so that the two VMs are in the same address space as the VM host. I switched the order of default gateways using netplan prior to setting up the cluster so that Kubernetes uses my home network address space.

I used Cilium as the CNI, kube proxy replacement and ingress provider. I used L2 Announcements for external and loadbalancer IPs with the following values for the Helm chart:

# values.yaml for Helm chart
ipam:
  operator:
    clusterPoolIPv4MaskSize: 24
    clusterPoolIPv4PodCIDRList:
    - 10.42.0.0/16
k8s:
  requireIPv4PodCIDR: true
ingressController:
  enabled: true
  default: true
  service:
    externalTrafficPolicy: Cluster
    type: LoadBalancer
  loadbalancerMode: dedicated
kubeProxyReplacement: true
l2announcements:
  enabled: true
externalIPs:
  enabled: true
loadBalancerIPs:
  enabled: true

The pod network cidr was also given to kubeadm init as --pod-network-cidr=10.42.0.0/16.

After setting everything up, I created a Cilium Loadbalancer IP Pool so that my new ingress loadbalancers would get external IP addresses:

apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
  name: "default-pool"
spec:
  blocks:
  - cidr: "10.31.0.0/24"

Then I created a test Pod using an nginx image and exposed it on port 80 using a ClusterIP service:

NAME             READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod/test-nginx   1/1     Running   0          18h   10.42.1.108   k8s-worker   <none>           <none>

NAME       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE   SELECTOR
test-svc   ClusterIP   10.100.113.198   <none>        80/TCP    18h   run=test-nginx

Then I created the corresponding ingress object:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    ingress.cilium.io/loadbalancer-class: io.cilium/l2-announcer
spec:
  ingressClassName: cilium
  rules:
  - host: test-site.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: test-svc
            port:
              number: 80

It got the external IP from the pool:

NAME           CLASS    HOSTS           ADDRESS     PORTS   AGE
test-ingress   cilium   test-site.com   10.31.0.0   80      45m

The basic ingress loadbalancer in the kube-system namespace and the dedicated loadbalancer are also running:

NAME             TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
cilium-ingress   LoadBalancer   10.109.159.64   10.31.0.1     80:31577/TCP,443:31560/TCP   18h

NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
cilium-ingress-test-ingress   LoadBalancer   10.99.218.182   10.31.0.0     80:30262/TCP,443:30638/TCP   47m

Finally, I wrote 10.31.0.0 test-site.com into /etc/hosts on the controlplane node and executed curl http://test-site.com:80 against it. This gave me the expected output:

root@k8s-master:~# curl http://test-site.com:80/

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>


<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>


<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Then I tried the same on the host machine (which is in the same address space as the controlplane node VM) and it did not work.

$: curl -v http://test-site.com:80/ --resolve "test-site.com:80:10.31.0.0"
* Added test-site.com:80:10.31.0.0 to DNS cache
* Hostname test-site.com was found in DNS cache
*   Trying 10.31.0.0:80...

I tried different IPs and ports.

Do I have to create another virtual connection to the 10.31.0.0/24 address space from my host machine?

Melvin
  • 101

0 Answers0