0

I am using traefik 2.10 to forward the http request into kuberentes cluster, the http forward path is: web->openresty->traefik->service->pod, in the host (the forward step is openresty->traefik) I am using tcpdump to check the http request and found the header like this:

...d...dGET /tex/project/list?archive_status=1&proj_status=3 HTTP/1.0
Host: tex.example.top
X-Real-IP: 183.228.57.204
X-Forwarded-For: 183.228.57.204
Connection: close
x-request-id: 8608b419-48b2-41b0-ad08-728fba4fec84
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGcImzcwfQ.wJGvMguWLGWjBwe_5fy_NH89ecEBKu7p0UDIXcxyfDc
sec-ch-ua-platform: "macOS"
sec-ch-ua: "Google Chrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36
Accept: application/json, text/plain, */*
x-action: GET_PROJ_LIST
DNT: 1
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://tex.example.top/doc/tab
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en,zh-CN;q=0.9,zh;q=0.8,zh-TW;q=0.7,fr;q=0.6

this is the nginx(this is the first reverse proxy node) config look like:

location / {
    client_max_body_size 1m;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_read_timeout 3600;
    proxy_pass http://127.0.0.1:8000;
}

in the kubernetes pod, I print the header like this:

Header: "sec-fetch-site" => "same-origin"
│ Header: "accept-encoding" => "gzip, deflate, br, zstd"
│ Header: "x-forwarded-for" => "127.0.0.1"
│ Header: "sec-ch-ua-platform" => "\"macOS\""
│ Header: "x-action" => "GET_PROJ_LIST"
│ Header: "accept" => "application/json, text/plain, */*"
│ Header: "x-forwarded-host" => "tex.poemhub.top"
│ Header: "user-agent" => "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36"
│ Header: "sec-ch-ua-mobile" => "?0"
│ Header: "x-forwarded-port" => "80"
│ Header: "host" => "tex.poemhub.top"
│ Header: "x-request-id" => "be7ce258-9513-421b-988e-5fc8d7ce312e"
│ Header: "authorization" => "Bearer eyJ0eXAiOiJKV1QiLCjoxNzM
│ Header: "dnt" => "1"
│ Header: "x-forwarded-server" => "iZm5e2jhfbrshckqh6qdbuZ"
│ Header: "x-forwarded-proto" => "http"
│ Header: "referer" => "https://tex.poemhub.top/doc/tab"
│ Header: "x-real-ip" => "127.0.0.1"
│ Header: "sec-ch-ua" => "\"Google Chrome\";v=\"129\", \"Not=A?Brand\";v=\"8\", \"Chromium\";v=\"129\""
│ Header: "sec-fetch-mode" => "cors"
│ Header: "accept-language" => "en,zh-CN;q=0.9,zh;q=0.8,zh-TW;q=0.7,fr;q=0.6"
│ Header: "sec-fetch-dest" => "empty"

all header forwad but the x-forwarded-for and x-real-ip changed to 127.0.0.1 . This is the traefik config look like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik
  namespace: default
  uid: c4166a59-f516-4691-99cf-7d33076bf13a
  resourceVersion: '17412632'
  generation: 7
  creationTimestamp: '2024-07-03T16:07:05Z'
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: traefik
    helm.sh/chart: traefik-10.1.1
    k8slens-edit-resource-version: v1
  annotations:
    deployment.kubernetes.io/revision: '6'
    meta.helm.sh/release-name: traefik
    meta.helm.sh/release-namespace: default
    prometheus.io/path: /metrics
    prometheus.io/port: '9100'
    prometheus.io/scrape: 'true'
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: traefik
      app.kubernetes.io/name: traefik
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: traefik
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: traefik
        helm.sh/chart: traefik-10.1.1
      annotations:
        kubectl.kubernetes.io/restartedAt: '2024-07-09T14:04:02Z'
    spec:
      volumes:
        - name: data
          emptyDir: {}
        - name: tmp
          emptyDir: {}
        - name: config-volume
          configMap:
            name: traefik-config
            defaultMode: 420
      containers:
        - name: traefik
          image: registry.cn-qingdao.aliyuncs.com/reddwarf-public/traefik:v2.10.1
          args:
            - '--global.checknewversion'
            - '--global.sendanonymoususage'
            - '--entryPoints.metrics.address=:9300/tcp'
            - '--entryPoints.traefik.address=:9000/tcp'
            - '--entryPoints.web.address=:8000/tcp'
            - '--entryPoints.web.forwardedHeaders.insecure'
            - '--entryPoints.websecure.address=:8443/tcp'
            - '--entryPoints.web.forwardedHeaders.trustedIPs=127.0.0.1/32,192.168.1.1/32,172.31.227.1/32'
            - '--api.dashboard=true'
            - '--ping=true'
            - '--accesslog=true'
            - '--tracing=true'
            - '--log.level=DEBUG'
            - ‘--log.filePath=/opt/traefik.log’
            - '--metrics.prometheus=true'
            - '--metrics.prometheus.entrypoint=metrics'
            - '--providers.kubernetescrd'
            - '--providers.kubernetesingress'
          ports:
            - name: metrics
              hostPort: 9300
              containerPort: 9300
              protocol: TCP
            - name: traefik
              hostPort: 9000
              containerPort: 9000
              protocol: TCP
            - name: web
              hostPort: 8000
              containerPort: 8000
              protocol: TCP
            - name: websecure
              hostPort: 8443
              containerPort: 8443
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: data
              mountPath: /data
            - name: tmp
              mountPath: /tmp
            - name: config-volume
              mountPath: /etc/traefik
          livenessProbe:
            httpGet:
              path: /ping
              port: 9000
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 2
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /ping
              port: 9000
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 2
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 1
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            runAsUser: 65532
            runAsGroup: 65532
            runAsNonRoot: true
            readOnlyRootFilesystem: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: traefik
      serviceAccount: traefik
      hostNetwork: true
      securityContext:
        fsGroup: 65532
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

Am I missing something? why the traefik changed the header x-forwarded-for and x-real-ip ?

Dolphin
  • 171
  • 1
  • 8

1 Answers1

1

If I understand correctly, the first proxy (nginx) is forwarding traffic to traefik. nginx is setting it's own ip 127.0.0.1 (localhost) for x-forwarded-for and x-real-ip.

This nginx-setting would create a forwared-for list, and set the original real-ip:

 location / {
        proxy_pass http://traefik:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    set_real_ip_from 127.0.0.1/32;    # localhost
    #set_real_ip_from 192.168.1.0/24; # local, docker network ip etc.
    #set_real_ip_from 172.18.0.0/16; 
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

Output:

X-Real-IP: 1.2.3.4
X-Forwarded-For: 1.2.3.4, 127.0.0.1

For just one IP in X-Forwarded-For and not a list of IPs:

proxy_set_header X-Forwarded-For $remote_addr;

set_real_ip_from should take care for sending the correct IP in x-forwared-for.

Add this to the Traefik entrypoint:

forwardedHeaders:
      trustedIPs:
        - "127.0.0.1/32"
        #- "192.168.1.0/24"   # local, docker network
        #- "172.18.0.0/16"

I couldn't test this, make sure to set the correct IPs in set_real_ip_from

Michael D.
  • 193
  • 6