I install a fluentd chart on OKD 3.11 with:
helm install kiwigrid/fluentd-elasticsearch \
--name fluentd \
-f ../../tools/fluentd/values.yaml \
--namespace logging
The pod is created, I can enter in it and I can curl to external elasticsearch successfully. However the liveness probe keeps failing and the pods on timeout get recreated:
Normal Killing 10m (x3 over 36m) kubelet node1.qa1.isp.ezbob.com
Killing container with id docker://fluentd-fluentd-elasticsearch:Container failed liveness probe.. Container will be killed and recreated.
And I can't find any indexes being created in ES.
The liveness config is the default for the chart:
Liveness: exec [/bin/sh -c LIVENESS_THRESHOLD_SECONDS=${LIVENESS_THRESHOLD_SECONDS:-300}; STUCK_THRESHOLD_SECONDS=${STUCK_THRESHOLD_SECONDS:-900}; if [ ! -e /var/log/fluentd-buffers ]; then
exit 1;
fi; touch -d "${STUCK_THRESHOLD_SECONDS} seconds ago" /tmp/marker-stuck; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-stuck -print -quit)" ]; then
rm -rf /var/log/fluentd-buffers;
exit 1;
fi; touch -d "${LIVENESS_THRESHOLD_SECONDS} seconds ago" /tmp/marker-liveness; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-liveness -print -quit)" ]; then
exit 1;
fi;
] delay=600s timeout=1s period=60s #success=1 #failure=3
The OS is CentOS 7.6. I can't get any info about this issue and no trace to catch, it looks like some permission issues on mounting points.
