Logstash to aggregate logs into ElasticSearch

Question

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS.

filebeat is already running in EKS to aggregate Kubernetes container logs.

I have configured rsyslog client with :

#/etc/rsyslog.d/50-default.conf
*.*;syslog;auth,authpriv.none   /var/log/syslog

# /etc/rsyslog.d/49-ship-syslog.conf
*.*;syslog;auth,authpriv.none action(
  type="omfwd"
  Target="logstash.dev.domain.com"
  Port="5000"
  Protocol="tcp"
)

and logstash with :

input {
  tcp {
    port => 5000
    type => syslog
  }
}

filter {
    grok {
        match => {
            "message" => "%{SYSLOGLINE}"
        }
    }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "logstash-%{+YYYY.MM.dd}"
  }
  stdout { codec => rubydebug }
}

I am able to do :

$nc -vz -u logstash.dev.domain.com 5000
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
    outif (null)
    src 10.24.11.90 port 58291
    dst 47.176.158.158 port 5000
    rank info not available

Connection to logstash.dev.domain.com port 5000 [udp/commplex-main] succeeded!

also able to send test message :

echo -n "test message" | nc -4u -w1 logstash.dev.domain.com 5000

But don't see anything for tcpdump port 5000 in logstash OR in Kibana UI under Logstash shows Events Received 0

But when I login to logstash container and type echo -n "test message" | nc -4u -w1 logstash.dev.domain.com 5000 then I see this message showing up for tcpdump port 5000.

What I am missing here ?

Version : logstash:7.2.1 rsyslog 3.5

Logstash to aggregate logs into ElasticSearch

0 Answers0