0

I have several servers (Linux, various distributions managed by different people) which logs I would like to centralize in splunk>. The logs are gathered in /var/logs but the sources either update them directly (Apache for instance) or via rsyslog. In other words I can assume that the logs will be in on place but the way they appear there is not that defined (and changes between servers).

I am therefore looking for a way to handle /var/logs as a whole by generating a daily delta which I would then send to splunk>. I can write a script which would do such things (parse the tree, gather the files, add to an archive, zero them, etc.) but I am sure this problem has already be resolved in a better way (something along the lines of logrotate but for a whole directory)

(Note: following up on comments I want to stress the fact that I do not have control nor knowledge about the files which will be created in /var/log. Specifically I do not want to rely on solutions which require me to configure the handlong log file by log file)

WoJ
  • 3,875

2 Answers2

3

You should be able to do this by using rsyslog on your systems to send all the logs to a centralised log server.

  • For applications that log via syslog this is fairly straightforward.
  • For applications that log directly to files, rsyslog provides a Text File Input Monitor module that sends lines from a text file to rsyslog for processing.

Regarding logrotate, it's fairly straightforward to configure but you'll have to analyse each log file and configure logrotate appropriately.

user9517
  • 117,122
1

I'm using splunk on my work to handle a logs from a bunch of servers, some linux and some windows

Highly advise you to look at splunk universal forwarder. With it you can choose, what logs to send and basically you can create any scenario on logs treatment, It's way easier to tune, then rsyslog, there is a deployment server, that can help you to manage forwarders later on,

Look at the splunk web-site, there is a simple explanation how to start use it

Hope it will be helpful to you,

AlexiusFlavius
  • 58