Questions tagged [log-files]

A log file is a file, often in clear text, which is generated by a software process to record some specific details about the workings of this process.

A log file is a file which is generated by a software process to record some specific details (events) about the workings of this process. Most logfiles are plain, human-readable text files, some however are using a binary format to conserve space and need specific treatment for post-processing. (Example: Unix login history in utmp/wtmp format)

A log file can be categorized into several types:

  1. Access log which records information related to the usage of the service provided by this process.
  2. Error log which records error conditions.
  3. Info/Debug log which records more detailed information as compared with other logs.
  4. Transaction log to document a chain of events, this often is required to be immutable to avoid later tampering.
  5. redo logs, containing the complete information to replay a chain of actions, often used where incomplete data transactions would lead to inconsistent data, for example in file systems or databases.

The main two goals of creating and maintaining log files are:

  1. Finding and debugging problems.
  2. Gathering usage information.

Depending on the frequence of events and the level of detail, logfiles may be growing very quickly, and usually require a strategy for managing their growth.

Depending on special reqirements, log-files may go to specialized devices to meet criteria of speed or immutability.

1115 questions
185
votes
7 answers

How can I read pcap files in a friendly format?

a simple cat on the pcap file looks terrible: $cat tcp_dump.pcap ?ò????YVJ? JJ ?@@.?E
Tony
  • 4,393
119
votes
1 answer

What is the purpose of the ".well-known"-folder?

If've found a new error message in our log files and would like to know, for what this .well_known folder is used for. Which application/client would need to access such a folder and which application would create files inside it? Here are some…
Sascha
  • 1,314
111
votes
1 answer

InnoDB: Error: log file ./ib_logfile0 is of different size

I just added the following lines in /etc/mysql/my.cnf after I converted one database to use InnoDB engine. innodb_buffer_pool_size = 2560M innodb_log_file_size = 256M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit =…
jack
  • 1,745
103
votes
9 answers

How can I fully log all bash scripts actions?

From my script output I want to capture ALL the logs data with error messages and redirect them all to log file. I have script like below: #!/bin/bash ( echo " `date` : part 1 - start " ssh -f admin@server.com 'bash…
BlueMark
  • 1,169
99
votes
7 answers

My /var/log/btmp file is huge! What should I do?

My /var/log/btmp file is 1.3 GB in size. I've read that the file is "Used to store information about failed login". What does this mean for my server? And can I delete this file?
Juddling
  • 1,125
  • 1
  • 7
  • 11
88
votes
11 answers

Where is my mysql log on OS X?

I checked /var/log and /usr/local/mysql and i can't seem to find the log. I am trying to troubleshoot an error establishing a database connection with a php function.
Tony
  • 4,393
75
votes
16 answers

Alternatives to Splunk?

I'm pretty impressed with Splunk, especially version 4. Pretty graphs, alerting (Enterprise only), and fast, accurate, searching. It's a great product. However, the cost just way too high to consider for full production use for our company. All we…
MichaelGG
  • 1,739
67
votes
4 answers

What is the difference between syslog, rsyslog and syslog-ng?

I am a bit confused in syslog, rsyslog and syslog-ng. From where can I get the source code for syslog()? Is there any difference between rsyslog and rsyslogd?
StackUser
  • 803
  • 1
  • 7
  • 7
61
votes
13 answers

How to monitor a windows log file in real time?

On windows how can I easily monitor a log file and see updates to the file in real time? Basically, same functionality like tail -f log_file on Unix systems Platform: Windows XP/2003/2008 server [Update] this is quite handy for a quick…
dance2die
  • 2,051
47
votes
2 answers

Understanding a Postfix log file entry

We have a Postfix hub and I'm trying to better understand the information in the mail.log file. I use tools like qshape, pflogsumm.pl and amavis-logwatch to summarize the log files, but I have still have questions about some of the elements of the…
KodeTitan
  • 921
41
votes
8 answers

Colorize Monitoring of Logs

I sometimes monitor apache and php error logs using tail under FreeBSD. Is there any way to get colorized output, either using tail or some other command line app? Alternatively, what is your favorite way to monitor the various web-related logs in…
Ian
  • 1,538
35
votes
3 answers

Grep in a huge log file (>14 GB) only the last x GB?

I need to search something in a huge log-file (over 14 GB). I'm pretty sure it's in the last 4 GB or so. Is there a way to skip the first X GB to speed things up?
Roger
  • 800
35
votes
1 answer

How to set up fail2ban to read multi log in a jail?

How can I configure multiple logpaths for the same rule? I'm trying to write a syntax like this: [apache-w00tw00t] enabled = true filter = apache-w00tw00t action = iptables-allports logpath = /var/log/apache*/*error.log logpath =…
Max121
  • 373
34
votes
8 answers

Can cron write job output to a log *by default* (instead of mail)?

We all know the right way to handle output from cron jobs is to redirect it to a file: 0 * * * * /bin/date >> /var/log/date.log 2>&1 However, sometimes admins are lazy, forgetful, or ignorant and don't put in those redirects; in this case the output…
Brad
  • 442
34
votes
6 answers

How can I display and log PHP errors on IIS7?

We're running PHP 5.2.5 on an IIS 7 Server and we're having problems making PHP errors visible... At the moment whenever we have a PHP error the server sends back a 500 error with the message "The page cannot be displayed because an internal server…
Ben
  • 473
1
2 3
74 75