Difference between amgtar and GNUTAR

Question

We have a few hundred systems backing up to a tape library using AMANDA. Every day we get an email with a long list of STRANGE backup results. Almost all of these are because a log file was being written or a file-based cache was being updated on a web server. For example (5 machines showing a variety of STRANGE messages):

STRANGE DUMP DETAILS:
  /-- av1 / lev 2 STRANGE
  sendbackup: start [avion:/ level 2]
  sendbackup: info BACKUP=/bin/tar
  sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
  sendbackup: info end
  ? /bin/tar: ./boot: directory is on a different filesystem; not dumped
  ? /bin/tar: ./dev: directory is on a different filesystem; not dumped
  ? /bin/tar: ./fs: directory is on a different filesystem; not dumped
  ? /bin/tar: ./proc: directory is on a different filesystem; not dumped
  ? /bin/tar: ./run: directory is on a different filesystem; not dumped
  ? /bin/tar: ./sys: directory is on a different filesystem; not dumped
  ? /bin/tar: ./wwwusers: directory is on a different filesystem; not dumped
  | /bin/tar: ./tmp/.X11-unix/X0: socket ignored
  | /bin/tar: ./tmp/gpg-v9zbWU/S.gpg-agent: socket ignored
  | Total bytes written: 795084800 (759MiB, 740KiB/s)
  sendbackup: size 776450
  sendbackup: end
  \--------
  /-- gitlab / lev 3 STRANGE
  sendbackup: start [gitlab:/ level 3]
  sendbackup: info BACKUP=/bin/tar
  sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
  sendbackup: info end
  | /bin/tar: ./tmp/gitaly-ruby215330643/socket.0: socket ignored
  | /bin/tar: ./tmp/gitaly-ruby215330643/socket.1: socket ignored
  | /bin/tar: ./var/lib/gssproxy/default.sock: socket ignored
  ? /bin/tar: ./var/log/gitlab/gitlab-workhorse/current: file changed as we read it
  ? /bin/tar: ./var/log/gitlab/nginx/gitlab_access.log: file changed as we read it
  | /bin/tar: ./var/opt/gitlab/gitaly/gitaly.socket: socket ignored
  | /bin/tar: ./var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: socket ignored
  | /bin/tar: ./var/opt/gitlab/gitlab-workhorse/socket: socket ignored
  | /bin/tar: ./var/opt/gitlab/postgresql/.s.PGSQL.5432: socket ignored
  | /bin/tar: ./var/opt/gitlab/redis/redis.socket: socket ignored
  | /bin/tar: ./var/spool/postfix/private/anvil: socket ignored
  | /bin/tar: ./var/spool/postfix/private/bounce: socket ignored
  | /bin/tar: ./var/spool/postfix/private/defer: socket ignored
  | /bin/tar: ./var/spool/postfix/private/discard: socket ignored
  | /bin/tar: ./var/spool/postfix/private/error: socket ignored
  | /bin/tar: ./var/spool/postfix/private/lmtp: socket ignored
  Look in the '/var/log/amanda/log.error/gitlab._.3.20180606231049.errout' file for full error messages
  \--------
  /-- haproxy00-127 / lev 3 STRANGE
  sendbackup: start [haproxy00-127:/ level 3]
  sendbackup: info BACKUP=/bin/tar
  sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
  sendbackup: info end
  ? /bin/tar: ./var/log/messages: file changed as we read it
  Look in the '/var/log/amanda/log.error/haproxy00-127._.3.20180607003248.errout' file for full error messages
  \--------
  /-- imap00 /srv/dovecot lev 0 STRANGE
  sendbackup: start [imap00:/srv/dovecot level 0]
  sendbackup: info BACKUP=/bin/tar
  sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
  sendbackup: info end
  ? /bin/tar: ./a1/Maildir/new/1528332018.M.imap00.example.com,S=6309,W=6430: File removed before we read it
  ? /bin/tar: ./a1/Maildir/new/1528335571.M.imap00.example.com,S=8299,W=8451: File removed before we read it
  ? /bin/tar: ./a2/Maildir/cur/1528306776.M.imap00.example.com,S=13517,W=13770\:2,Scd: File removed before we read it
  ? /bin/tar: ./a2/Maildir/cur/1528332257.M.imap00.example.com,S=22257,W=22562\:2,STcd: File removed before we read it
  ? /bin/tar: ./a3/Maildir/cur/1528287062.M.imap00.example.com,S=10343,W=10558\:2,: File removed before we read it
  ? /bin/tar: ./a3/Maildir/cur/1528297383.M.imap00.example.com,S=4179,W=4265\:2,: File removed before we read it
  ? /bin/tar: ./a3/Maildir/cur/1528312816.M.imap00.example.com,S=29364,W=29796\:2,: File removed before we read it
  ? /bin/tar: ./a3/Maildir/new/1528327304.M.imap00.example.com,S=52059,W=53029: File removed before we read it
  ? /bin/tar: ./a4/Maildir/.folder/dovecot.index.log.2: File removed before we read it
  ? /bin/tar: ./a4/Maildir/.Personal/dovecot.index.log.2: File removed before we read it
  ? /bin/tar: ./a4/Maildir/.Trash/cur/1527783480.M.imap00.example.com,S=31976,W=32433\:2,Sab: File removed before we read it
  ? /bin/tar: ./a4/Maildir/cur/1528336066.M.imap00.example.com,S=122100,W=124740\:2,ab: File removed before we read it
  ? /bin/tar: ./a4/Maildir/cur/1528337059.M.imap00.example.com,S=28553,W=28964\:2,ab: File removed before we read it
  Look in the '/var/log/amanda/log.error/imap00._srv_dovecot.0.20180606220538.errout' file for full error messages
  \--------
  /-- webserver1 / lev 2 STRANGE
  sendbackup: start [webserver1:/ level 2]
  sendbackup: info BACKUP=/bin/tar
  sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
  sendbackup: info end
  ? /bin/tar: ./var/log/httpd/kemp_access_log: file changed as we read it
  ? /bin/tar: ./var/log/httpd/wiki.example.com_access_ssl.log: file changed as we read it
  ? /bin/tar: ./var/log/httpd/www.example.com_access.log: file changed as we read it
  ? /bin/tar: ./var/log/httpd/www.example.com_ssl_access_ssl.log: file changed as we read it
  Look in the '/var/log/amanda/log.error/webserver1._.2.20180607004437.errout' file for full error messages
  \--------

(brought to you by Amanda version 3.3.3)

Relevant parts of our current configuration:

define dumptype nocomp-root-tar {
    program "GNUTAR"
    comment "Root partitions with gtar"
    compress none
    estimate server
    priority low
    exclude list "/etc/amanda/amanda.exclude.list"
    auth "bsd"
}
define dumptype nocomp-user-tar {
    program "GNUTAR"
    comment "Non-root partitions with gtar"
    compress none
    estimate server
    priority medium
    exclude list "/etc/amanda/amanda.exclude.list"
    auth "bsd"
}

We don't want to exclude these files from backup just because they might change or disappear. We also can't disable the IMAP server while backups are running. It looks like I can change our systems to use amgtar as their backup method which will let us eliminate all these messages.

define application-tool app_amgtar {
    plugin "amgtar"

    property        "NORMAL" ": socket ignored$"
    property append "NORMAL" "file changed as we read it$"
    property append "NORMAL" ": directory is on a different filesystem; not dumped$"
    property append "NORMAL" ": File removed before we read it$"

}
define dumptype nocomp-root-amgtar {
    program "APPLICATION"
    application "app_amgtar"
    comment "Root partitions with amgtar"
    compress none
    estimate server
    priority low
    exclude list "/etc/amanda/amanda.exclude.list"
    auth "bsd"
}
...

Are there any downsides to amgtar?

Will it be more difficult to recover older backups if we make this change?

The documentation seems to say that only GNUTAR supports exclude list. Does amgtar also support it since it uses GNUTAR underneath?

score 0 · Accepted Answer · answered Oct 31 '19 at 19:09

0

We made the change and haven't had any issues. Unfortunately, no more information about it...

answered Oct 31 '19 at 19:09

yakatz

1,313

Difference between amgtar and GNUTAR

1 Answers1