How can I capture output from LFTP? (Output not written to STDOUT or STDERR?)

Question

I would like get access to progress information from lftp. Currently, I'm using curl like so:

curl http://example.com/file -o file -L 2> download.log

This writes curl's progress information to the download.log file, which I can tail to get real-time progress.

But the same approach doesn't work with lftp, either with stdout or stderr. I end up with an empty download.log file, until the transfer is complete.

lftp -e 'get http://example.com/file;quit' 2> download.log
lftp -e 'get http://example.com/file;quit' 1> download.log

When I don't redirect output, I see progress on the screen. When I do redirect output, I stop seeing progress on the screen, but nothing shows up in download.log. After the file transfer is complete, I see the final result, like this - but nothing before:

97618627 bytes transferred in 104 seconds (913.1K/s)

Is lftp doing something unusual with its output - printing to screen without printing to stdout/stderr? Are there other ways of capturing screen output than redirecting stdout/stderr?

Answer 1

It sounds to me like it's buffering its output. You might try the unbuffer expect script (man page).

Answer 2

See for the "log:file/xfer" default setting by entering set command in lftp.

Possible output:

set log:file/xfer /home/USERNAME/.local/share/lftp/transfer_log

You can change this in different ways off course.

lftp -c 'open -e "set log:file/xfer /home/USERNAME/myown.log; mget *.csv; bye" -u uname,psswd 172.16.100.101' >> $log

Answer 3

Check the xfer domain variables :

set xfer:log 1

set xfer:eta-period 5 # every 5 seconds

set xfer:rate-period 20 # average rate

will put log transfer information into ~/.lftp/transfer_log

Not sure you can change the log file destination nevertheless

Answer 4

try thisto capture progress info: lftp sftp://$SFTPUSER:$SFTPPASS@$SFTPHOST -e "cd $DIRECTORY; mget $SFTPFILE; bye" > $SFTPLOG 2>&1