How to make scp copy hidden files?

Question

I often use SCP to copy files around - particularly web-related files. The problem is that whenever I do this, I can't get my command to copy hidden files (eg, .htaccess).

I typically invoke this:

scp -rp src/ user@server:dest/

This doesn't copy hidden files. I don't want to have to invoke this again (by doing something like scp -rp src/.* ... - and that has strange . and .. implications anyway.

I didn't see anything in the scp man page about an "include hidden files".

How can I accomplish this?

Answer 1

That should absolutely match hidden files. The / at the end of the source says "every file under this directory". Nevertheless, testing and research bear you out. This is stupid behavior.

The "answer" is to append a dot to the end of the source:

scp -rp src/. user@server:dest/

The real answer is to use rsync.

Answer 2

You can try rsync. It's better suited for this job:

rsync -av src/ user@server:dest/

(And its manual page is worth reading.)

Answer 3

To copy only hidden files, Use this command

scp -rp /path_to_copy_hidden/.[!.]* user@host:/path_to_paste/

Actual game is the /.[!.]* tag that is referring to files starting with .(hidden)

Answer 4

Don't put a slash after the source directory. Your code would look like this:

scp -rp src user@server:dest/

This will create a directory 'src' under 'dest' on the remote machine, with all the hidden files included. It's probably not exactly what you want, but it will copy hidden files in src.

Answer 5

The following will solve the problem, this has been fully tested on our continuous integration environment

scp -rp src/ user@server:dest/
example scp -rp /usr/src/code/ content001@172.11.11.11:/usr/dest/code/

Hope it helps

Answer 6

You can use "shopt -s dotglob" to enable including hidden files and use "shopt -u dotglob" to disable it again

    # Enable dotglob option to include hidden files
    shopt -s dotglob
    # Copy files to the server
    scp -rp src/ user@server:dest/
    # Reset dotglob option
    shopt -u dotglob

This way you keep the same tools and reduce changes ‍♂️

Answer 7

As scp supports regular expressions, this will nicely do the trick for you:

scp -rp src/(*|.*) user@server:dest/

Answer 8

Solution-1

1. Following command recursively copies all contents in current directory, but excludes hidden files directly inside the current directory.

   current_dir$ scp -r ${pwd}/* user@server:dest/
   

2. Following command copies all the hidden files which are there directly inside the current directory, but not inside any of its sub-directories

   current_dir$ scp -r ${pwd}/.[!.]* user@server:dest/
   

Solution-2

# Enable dotglob option to include hidden files
current_dir$ shopt -s dotglob
Copy files to the server "and this should include hidden files as well"
current_dir$ scp -r ${pwd}/* user@server:dest/
Reset dotglob option
current_dir$ shopt -u dotglob

Refs:

https://serverfault.com/a/915383/98910

https://superuser.com/questions/1403473/scp-error-unexpected-filename

https://superuser.com/a/1682929/478378

https://serverfault.com/a/1163655/98910

Answer 9

None of the above scp solutions worked for me. However, I did find that the following worked on cygwin: scp -r directory/* host:directory The '*' matched all visible files and skipped the invisible.

Answer 10

If password login is disabled on the remote machine, and the only way to login is via public key, then you can use this:

$ rsync -av -s 'ssh -i /path/to/your/private/SSH/key' --progress user1@remote.host:/remote/source/directory/ /local/destination/directory/

It copies hidden files too.

Also please note that "user1" must have the permissions to read those files, for example you can't copy other user's ssh folders with this method.

Answer 11

Distributed revision control handles hidden files

Because of the CVE-2018-20685 vulnerability, the /. trick can no longer be used with scp. However, distributed revision control like git or Hg Mecurial will handle hidden files like any other files. Here are the commands for my favourite Hg Mercurial:

server:$ sudo apt install mercurial
client:$ sudo apt install mercurial
client:$ hg init src
client:$ cd src/
client:$ hg addr
client:$ hg com -m "first commit"
client:$ cd
client:$ hg clone src ssh://user@server/dest/

Subsequent changes will need to be committed again with client:$ hg com -m "commit message" and then pushed using the client:$ hg push command. Learn more about pushing changes from this Hg Mercurial cheat sheet.