I would like to copy files from remote directory to local directory with Ansible but fetch module allows me to copy only one file. I have many servers from which I need files (same directory each server) and I don't now how to do this with Ansible.
Any ideas?
You should use the synchronise module to do this. This uses the awesome power of rsync. It will copy file & directory structures of any depth, is bulletproof and highly efficient - only copying the actual bytes that have changed:
- name: Fetch stuff from the remote and save to local
synchronize: src={{ item }} dest=/tmp/ mode=pull
with_items:
- "folder/one"
- "folder/two"
The key is the mode parameter:
Specify the direction of the synchronization. In push mode the localhost or delegate is the source; In pull mode the remote host in context is the source.
You will probably need to register remote content and than loop over it, something like this should work:
- shell: (cd /remote; find . -maxdepth 1 -type f) | cut -d'/' -f2
register: files_to_copy
- fetch: src=/remote/{{ item }} dest=/local/
with_items: "{{ files_to_copy.stdout_lines }}"
where /remote should be changed with directory path on your remote server and /local/ with directory on your master
i dont have enough rep to comment otherwise i would add it.
I used what Kęstutis posted. i had to make a slight modification
- shell: (cd /remote; find . -maxdepth 1 -type f) | cut -d'/' -f2
register: files_to_copy
- fetch: src=/remote/{{ item }} dest=/local/
with_items: "{{ files_to_copy.stdout_lines }}"
The with_items was the area i had to change. it was not able to locate the files otherwise.
well, if you are using latest ansible version, like 2.9.9, I think we need quotes to the item
- name: use find to get the files list which you want to copy/fetch
find:
paths: /etc/
patterns: ".*passwd$"
use_regex: True
register: file_2_fetch
- name: use fetch to get the files
fetch:
src: "{{ item.path }}"
dest: /tmp/
flat: yes
with_items: "{{ file_2_fetch.files }}"
Fixing the example above
- hosts: srv-test
tasks:
- find: paths="/var/tmp/collect" recurse=no patterns="*.tar"
register: files_to_copy
- fetch: src={{ item.path }} dest=/tmp
with_items: "{{ files_to_copy.files }}"
i have prepared a playbook which will help to fetch the nmon performance report from server i have used fetch module with reference of shell module output and able to download the nmon data perfectly
---
#NMON PERFORMANCE REPORT FETCH
- hosts: all
gather_facts: yes
become: yes
vars_prompt:
- name: "date_input"
prompt: "Enter date to fetch nmon report (YYMMDD) = "
private: no
tasks:
- pause:
prompt: "\n\n Enter date to fetch nmon report?\n\n--------------------------------------\n\n1. Fetch nmon:\n2. Exist from Playbook:\n\nPlease select Action: \n--------------------------------------\n"
register: menu
- set_fact:
option: "{{ menu.user_input }}"
- name: Find-out nmon report for given date {{ inventory_hostname }}
shell: |
ls -l /var/nmon/data/* | grep -i {{ date_input }} |awk '{print $NF}'
register: nmon_files_to_copy
when: option == "1"
- name: 'Fetch the NMON Report {{ date_input }} output from server {{ inventory_hostname }}'
fetch:
src: "{{ item }}"
dest: /tmp/
flat: yes
with_items: "{{ nmon_files_to_copy.stdout_lines }}"
when: option == "1"
Try this -
- shell: (cd /remote; find . -maxdepth 1 -type f) | cut -d'/' -f2
register: files_to_copy
- fetch: src=/remote/{{ item }} dest=/local/ flat=yes
with_items: "{{ files_to_copy.stdout_lines }}"
you can compress your folder and fetch that. {{inventory_hostname}} good file name for 'tgz' file in your ansible server
- name: Compress folder
community.general.archive:
path: "/folder/dir"
dest: "/dst/{{inventory_hostname}}.tgz"
- name: Fetch the file from server
fetch:
src: "/dst/{{inventory_hostname}}.tgz"
dest: ./backup/
flat: yes
- hosts: srv-test
tasks:
- find: paths="/var/tmp/collect" recurse=no patterns="*.tar"
register: file_to_copy
- fetch: src={{ item }} dest=/tmp
with_items: files_to_copy.stdout_lines
I use this: 1. Pull directories from remote host to specific hosts
- name: Gather hosts stats from other hosts
shell: " scp -r {{results_root_dir_src}} root@{{groups['profiling_server'][0]}}:{{results_root_dir_dest}}/abc/"
when: "'profiling_server' not in group_names"
#It will not run on the node where the directories need to be copied.
- name: Gather from host to local
delegate_to: 127.0.0.1
run_once: true
become: false
shell: "scp -r root@{{groups['profiling_server'][0]}}:{{results_root_dir}} ./results_local_location "
inventory
[nodes]
server1
server2
server3
[profiling_server]
server1
