1

This is a recurrent topic and I have read some answers on serverfault amongst them:

Folder sizes differ within two copied NTFS file trees (using ntfs-3g)

We have a ubuntu client from which researchers can export to their usb drive or dongle DICOM files that they have scanned with MRI. Typically, it consists of hundreds of files with size from 100kB to a couple of MB in 10 folders. So an exam that is copied can be 3 GB in total. In order to avoid having people plugin in their drive with potential viruses in the windows driven MRI scanner, I've programmed a transfer of the images to a ubuntu 22.04 client where people can download their data. Now some of them get 0 byte files. Here are my thoughts:

  1. I first thought it was because of the format they use on their drives which is often exfat and have recommended ntfs. I have so far no report of 0 byte on ntfs.

  2. When the error happens, the copying process is extremely fast.

  3. I have also recommended using rsync (actually grsync) in order to get a checksum on the target. Some of them, not all of the users, apparently still get the 0 byte files although the system says the copying was done successfully. On the ubuntu client, there are no 0 byte files. But then when they read data on their mac or windows computer, the size of some of the files is 0.

With the same drives they never got the error on the windows driven MRI computer. What can be the issue? The source files are good.

My setup: Ubuntu 22.04 Nvme drive 2TB × 2 in zfs mirror Hdd long term storage 20TB x 2 zfs mirror (data is transferred from nvme to there after 24h)

Is the problem zfs, nvme driver or bad partitions on the exfat dongles?

Thanks

Jonathan Berrebi
  • 11

1 Answers1

0

Thank you Gerald, you were totally right! I made some tests and could reproduce the 0 byte error by just plugging out the usb flash drive when rsync or nautilus copying were said to be done. Once I red the files on Windows, some of the files had 0 byte in size. Then I did the same test but this time I properly unmounted the drive. The files had time to be sync:ed and were correct once read in Windows. The sync can take some time so if many large files are involved one should read on the top of the screen: "device should not be unplugged" after pushing the unmount button in nautilus. Wait until you can read: "Device can be removed".

Jonathan Berrebi
  • 11