How do "correctly" sync millions of files with rsync and inotify?

Question

Assume I have 1 million files, the file list would be something like 20MB. If I add just one file, inotify will tell rsync to transfer the file list and the new file. My network is not good and I might have to limit the bandwidth (--bwlimit).

Multiple files might be added at once and, given the network transfer speed, there may be multiple rsyncs running at once.

Is it worth it? Is there a better way?

Answer 1

Your case sounds exactly what lsyncd was made for.
It watches the directory with inotify and spawns an rsync with only the changed file(s).
It aggregates changes for a few seconds.
So when you make 5 changes in one second it will only spawn one new sync process to the destination server(s).