Show HN: Dsync: a tool to sync source directories/files to a target directory

7 points

2 years ago

Hello HN. I worked on this small project recently called dsync. I have a few directories where I keep adding stuff and I need to sync them to an external drive from time to time. In the spirit of "write your own tools", I wrote my own. The tool itself is really simple. It tries to sync source directories or files to a destination directory based on size and modification time, basically copying the source files if they don't match.

dsync can use multiple threads (specified via an option). It uses a bounded multi-producer multi-consumer queue (C implementation of Dmitry Vyukov's bounded MPMC queue using gcc atomic builtins). The main thread traverses the given sources and adds the files to the queue. Other threads dequeue entries from the queue and do the sync/copy work concurrently.

dsync performs well with multiple threads compared to GNU cp when copying directories with a lot of small files such as the linux kernel repository. More details are in the README.

8 comments