syncthing

May 28, 2020

Syncthing is a peer-to-peer file sync app, this means that each of your devices communicates with it’s peers, within your own private syncthing network structure. To enable this, you need to decide how you want your devices to interact with each other.

Syncthing has the concepts of ‘devices’ and ‘folders’. Devices are the participants in your peer-to-peer network. Folders are a grouping of files that have the same sharing properties. A single folder can be configured to be shared with all, or just some, or the devices on your network.

devices and network structure

I have five devices participating in my network, each of which has different levels of “uptime”. Devices need to be ON and able to establish a connection to each other for the file sync to happen. This means that it’s important to understand the expected “uptime” of each device. If devices are not “up” at the same time, file changes will not be synced.

Always ON devices:

  • server

Sometimes ON devices:

  • laptop
  • chromebook
  • desktop
  • phone

Which devices are ON at which time impacts the network structure that we choose.

Our options are:

  1. bus
  2. ring
  3. mesh
  4. star

bus networks

Bus networks, are a line. Each device is connected to a single peer. For changes to propagate, both peers in the connection must be ON.

bus network

If a device in the middle of the bus is OFF, this will partition the network into two. This could cause files not to be synchronised correctly. As laptop cannot communicate with server if chromebook is OFF.

A bus network is susceptible to network partitioning when a device is OFF.

Ease of use:

  • Add a device; easy. A new device can be appended to the end of the bus by updating a single ‘end’ peer.
  • Removing a device; easy if an ‘end’, tricky otherwise.

ring network

ring network

A ring network is an extension of a bus network, where the two ends are joined. A ring is more partition tolerant as there are two peer-to-peer propagation routes around the network for changes. It requires two or more devices to be OFF for the network to partition.

Ease of use:

  • Adding a device is a little trickier, as it has to be inserted into the ring by breaking an existing peer-to-peer connection.
  • Removing a device is equivalent to a bus when removing a non ‘end’ device from the bus.

mesh network

mesh network

A mesh network connects every device to every other device. It’s tolerant to single, or multiple device failures, changes will eventually propagate through the entire mesh (assuming the sets of OFF and ON devices are not always exclusive).

Ease of use:

  • Adding a device is very easy. Syncthing includes a setting called “introducer”. The introducer automatically introduces new devices to all peers. A device needs to be manually connected to a single other device in the network, and the introducer will propagate that new devices across the entire network. Meshing the new device in.
  • Removing a device is harder. Every device in the network needs to be updated to fully remove a device from the network.

This setup handles ‘devices’ well, but folders must also be managed. You must choose if you wish new folders to be shared across all peers in the network, and if you want new peers to automatically accept new folder shares.

You will have to configure each device to specify if it should auto accept new folders, and which peers auto-accept can happen from.

The simplest thing would be to auto accept all folders, allowing all folders to mesh over all peers. This is the most resilient, decentralised syncthing setup.

star network

this is the network structure that I use

star network

A star network utilises a centralised model. A single server at the center of the network. Each device interacts only with the central server. This server should be always ON.

I am using a raspberry pi with an external SSD as my server.

The star network structure allows for a number of benefits:

  • Adding a new device only requires changes to the server’s config.
  • Removing a device only requires changes to the server’s config.
  • The server and each device can easily agree which folders should be synced.
  • Individual devices do not need to be online at the same time, as the server will handle the file synchronisation.
  • A network partition will only affect a single device, as the server can still communicate with the other devices.

The downside is that we’ve created a centralised structure where the server going down takes down the whole network.

I’ve setup this structure with the raspberry pi set to auto accept new folder shares from existing devices, so that all folders will be synchronised with the server. Then I can decide individually which other devices that new folder should be propagated to.

synchronisation uses

sync for desktop apps

Some apps, e.g. gnucash, have both a desktop and mobile app. These apps often rely on an external sync functionality, e.g. dropbox, google drive, etc. Syncthing can implement this file synchronisation for you without you needing to signup and use a cloud service. You own and control all your data.

file versioning

Syncthing says that it’s explicitly not a backup tool, but it does support file versioning. Versioning is implemented per-device, per-folder. That is to say that a single device can hold versioned files, and other devices can hold only the most current version.

Using a server (raspberry pi), you can have the server store the file version history (as a pseudo-backup) and each of the client devices can hold only the most recent version. This simplifies the work each client needs to do, and creates a single point of file version history.

conclusion

Syncthing is awesome, it’s got a bunch of different uses and settings. Understanding the structure of your network, and which devices are responsible for implementing which parts of your file synchronisation infrastructure helps to simplify the setup required, and mitigate some gotchas when your files are not propagated as you expect.