Syncing things

Even though I could reduce the pain of using git-annex on the command line with a Ranger plugin, the weird concept and the workflow concerning files which are not read-only let me use it less and less. Moreover, when collaborating with others, I could feel that Dropbox does something right with their system which was not possible to with git-annex in any way. However, since Dropbox is a US-based, commercial system, I was looking for alternatives …

Peer-to-peer synchronization with Syncthing

Syncthing is a dead simple tool to organize arbitrary synchronization networks between devices. Install the syncthing binary and point your browser to localhost:8384. Each device that you run Syncthing on gets a device ID – a secure certificate really – for identification. Devices who know each others device IDs form your personal cluster. You can then setup shared folders between subsets of that cluster that are then synchronized securely using TLS.

2015-11-18/syncthing.png

As you can see, I run Syncthing on three different machines, one is currently off, while the one I currently configure and another are able to synchronize. Note, though that Titan and ipemv1 cannot reliably see each other1, I use an intermediate device to distribute data from these four folders. This device’s rescan interval is set to a relatively large value because I usually won’t change data from there.

Running Syncthing on a VPS

To have a real Dropbox replacement you will need at least one server that is always on – for example a cheap VPS – which will act as the intermediate file server. Be aware though that in case you want to configure this server via the internet, you will either have to secure the connection with HTTPS2 and set up a user and a password or tunnel the HTTP port via SSH to your local machine.

To enable HTTPS, you replace the https-cert.pem and https-key.pem in ~/.config/syncthing or use the default created by Syncthing and modify

<gui enabled="true" tls="false">

in ~/.config/syncthing/config.xml accordingly. To enable SSH tunneling, just forward the remote port (in this case 8384) to some local port which you then use to access the configuration page:

$ ssh -L 9876:127.0.0.1:8384 whereever
$ firefox http://localhost:9876

Ignoring these measures could seriously harm your setup!

Long-term use

Because I only set up this system recently, I do not have any long-term experiences to share with you. Most importantly, I have no clue how often and severe conflicts happen or if the delta sync algorithm is good enough for my use cases. Only time will tell, but so far I am more than happy to retire git-annex for a much simpler setup.

  1. One is behind the home router, the other only accessible via VPN. Moreover, either one is off while the other is on.

  2. One more good reason to get a free certificate.


Discussion