BitTorrent backups - BTsync and Syncthing


Updated: September 11, 2015

Backups are one of the most overlooked pieces of modern computing, both at home and in the work environment. And they usually come down to neglect or overload, with little to no restore ability, regardless of the method and technology chosen for the task. The chief reason is that they are a pain to administer. They can be quite slow.

An alternative is to send your files into the cloud and let them be. But that's risky, right. What about your privacy? What about how slow this can be? Could there be a third option? Indeed, there is one. Backing up your data using P2P networks.

Teaser

Introduction

I have always been a great if silent proponent of decentralized peer-to-peer backup models, ever leery of the security implications of sending your data out there, into the cloud and whatnot. Kind of a double-edged sword. But maybe it can be practical?

Programs like BTsync and Syncthing try to deliver this great promise. In a nutshell, both these tools are glorified BitTorrent clients with a fancy GUI plus added security revolving around the public-private key concept, allowing you to seed your files as secure, private torrents, and then share them among your clients, dispersing your data. This is no different than any download or upload of a Linux distro image, except you do that with your own photos, documents and whatnot. Let's see what these tools can do.

BTSync

The software is available for all platforms, including Windows, Linux, Mac, various mobile operating systems and such. On Linux, the setup is trivial. You just unpack an archive anywhere you want and launch the main program. Then, you connect through the Web console. Just open a browser and navigate to localhost:8888. That's all.

Start BTSync

In the GUI, you go through a quick wizard. You will need to setup a local account, which is basically an .htaccess username and password, so that other people cannot access your sync interface. Then, you setup your devices.

GUI setup

At this point, you will also be somewhat mildly annoyingly reminded that BTSync has a free and a payware (pro) version, which you can get for a 30 day trial, if you are interested.

Get pro version

The next step is to create an identity. And then setup your first device. Later on, we will learn what to do when we add a second device into our pool. As you can imagine, this technology has no value if you have a single device. It becomes more and more useful as the number of inter-connected nodes grows.

Identity

Sync devices

Add folder

Now, you will need to add a folder, name it, set permissions, set expiry, and decide whether you want to explicitly approve new peers, so they can have access to the data. Pretty straightforward.

Permissions

General settings Other options

Once the folder has been created, you can share the link via email, manually or using QR code scans. Likewise, on a second host (or any other for that matter), you will need to provide the identifier hash before you can start syncing and sharing data.

Sync code

Manual connection

Initially, I setup BTSync in Netrunner 15, and then added a Kubuntu 15.04 beta client onto my network. After starting the client, I fired up the Web wizard, then in the link devices step, rather than setting a first device, I decided to connect to a device already running BTSync. And that's about as far as my testing went, because my two devices simply failed to pair. I wasn't able to make any progress beyond this point, and I decided to try the second tool.

Connecting

Well, sorry, I did try another thing. That was to setup BTSync on iPhone, hoping things might actually work. Apple Store asked me for my credit card details, and would not let me install the app, even though it comes with a free version and a trial. But apparently, the mobile version has been designed in a different way. Nope. Don't want.

Syncthing

The second client at our disposal is entirely free. Like BTSync, it is available on multiple platforms. Once again, you grab the tool and run it from the command line. Then, you connect to the Web interface in a browser, port 8080.

Web interface

Syncthing has a simpler if somewhat more confusing GUI. The first thing is to add a new folder for sharing. There aren't that many options, but you can handle or ignore existing file permissions, change the sync check interval, and there's even a decent version control in place. You can also decide whether the syncing should be uni- or bi-directional.

Folder options

Be careful. You need to add folders, and you cannot use symbolic links. If you choose a location that Syncthing cannot write to, it will complained about permissions, but you may not immediately figure out if and why this might pose a problem.

Permission denied error

Not a directory

Every small change requires a client restart, but it's just a quick if somewhat annoying operation, and within seconds, you'll be back in business. It take a few moments for all the nodes to be identified, and to show up in the GUI.

Restart client

To add new nodes into your cluster, click on the cogwheel. Then, you will need to provide long hashes (ID) for each node, on each node. Luckily, if you have multiple devices on the same network, there will be an element of auto-discovery involved. You will need to approve the devices.

Approve new device

After you successfully configure the folder sharing options, it will show in green. Yellow or red means you have a problem. Then, once you restart Syncthing, through the interface, it will run a network scan, searching for available known hosts. You might be prompted to allow sharing your folders with remote nodes.

Share folder prompt

I decided to share a "media" file, which was just 2GB worth of zeros, but good enough to illustrate what a typical sync might look like in some cases. You must appreciate the file name for a moment or two.

File for sharing

The sync worked fine, but it wasn't really fast. It was rather slow, actually, so there's the obvious question of how fast or congested your network really is. P2P and BitTorrent does not mean miracles as some might want you to believe. Then, there was no percentage progress on the source host, but it did show on the target.

Percentage does not show

Syncing

General problems

In general, as I've outlined earlier, the real power of P2P is in numbers. Decentralized numbers. If you happen to have only a small number of devices, you won't really be enjoying the promised ultra-fast backup speeds. Rsync or plain copy will do just as fine. Moreover, how many computing devices does an average household have? One? Five? But not many more than that.

Then, these devices might not have similar storage capabilities. Your smartphone may only hold about 15-30GB worth of data, while desktops can easily swallow TBs without any problem. So you can't really backup anything anywhere. Furthermore, torrents undermine the concept of multiple hard disks, since devices are treated as singular entities, regardless of how many internal storage layers and redundancy they may have.

The third issue is bandwidth. If you don't have a fast line with a hefty upload, you will suffer from slow speed, congestion, and your network will choke while seeding. Some of you may also have quotas, and these can easily be overrun with frequent torrent updates.

If you're running a business with a few hundred identical servers and a very fast 10Gb internal network, this sounds like an ideal concept. But for home users, how is this any different than cloud, unless the focus is entirely on the security, and the fact your data is stored in multiple locations. Then again, most cloud providers have a much better uptime and geo-spread than you can achieve with your own tiny swarm.

Firewall! If you block torrent ports, or do not allow uPnP, then torrents won't magically make your network faster. Speed wise, TCP might be a little slower than UDP, but overall, for home users, you won't see a difference between classic HTTP requests and torrents, especially not when the number of seeds is very small, and this is going to be the typical scenario for most people.

Finally, privacy. You don't want cloud providers to see your data, but what about your family? Or friends? Are you going to share your data with their machines? Can you trust that they will know what to do, not to poke, or worse, get their hosts compromised, allowing total strangers access to your files? Because if you're going to let your personal information out of your hands, then the basic premise of the cloud not being suitable for this purpose kind of becomes pointless.

More reading

If you're interested, then perhaps:

An overview of Grsync and what it can do

A comparison and review of several Rsync frontends

Conclusion

Well, after half a day of frustration and tinkering, I must say that the concept needs a lot of work. It is ideal for large, fast, low-latency, unrestricted, unlimited networks, but not so when you are trying to sync up a bunch of highly diverse devices across the Internet. In that case, you won't really be enjoying the benefits of decentralized storage, and if you have to, the cloud is a better option. Because if you are concerned about privacy or security, then just go for local backups. Faster, more practical.

Ignoring the harsh obstacles of the reality, BTSync did not impress me at all. Syncthing is a better solution, but it might be a little clunky, and it could take a while to setup the cluster. However, there's a lot of potential here, and using encryption and public sharing might actually be a better option. Rather than worrying who you share with, just send your bits and pieces everywhere. Seems to work for all those other torrents. Anyhow, my thanks go to Floris for this recommendation. Take care.

Cheers.

RSS Feed icon

del.icio.us del.icio.us stumbleupon stumble digg digg reddit reddit slashdot slashdot



Advertise!

Would you like to advertise your product/site on Dedoimedo?

Read more

Donate to Dedoimedo!

Do you want to
help me take early retirement? How about donating
some dinero to
Dedoimedo?

Read more

Donate