{"id":64,"date":"2007-07-04T20:53:48","date_gmt":"2007-07-04T19:53:48","guid":{"rendered":"https:\/\/www.dont-panic.cc\/capi\/2007\/07\/04\/sharing-and-synchronizing-data-across-multiple-computers\/"},"modified":"2007-07-04T20:54:45","modified_gmt":"2007-07-04T19:54:45","slug":"sharing-and-synchronizing-data-across-multiple-computers","status":"publish","type":"post","link":"https:\/\/www.dont-panic.cc\/capi\/2007\/07\/04\/sharing-and-synchronizing-data-across-multiple-computers\/","title":{"rendered":"Sharing and Synchronizing Data Across Multiple Computers"},"content":{"rendered":"

I have several computers, one at the office (Windows Vista<\/a>), one at home (Gentoo Linux<\/a>), and one notebook (Windows XP<\/a>). On most of them I want to share a common set of files, including letters and other documents, but also Miranda. This time I am going to tell you how I keep my shared data in sync using Unison<\/a>, PuTTY<\/a>, and OpenSSH<\/a>, using a dedicated server as central hub.<\/p>\n

(Note: this is a rather advisory level HOWTO, not a step-by-step, command-by-command tutorial. It might give you some ideas nevertheless.)<\/p>\n

<\/p>\n

This version of my synchronization solution is based on a central dedicated server on which you can install Unison and login via SSH using public key authentication. You can also do the same with an external media, like an USB-Stick, but there you could use the external media directly without syncing. I would recommend using synching even in this case, as Flash-based media like most USB memory sticks only have limited write cycles. Additionally, if you use synching, you get some redundancy (I wouldn’t go as far as call it “backup”) in case one of your storage devices fails.<\/p>\n

Step 1 – Preparation<\/strong><\/p>\n

Unison<\/a> is one of my favorite two-way synchronization tools. It works by keeping a database for every replica, so it can detect new, modified, and deleted files. The program was originally target at Linux, but there is a client for Windows as well, which works well. Its major advantage in my opinion is that it uses the rsync<\/a>-protocol for transferring the data, which keeps the ordinary synchronization bandwidth reasonably small.<\/p>\n

You have to install Unison on every client that is going to participate. Important: you need to install the same version for every peer!<\/em><\/p>\n

Step 2 – The Central Hub
\n<\/strong><\/p>\n

Then you need the central server. I am using a Gentoo-based dedicates server at Hetzner<\/a>. You don’t need a real server, but it must be an “always-on”, always reachable computer. It could be your gateway at home, but keep in mind you need to contact the host. Consider using DynDNS<\/a> or similar services if you don’t have a static IP for the box. The server can even be one of the target computers, but as mentioned, it needs to be reachable via the Internet for the other sync-partners.<\/p>\n

I created a dm_crypt encrypted partition of 1GB size and mounted it at \/home\/martin\/shared. The 1GB size is enough for me, in fact, the shared data is around 100MB. I am sharing data, not programs or (large) multimedia files.<\/p>\n

Step 3 – Configuring a Linux Peer
\n<\/strong><\/p>\n

Configuration of a Linux peer is the easier than configuration of a Windows peer, that’s why I am starting with this. First you need to create a SSH-key which will be used for authentication if you don’t already have one. Unison needs to be able to login via SSH without user interaction, still I do not recommend to create a key without passphrase. Protect the key with a passphrase, but use something like keychain<\/a> to authenticate only once against the key before starting your synchronization.<\/p>\n

You need to add the key to the ~\/.ssh\/authorized_keys2 file at the central hub<\/em>. Ensure you can now login as the desired user using ssh without a password prompt<\/em>.<\/p>\n

Once SSH is working, you can create a profile for unsion, like my shared.prf<\/em>:<\/p>\n

root = \/home\/martin
\nroot = ssh:\/\/martin@<server>\/\/home\/martin
\npath = shared
\nignore = Path {shared\/local}<\/code><\/p><\/blockquote>\n

This profile synchronizes the ~\/shared\/ directory to the central hub’s ~\/shared\/ directory, ignoring anything at ~\/shared\/local, which gives you the easy possibility to exclude some of your files from synchronization.<\/p>\n

That’s it, you can now synchronize the Linux host to the central hub. Do this anytime you quit working with the computer to prevent conflicts due to concurrent modifications on different workstations.<\/p>\n

Step 4 – Configuring a Windows Peer<\/strong><\/p>\n

Windows configuration is a little bit different (but not much). First I create a TrueCrypt<\/a> volume of the desired size. I like to have all shared folders at the same location, just out of habit. So I mount them all at S:\\ and create a #shared subdirectory, which is going to contain the shared files. (Using TrueCrypt is entirely optional and not required, I just want to make it thieves a little bit harder to read my personal data.)<\/p>\n

Then you need to have PuTTY<\/a> (including plink and (advisable) pageant). Create a key and add it to the central hub’s ~\/.ssh\/authorized_keys2 files. I again advise against keys without password. Use pageant to cache the credentials for synchronization.<\/p>\n

The Unison profile needs to be different for Windows, as you need to convince Unison to use PuTTY instead of ssh. Therefore a small batch-script is required, I call it launcher_server.cmd<\/code> (containing just one line, remove the word wrap):<\/p>\n

@\"C:\\Program Files\\PuTTY\\plink.exe\" -i \"S:\\putty-key.ppk\" martin@my.server.domain unison -server<\/code><\/p><\/blockquote>\n

Make sure the connection is able to login without further authentication.<\/p>\n

The shared.prf<\/code> needs to be a little bit different here:<\/p>\n

root = S:\/#shared
\nroot = ssh:\/\/martin@<server>\/\/home\/martin
\nsshcmd = S:\/Program Files\/Unison\/launcher_server.cmd
\nfastcheck = true
\npath = shared
\nignore = Path {shared\/local}<\/code><\/p><\/blockquote>\n

(Note: the “fastcheck” attribute is optional, but I recommend using it on Windows, but see the Unison documentation<\/a> for further infos before using it!)<\/p>\n

Step 5 – Synchronize<\/strong><\/p>\n

After you have set up all the hosts you want to participate, simply use it. Use keychain and pageant to ensure you don’t need a password for login at the central hub. You should synchronize whenever you log off or shut down a workstation, to ensure there will be as little conflicts as possible. As long as you don’t create conflicts, even concurrent modifications can be handled automatically.<\/p>\n

And now, have fun and enjoy working with a consistent dataset over multiple computers. \ud83d\ude09<\/p>\n","protected":false},"excerpt":{"rendered":"

I have several computers, one at the office (Windows Vista), one at home (Gentoo Linux), and one notebook (Windows XP). On most of them I want to share a common set of files, including letters and other documents, but also Miranda. This time I am going to tell you how I keep my shared data … Continue reading “Sharing and Synchronizing Data Across Multiple Computers”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,5,10],"tags":[36,44,43],"class_list":["post-64","post","type-post","status-publish","format-standard","hentry","category-internet","category-software","category-sysadmin","tag-howto","tag-synchronization","tag-unison"],"_links":{"self":[{"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/posts\/64","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/comments?post=64"}],"version-history":[{"count":0,"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/posts\/64\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/media?parent=64"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/categories?post=64"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dont-panic.cc\/capi\/wp-json\/wp\/v2\/tags?post=64"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}