unison + inotify 实现文件实时双向同步部署步骤

一. Unison简介

Unison是Windows、Linux以及其他Unix平台下都可以使用的文件同步工具，它能使两个文件夹（本地或网络上的）保持内容的一致。Unison拥有与其它一些同步工具或文件系统的相同的特性，但也有自身的特点：

1. 跨平台使用；

2. 对内核和用户权限没有特别要求；

3. Unison是双向的，它能自动处理两分拷贝中更新没有冲突的部分，有冲突的部分将会显示出来让用户选择更新策略；

4. 只要是能连通的两台主机，就可以运行unison，可以直接使用socket连接或安全的ssh连接方式，对带宽的要求不高，使用类似rsync的压缩传输协议。

二. 编译安装Unison

unison各种版本下载地址：

http://www.seas.upenn.edu/~bcpierce/unison//download.html

unison编译器下载地址：

http://caml.inria.fr/pub/distrib

从以上地址可以下载各种平台,各种版本的unison,有基于源码安装的,有二进制的,我下载的是二进制的,可以直接使用.这里介绍源码安装:

1.实验环境：

txzlkhdweb1 192.168.16.1 同步目录 /root/test

txzlkhdweb2 192.168.16.2 同步目录 /root/test

2.创建共享文件目录

txzlkhdweb1 :

[root@txzlkhdweb1 /]# mkdir /root/test

txzlkhdweb2:

[root@txzlkhdweb2 ~]# mkdir /root/test

3.安装Objective Caml compiler

Linux下从源码包编译安装，需要一个叫做Objective Caml compiler的工具，版本至少3.0.7，可以从这里下载：http://caml.inria.fr/

[tomcat@txzlkhdweb1 ~]$ cd /usr/local/src/

[tomcat@txzlkhdweb1 ~]$ wget http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz

--2013-05-03 20:23:02-- http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz

正在解析主机 caml.inria.fr... 128.93.11.35

Connecting to caml.inria.fr|128.93.11.35|:80... 已连接。

已发出 HTTP 请求，正在等待回应... 200 OK

长度：3660473 (3.5M) [application/x-gzip]

Saving to: `ocaml-3.12.1.tar.gz'

100%[======================================>] 3,660,473 150K/s in 44s

2013-05-03 20:23:49 (81.5 KB/s) - `ocaml-3.12.1.tar.gz' saved [3660473/3660473]

[root@txzlkhdweb1 src]# tar -zxf ocaml-3.12.1.tar.gz

[root@txzlkhdweb1 src]# cd ocaml-3.12.1

[root@txzlkhdweb1 ocaml-3.12.1]# ./configure

Configuring for a x86_64-unknown-linux-gnu ...

gcc found

........................................

options for linking ...... -lX11

The "labltk" library: not supported

** Objective Caml configuration completed successfully **

[root@txzlkhdweb1 ocaml-3.12.1]# make world opt

make coldstart

make[1]: Entering directory `/usr/local/src/ocaml-3.12.1'

cd byterun; make all

make[2]: Entering directory `/usr/local/src/ocaml-3.12.1/byterun'

sed -n -e '/^ /s/ ([A-Z])/ &&lbl_1/gp'

-e '/^}/q' instruct.h > jumptbl.h

......................................................

../ocamlcompopt.sh -nostdlib -c -g -annot -warn-error A -w L -w R -w Z -I ../otherlibs/unix -I ocamlbuild -I stdlib -o ocamlbuild/ocamlbuild_unix_plugin.cmx ocamlbuild/ocamlbuild_unix_plugin.ml

../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx ocamlbuild/ocamlbuild_executor.cmx ocamlbuild/ocamlbuild_unix_plugin.cmx -o ocamlbuild/ocamlbuildlib.cmxa

../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx -o ocamlbuild/ocamlbuildlightlib.cmxa

make[1]: Leaving directory `/usr/local/src/ocaml-3.12.1'

[root@txzlkhdweb1 ocaml-3.12.1]# make install

if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi

if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi

if test -d /usr/local/lib/ocaml/stublibs; then : ; else mkdir -p /usr/local/lib/ocaml/stublibs; fi

..................................

install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild_executor.o

install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo

install /usr/local/man/man1/ocamlbuild.1

4.编译安装Unison

[root@txzlkhdweb1 src]# wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz

--2013-05-03 22:29:49-- http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz

正在解析主机 www.seas.upenn.edu... 158.130.68.91, 2607:f470:8:64:5ea5::9

Connecting to www.seas.upenn.edu|158.130.68.91|:80... 已连接。

已发出 HTTP 请求，正在等待回应... 200 OK

长度：2694761 (2.6M) [application/x-gzip]

Saving to: `unison-2.40.102.tar.gz'

100%[======================================================>] 2,694,761 113K/s in 34s

2013-05-03 22:30:26 (76.8 KB/s) - `unison-2.40.102.tar.gz' saved [2694761/2694761]

[root@txzlkhdweb1 ocaml-3.12.1]# make install

if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi

if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi

............................

install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo

install /usr/local/man/man1/ocamlbuild.1

[root@txzlkhdweb1 src]# cd unison-2.40.102

[root@txzlkhdweb1 unison-2.40.102]# make UISTYLE=text

ocamlc -o mkProjectInfo unix.cma str.cma mkProjectInfo.ml

./mkProjectInfo > Makefile.ProjectInfo

UISTYLE = text

Building for Unix

NATIVE = true

THREADS = false

STATIC = false

OSTYPE =

OSARCH =

ocamlopt: ubase/rx.mli ---> ubase/rx.cmi

ocamlopt -I lwt -I ubase -I system -I system/generic -I lwt/generic -c /usr/local/src/unison-2.40.102/ubase/rx.mli

.........................................

make[1]: Entering directory `/usr/local/src/unison-2.40.102'

if [ -f `which etags` ]; then

etags *.mli */*.mli *.ml */*.ml */*.m *.c */*.c *.txt

; fi

make[1]: Leaving directory `/usr/local/src/unison-2.40.102'

[root@txzlkhdweb1 unison-2.40.102]# make install

mv /root/bin//unison /tmp/unison-15209

mv: 无法 stat “/root/bin//unison”: 没有那个文件或目录

make: [doinstall] 错误 1 (忽略)

cp unison /root/bin/

cp: 无法创建一般文件“/root/bin/”: 是一个目录

make: *** [doinstall] 错误 1

#出现错误的原因在与Unison默认是将文件Copy到/root/bin目录，但Linux默认是没有该目录的，因此我们需要将生成的可执行文件unison复制到系统的PATH目录。

[root@txzlkhdweb1 unison-2.40.102]# cp unison /usr/local/bin

5.通过SSH登陆到远程主机txzlkhdweb2 ，再将unison复制到slave的PATH目录

[root@txzlkhdweb1 unison-2.40.102]# scp unison root@192.168.16.2:/usr/local/bin/unison

The authenticity of host '192.168.16.2 (192.168.16.2)' can't be established.

RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.16.2' (RSA) to the list of known hosts.

root@192.168.16.2s password:

unison 100% 1473KB 1.4MB/s 00:00

三．配置双机ssh信任

由于unison在远程同步文件夹要登陆远程服务器,因此要配置两机互相信任

本例假设本地机为：192.168.16.1(txzlkhdweb1)

远程机：192.168.16.2(txzlkhdweb2)

1. 在两台机器上创建 RSA密钥

以下操作要在本地机和远程机上都执行一遍

（1）以 root 用户登录

（2）在 root 用户的主目录内创建.ssh 目录并设置正确的权限

[root@txzlkhdweb1 ~]# mkdir ~/.ssh

[root@txzlkhdweb1 ~]# chmod 700 ~/.ssh

[root@txzlkhdweb1 ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

20:4e:83:71:d4:12:e1:b1:a3:d0:fe:b1:8e:9e:ed:a8 root@txzlkhdweb1

[root@txzlkhdweb1 ~]# ssh 192.168.16.1cat /root/.ssh/id_rsa.pub >> authorized_keys

The authenticity of host '192.168.16.1(192.168.16.1)' can't be established.

RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.16.1(RSA) to the list of known hosts.

root@192.168.16.1's password:

[root@txzlkhdweb1 ~]# ssh 192.168.16.2cat /root/.ssh/id_rsa.pub >> authorized_keys

root@192.168.16.2s password:

[root@txzlkhdweb1 ~]# scp authorized_keys 192.168.16.2:/root/.ssh/

root@192.168.16.2's password:

authorized_keys 100% 785 0.8KB/s 00:00

#重启两台服务器的SSH服务

[root@txzlkhdweb1 ~]# /etc/init.d/sshd restart

[root@txzlkhdweb2~]# /etc/init.d/sshd restart

四．unison的使用

1.unison远程使用

使用方法：

# unison <本地目录> ssh://remotehostname(IP)/<远程目录的绝对路径>

例如:

[root@txzlkhdweb1 ~]# /usr/local/bin/unison /root/test ssh://root@192.168.16.1//root/test

Contacting server...

Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]

Looking for changes

Waiting for changes from server

Reconciling changes

Nothing to do: replicas have not changed since last sync.

表示将本机的目录/txzlkhdweb1/root/test和远端主机的/txzlkhdweb2/root/test进行同步。一般的，需要两台机能ssh连接。

注意在主机和目录间又多加了一个 "/"

2.修改参数说明

[root@txzlkhdweb1 .unison]# vim ~/.unison/default.prf

#!/bin/bash

# this script is created by yourname.

# Unison preferences file

#root = /root/test

#root = ssh://root@192.168.16.2//root/test

#force =

#ignore =

batch = true

#repeat = 1

#retry = 3

owner = true

group = true

perms = -1

fastcheck = false

rsync = false

sshargs = -C

xferbycopying = true

log = true

logfile = /root/.unison/unison.log

如果服务器端 unison 可执行文件不在默认目录下，甚至没有 unison 命令（需要你编译一个上传到服务器），则需要使用 -servercmd 参数告诉要执行的服务器 unison 命令位置。

使用 -testserver 参数，则成功链接即退出，也不会去执行目录的比较等后续操作。

• -servercmd xxx

告诉 unison，服务器端的 unison 命令是什么。参见上面的示例。

• -auto

接受缺省的动作，然后等待用户确认是否执行。

• -batch

batch mode, 全自动模式，接受缺省动作，并执行。

• -ignore xxx

增加 xxx 到忽略列表中

• -ignorecase [true|false|default]

是否忽略文件名大小写

• -follow xxx

是否支持对符号连接指向内容的同步

• owner = true (保持同步过来的文件属主)

• group = true (保持同步过来的文件组信息)

• perms = -1 (保持同步过来的文件读写权限)

• repeat = 1 (间隔1秒后,开始新的一次同步检查)

• retry = 3 (失败重试)

• sshargs = -C (使用ssh的压缩传输方式)

• xferbycopying = true

• -immutable xxx

不变目录，扫描时可以忽略

• -silent

安静模式

• -times

同步修改时间

• -path xxx 参数

只同步 -path 参数指定的子目录以及文件，而非整个目录。-path 可以多次出现,例如

unison /home/username ssh://remotehost//home/username

-path shared

-path pub

-path .netscape/bookmarks.html

五．安装inotify 软件

1.查看当前系统是否支持inotify

[root@txzlkhdweb1 test]# ls -l /proc/sys/fs/inotify

总计 0

-rw-r--r-- 1 root root 0 05-04 00:56 max_queued_events

-rw-r--r-- 1 root root 0 05-04 00:56 max_user_instances

-rw-r--r-- 1 root root 0 05-04 00:56 max_user_watches

2.下载inotify 源码包

[root@txzlkhdweb1 ~]# wget https://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

正在解析主机 github.com... 192.30.252.129

Connecting to github.com|192.30.252.129|:443... 已连接。

WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV CA-1':

Issued certificate not yet valid.

已发出 HTTP 请求，正在等待回应... 302 Found

位置：http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz [跟随至新的 URL]

--2013-05-04 01:23:55-- http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

正在解析主机 cloud.github.com... 54.240.164.110, 54.230.113.195, 54.240.164.29, ...

Connecting to cloud.github.com|54.240.164.110|:80... 已连接。

已发出 HTTP 请求，正在等待回应... 200 OK

长度：358772 (350K) [null]

Saving to: `inotify-tools-3.14.tar.gz'

100%[=================================================>] 358,772 356K/s in 1.0s

2013-05-04 01:23:56 (356 KB/s) - `inotify-tools-3.14.tar.gz' saved [358772/358772]

3.编译安装inotify

[root@txzlkhdweb1 ~]# tar zxf inotify-tools-3.14.tar.gz

[root@txzlkhdweb1 ~]# cd inotify-tools-3.14

[root@txzlkhdweb1 inotify-tools-3.14]# make && make install

make all-recursive

make[1]: Entering directory `/root/inotify-tools-3.14'

.......

make[2]: Nothing to be done for `install-data-am'.

make[2]: Leaving directory `/root/inotify-tools-3.14'

make[1]: Leaving directory `/root/inotify-tools-3.14'

4.编写inotify 实时监控脚本

[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches

5000000000

[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events

327679

#事件监控脚本：

[root@txzlkhdweb1 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh

#!/bin/bash

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin

LOCAL_DIR="/root/test/"

REMOTE_DIR="/root/test/"

REMOTE_USER="root"

INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"

UNISION="/usr/local/bin/unison"

#目前iplist.txt 文件中就一个地址 192.168.16.2

IP_LIST_FILE="/usr/local/inotify/iplist.txt"

ATTRIBUTES="modify,delete,create,attrib"

${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files

for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`

${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"

done

Done

[root@txzlkhdweb1 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh

[root@txzlkhdweb1 inotify-tools-3.14]# cd /usr/local/inotify

5.在txzlkhdweb2 上安装同样的inotify工具，步骤同上，此处省略

6.编写同步脚本

[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches

5000000000

[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events

327679

#事件监控脚本：

[root@txzlkhdweb2 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh

#!/bin/bash

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin

LOCAL_DIR="/root/test/"

REMOTE_DIR="/root/test/"

REMOTE_USER="root"

INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"

UNISION="/usr/local/bin/unison"

#目前iplist.txt 文件中就一个地址 192.168.16.1

IP_LIST_FILE="/usr/local/inotify/iplist.txt"

ATTRIBUTES="modify,delete,create,attrib"

${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files

for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`

${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"

done

Done

[root@txzlkhdweb2 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh

[root@txzlkhdweb2 inotify-tools-3.14]# cd /usr/local/inotify

[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &

7. 执行inotify 监控脚本

[root@txzlkhdweb1 inotify-tools-3.14]# nohup ./unison.sh &

[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &

8. 查看命令脚本执行程序是否在执行

[root@txzlkhdweb1 ~]# ps -ef | grep inotify

root 31194 31193 0 May18 pts/0 00:00:00 /usr/local/inotify-tools-3.14/bin/inotifywait -mrq --timefmt %d/%m/%y %H:%M --format %T %w%f -e modify,delete,create,attrib /root/test/

root 31469 24614 0 01:36 pts/0 00:00:00 grep inotify

9. 手工创建文件测试

[root@txzlkhdweb1 ~]# touch /root/test/f1.txt

[root@txzlkhdweb1 ~]# Contacting server...

Contacting server...

Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2//root/test]

Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2 //root/test]

Looking for changes

Waiting for changes from server

Reconciling changes

new file ----> f1.txt

local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0

slave : absent

Propagating updates

UNISON 2.40.102 started propagating changes at 00:22:59.08 on 19 May 2013

[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test

Looking for changes

Waiting for changes from server

Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf

[END] Copying f1.txt

UNISON 2.40.102 finished propagating changes at 00:22:59.09 on 19 May 2013

Saving synchronizer state

Synchronization complete at 00:22:59 (1 item transferred, 0 skipped, 0 failed)

Contacting server...

Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]

Looking for changes

Waiting for changes from server

Reconciling changes

new file ----> f1.txt

local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0

slave : absent

Propagating updates

UNISON 2.40.102 started propagating changes at 00:23:00.09 on 19 May 2013

[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test

Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf

Failed: Destination updated during synchronization

The file f1.txt has been created

100% 00:00 ETAFailed [f1.txt]: Destination updated during synchronization

The file f1.txt has been created

UNISON 2.40.102 finished propagating changes at 00:23:00.09 on 19 May 2013

Saving synchronizer state

Synchronization incomplete at 00:23:00 (0 items transferred, 0 skipped, 1 failed)

failed: f1.txt

#这里同步多次问题暂时不知道如何解决，希望高手指点

Contacting server...

Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]

Reconciling changes

Nothing to do: replicas have not changed since last sync.

Looking for changes

Waiting for changes from server

Reconciling changes

Nothing to do: replicas have been changed only in identical ways since last sync.

参考网址：

http://ixdba.blog.51cto.com/2895551/584334

http://sofar.blog.51cto.com/353572/1271608

来自为知笔记(Wiz)