之前在项目里用MQ是用单节点，因为业务量不大没有主从。这样风险很大，会有单点问题。新项目起来了，需要一个高可用的MQ，故研究了下AMQ的几种master-slave方式：
1、Pure Master-Slave
5.8版本之后已经不支持了，主要原因是Master挂了之后再起来无法自动融入集群，需要停掉MQ重新启动才可以。这里略过。
http://activemq.apache.org/pure-master-slave.html
2、Share File System Master-Slave
通过共享文件系统实现主从节点，解决单点问题。可以理解为将数据放在第三方。
http://activemq.apache.org/shared-file-system-master-slave.html
3、JDBC Master-Slave
数据放在库里，严重依赖数据库的性能，要是数据库挂了更麻烦。
http://activemq.apache.org/jdbc-master-slave.html
3、zookeeper Master-Slave
用过Hadoop的都知道zookeeper是一个强大的调度工具。它可以自动选择可用节点来实现Master-Slave，但太费劲，感兴趣可以戳下面链接
http://activemq.apache.org/replicated-leveldb-store.html
我们准备采用第二种Share File System Master-Slave，简单粗暴，快捷有效。
在192.168.3.31上下载安装AMQ就不用说了，注意机器名hostname里面不能有下划线，一开始我被坑死了，死活启动不了。。。
在192.168.3.31上安装NFS：
yum install nfs-utils nfs4-acl-tools portmap
其中portmap是NFS依赖的服务，必须确认先安装：（记住名字rpcbind）
[root@bxcsweb data]# yum install portmap Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.btte.net * extras: mirrors.btte.net * updates: mirrors.btte.net Setting up Install Process Package rpcbind-0.2.0-11.el6.x86_64 already installed and latest version Nothing to do
直接启动NFS会报错：
[root@bxcsweb data]# service nfs restart 关闭 NFS 守护进程： [失败] 关闭 NFS mountd： [失败] Shutting down RPC idmapd: [失败] 启动 NFS 服务： [确定] 启动 NFS mountd： [失败] 启动 NFS 守护进程：rpc.nfsd: writing fd to kernel failed: errno 111 (Connection refused) rpc.nfsd: unable to set any sockets for nfsd [失败]
先启动依赖，再启动NFS:
service rpcbind restart
service nfs restart
然后在被挂载的主节点机器上设置共享哪些目录、以及共享策略(从节点不能设置,否则会有问题)：（编辑/etc/exports文件）
这个文件默认是空的，编辑的格式：
[共享的目录] [主机名1或IP1(参数1,参数2)] [主机名2或IP2(参数3,参数4)] 下面是一些NFS共享的常用参数： ro 只读访问 rw 读写访问 sync 所有数据在请求时写入共享 async NFS在写入数据前可以相应请求 secure NFS通过1024以下的安全TCP/IP端口发送 insecure NFS通过1024以上的端口发送 wdelay 如果多个用户要写入NFS目录，则归组写入（默认） no_wdelay 如果多个用户要写入NFS目录，则立即写入，当使用async时，无需此设置。 hide 在NFS共享目录中不共享其子目录 no_hide 共享NFS目录的子目录 subtree_check 如果共享/usr/bin之类的子目录时，强制NFS检查父目录的权限（默认） no_subtree_check 和上面相对，不检查父目录权限 all_squash 共享文件的UID和GID映射匿名用户anonymous，适合公用目录。 no_all_squash 保留共享文件的UID和GID（默认） root_squash root用户的所有请求映射成如anonymous用户一样的权限（默认） no_root_squas root用户具有根目录的完全管理访问权限 （不安全） anonuid=xxx 指定NFS服务器/etc/passwd文件中匿名用户的UID anongid=xxx 指定NFS服务器/etc/passwd文件中匿名用户的GID
后来编辑为：
使得31、32机器可以有NFS的/home/shareMQ目录的读写权限
/home/shareMQ 192.168.3.32(rw,sync) 192.168.3.31(rw,sync)
然后restart31机器的NFS服务 （切记要重启一下）
到32机器上尝试mount MQ的数据到31的NFS Server上：
结果报错，错误很明显，不识别-t后面的类型：
[root@bxcscache data]# mount -t nfs 192.168.3.31:/home/shareMQ /usr/local/apache-activemq-5.10.0/data/kahadb/ mount: wrong fs type, bad option, bad superblock on 192.168.3.31:/home/shareMQ, missing codepage or helper program, or other error (for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program) In some cases useful info is found in syslog - try dmesg | tail or so
是因为需要安装NFS工具：
yum install nfs-utils nfs-utils-lib
然后主从节点都执行挂载命令：
mount -t nfs 192.168.3.31:/home/shareMQ /usr/local/apache-activemq-5.10.0/data/kahadb/
31机器上也挂载：
[root@bxcsweb shareMQ]# mount -t nfs 192.168.3.31:/home/shareMQ/ /usr/local/apache-activemq-5.10.0/data/kahadb/
查看已挂载的情况：
31(31是主节点,可以发现后面的服务端客户端两个ip都是自己 而且要在主节点新建/home/shareMQ目录,从节点不用建目录)
[root@bxcsweb kahadb]# mount /dev/vda2 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/vda1 on /main type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) nfsd on /proc/fs/nfsd type nfsd (rw) 192.168.3.31:/home/shareMQ/ on /usr/local/apache-activemq-5.10.0/data/kahadb type nfs (rw,vers=4,addr=192.168.3.31,clientaddr=192.168.3.31)
32
[root@bxcscache kahadb]# mount /dev/vda2 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/vda1 on /main type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) 192.168.3.31:/home/shareMQ on /usr/local/apache-activemq-5.10.0/data/kahadb type nfs (rw,vers=4,addr=192.168.3.31,clientaddr=192.168.3.32)
启动两台机器的activeMQ：
发现有一台一直打出如下日志，注意红色部分。后启动这这个节点一直被当作slave节点，khaha的文件锁已经被先启动的master拿到了，所以10秒尝试获取一次锁：
./activemq start
INFO: Using java '/usr/local/jdk1.6.0_45/bin/java' INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details INFO: pidfile created : '/usr/local/apache-activemq-5.10.0/data/activemq-bxcsweb.pid' (pid '3439') [root@bxcsweb bin]# tail -f ../data/activemq.log 2014-06-13 04:36:09,628 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main 2014-06-13 04:36:09,631 | WARN | Store limit is 102400 mb (current store usage is 0 mb). The data directory: /usr/local/apache-activemq-5.10.0/data/kahadb only has 7555 mb of usable space - resetting to maximum available disk space: 7555 mb | org.apache.activemq.broker.BrokerService | main 2014-06-13 04:36:09,645 | ERROR | Temporary Store limit is 51200 mb, whilst the temporary data directory: /usr/local/apache-activemq-5.10.0/data/localhost/tmp_storage only has 7554 mb of usable space - resetting to maximum available 7554 mb. | org.apache.activemq.broker.BrokerService | main 2014-06-13 04:36:10,104 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main 2014-06-13 04:36:10,167 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main 2014-06-13 04:36:10,439 | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | main 2014-06-13 04:45:26,441 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@68814013: startup date [Fri Jun 13 04:45:26 CST 2014]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main 2014-06-13 04:45:27,850 | INFO | PListStore:[/usr/local/apache-activemq-5.10.0/data/localhost/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main 2014-06-13 04:45:27,895 | INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/usr/local/apache-activemq-5.10.0/data/kahadb] | org.apache.activemq.broker.BrokerService | main 2014-06-13 04:45:27,906 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 04:45:37,911 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 04:45:47,920 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 04:45:57,928 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 04:46:07,936 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 04:46:17,940 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
如果要是master挂了，这个节点会立马获取到锁，然后启动起来：
比如我关掉master，在slave上看到：
2014-06-13 06:25:30,904 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 06:25:40,908 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 06:25:50,913 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 06:26:00,917 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main 2014-06-13 06:26:11,381 | INFO | KahaDB is version 5 | org.apache.activemq.store.kahadb.MessageDatabase | main 2014-06-13 06:26:11,399 | INFO | Recovering from the journal ... | org.apache.activemq.store.kahadb.MessageDatabase | main 2014-06-13 06:26:11,454 | INFO | Recovery replayed 1218 operations from the journal in 0.068 seconds. | org.apache.activemq.store.kahadb.MessageDatabase | main 2014-06-13 06:26:11,622 | INFO | Apache ActiveMQ 5.10.0 (localhost, ID:bxcsweb-43642-1402611971485-0:1) is starting | org.apache.activemq.broker.BrokerService | main 2014-06-13 06:26:11,642 | INFO | Listening for connections at: tcp://bxcsweb:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main 2014-06-13 06:26:11,644 | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | main 2014-06-13 06:26:11,650 | INFO | Listening for connections at: amqp://bxcsweb:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main 2014-06-13 06:26:11,651 | INFO | Connector amqp started | org.apache.activemq.broker.TransportConnector | main 2014-06-13 06:26:11,658 | INFO | Listening for connections at: stomp://bxcsweb:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main 2014-06-13 06:26:11,659 | INFO | Connector stomp started | org.apache.activemq.broker.TransportConnector | main 2014-06-13 06:26:11,669 | INFO | Listening for connections at: mqtt://bxcsweb:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main 2014-06-13 06:26:11,670 | INFO | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main 2014-06-13 06:26:11,821 | INFO | Listening for connections at ws://bxcsweb:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main 2014-06-13 06:26:11,823 | INFO | Connector ws started | org.apache.activemq.broker.TransportConnector | main 2014-06-13 06:26:11,825 | INFO | Apache ActiveMQ 5.10.0 (localhost, ID:bxcsweb-43642-1402611971485-0:1) started | org.apache.activemq.broker.BrokerService | main 2014-06-13 06:26:11,826 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main 2014-06-13 06:26:11,827 | WARN | Store limit is 102400 mb (current store usage is 0 mb). The data directory: /usr/local/apache-activemq-5.10.0/data/kahadb only has 7554 mb of usable space - resetting to maximum available disk space: 7554 mb | org.apache.activemq.broker.BrokerService | main 2014-06-13 06:26:11,829 | ERROR | Temporary Store limit is 51200 mb, whilst the temporary data directory: /usr/local/apache-activemq-5.10.0/data/localhost/tmp_storage only has 7554 mb of usable space - resetting to maximum available 7554 mb. | org.apache.activemq.broker.BrokerService | main 2014-06-13 06:26:12,296 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main 2014-06-13 06:26:12,355 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main 2014-06-13 06:26:12,680 | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | main
至此，ActiveMQ的Share File System Master-Slave部署就算成功结束了。
然后是安全性配置：
activemq.xml已经默认打开了安全性。只用在conf/jetty-realm.properties配控制台登录用户名、密码。
另外连接队列的用户名密码在activemq.xml的<broker>节点中添加以下元素以提供对建立连接时的用户名/密码的支持：
参考了这篇博文：http://www.cnblogs.com/zhongkl/archive/2011/08/12/2136553.html
<plugins> <simpleAuthenticationPlugin> <users> <authenticationUser username="system" password="manager" groups="users,admins"/> <authenticationUser username="user" password="password" groups="users"/> <authenticationUser username="guest" password="password" groups="guests"/> </users> </simpleAuthenticationPlugin> <!-- lets configure a destination based authorization mechanism --> <authorizationPlugin> <map> <authorizationMap> <authorizationEntries> <authorizationEntry queue=">" read="admins" write="admins" admin="admins" /> <authorizationEntry queue="USERS.>" read="users" write="users" admin="users" /> <authorizationEntry queue="GUEST.>" read="guests" write="guests,users" admin="guests,users" /> <authorizationEntry topic=">" read="admins" write="admins" admin="admins" /> <authorizationEntry topic="USERS.>" read="users" write="users" admin="users" /> <authorizationEntry topic="GUEST.>" read="guests" write="guests,users" admin="guests,users" /> <authorizationEntry topic="ActiveMQ.Advisory.>" read="guests,users" write="guests,users" admin="guests,users"/> </authorizationEntries> <!-- let's assign roles to temporary destinations. comment this entry if we don't want any roles assigned to temp destinations --> <tempDestinationAuthorizationEntry> <tempDestinationAuthorizationEntry read="tempDestinationAdmins" write="tempDestinationAdmins" admin="tempDestinationAdmins"/> </tempDestinationAuthorizationEntry> </authorizationMap> </map> </authorizationPlugin> </plugins>
接下来我们看看spring里怎么配：
<!--连接工厂-->
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<!--31、32两台机器连接，断线重连，10毫秒一次，最多尝试60次-->
<value>failover:(tcp://192.168.3.31:61616,tcp://192.168.3.32:61616)?initialReconnectDelay=10&maxReconnectAttempts=60</value>
</property>
<property name="userName" value="***" />
<property name="password" value="***" />
</bean>
OK 结束。
其中，各种搭建过程部分参考了：
http://harbey.iteye.com/blog/1756607
http://shensy.iteye.com/blog/1752529