zookeeper分布式算法和部署


####二阶段提交(Two-Phase Commit) 投票和执行 + 协调者向参与者发送事务内容,询问是否可以提交,各参与者节点执行事务并向协调者反馈 + 如果所有参与者反馈yes,那么提交事务;如果有no或者超时,中断事务回滚。参与者执行完操作后再向协调者发送ack消息; 协调者收到ack消息,完

Paxos算法

提高分布式系统容错性的一致性算法

三个角色

Proposer(提案人),Acceptor(接收人),Learner(学习人)
一个提案被选定需要被半数以上的Acceptor接受

chubby的Quorum机制

过半机制

chubby分布式锁处理

  • 锁延迟,如果锁因为异常情况被释放,chubby会为锁保留一定时间,类似自旋等待
  • 锁序列器,锁的持有者向chubby请求一个锁序列器,包括锁名字,模式,序号,服务端检测验证

Zookeeper

所有机器通过选举确定Leader。Follower和Observer都能提供读服务,Observer机不参与Leader选举过程,也不参与过半写成功策略。

机器节点和数据节点

机器节点是集群中的机器,数据节点是数据模型中的数据单元-Znode,所有数据存储在内存中,数据模型就是一棵树Znode Tree

ZAB(Zookeeper Atomic Broadcast)原子广播协议

崩溃恢复(恢复模式下选举产生新的Leader)和消息广播,其中消息广播协议是基于具有FIFO特性的tcp协议来进行网络通信的,因此容易保证广播过程中消息接受与发送的顺序性


####安装单机伪分布式zk ```bash cd /usr/local/src/ tar xvf zookeeper-3.4.11.tar.gz mv zookeeper-3.4.11 ../zookeeper00 cd ../zookeeper00/ cd conf/ cp zoo_sample.cfg zoo.cfg

cd ../
mkdir {dataDir,dataLogDir}
cd dataDir/
vim myid # 配置该节点的id为1

cp zookeeper00 zookeeper01 -rf
cp zookeeper00 zookeeper02 -rf

<span id="config">
####节点1配置文件
其他节点修改clientPort端口号和dataDir,dataLogDir对应路径,创建对应的myid为2和3
```bash
[root@sjck-node03 conf]# cat zoo.cfg 
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/usr/local/zookeeper00/dataDir
dataLogDir=/usr/local/zookeeper00/dataLogDir
# the port at which the clients will connect
clientPort=2181
# master
server.1=172.16.67.131:2888:3888
server.2=172.16.67.131:2889:3889
server.3=172.16.67.131:2890:3890
####简单集群zk监控脚本 ```bash [root@sjck-node03 zk]# cd /home/soft/zk/ [root@sjck-node03 zk]# cat start.sh #!/bin/bash /usr/local/zookeeper00/bin/zkServer.sh start /usr/local/zookeeper01/bin/zkServer.sh start /usr/local/zookeeper02/bin/zkServer.sh start [root@sjck-node03 zk]# cat stop.sh #!/bin/bash /usr/local/zookeeper00/bin/zkServer.sh stop /usr/local/zookeeper01/bin/zkServer.sh stop /usr/local/zookeeper02/bin/zkServer.sh stop [root@sjck-node03 zk]# cat checkzk.sh #!/bin/bash /usr/local/zookeeper00/bin/zkServer.sh status /usr/local/zookeeper01/bin/zkServer.sh status /usr/local/zookeeper02/bin/zkServer.sh status ``` ####集群zk状态监控 ```bash [root@sjck-node03 zk]# sh start.sh ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper02/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@sjck-node03 zk]# sh checkzk.sh ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg Mode: follower ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg Mode: leader ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper02/bin/../conf/zoo.cfg Mode: follower ``` ####常用监控命令 jps(Java Virtual Machine Process Status Tool) 查看java进程pid,-l输出完整package名,-v输出jvm参数 ```bash [root@sjck-node03 zk]# jps 7984 Jps 7746 QuorumPeerMain 7804 QuorumPeerMain 7773 QuorumPeerMain ``` ####查看监听的端口 netstat -lntp |grep java ####连接 其他节点对应连接,2182,2183端口 ```bash [root@sjck-node03 bin]# ./zkCli.sh -server localhost:2181 ``` ####创建znode,其他节点连接查看数据是否同步 ```bash [zk: localhost:2181(CONNECTED) 0] create /data test-data Created /data [zk: localhost:2181(CONNECTED) 1] ls / [zookeeper, data] [zk: localhost:2181(CONNECTED) 2] get /data ``` ####退出 ```bash [zk: localhost:2181(CONNECTED) 0] quit ```
原文地址:https://www.cnblogs.com/wanli002/p/10461908.html