Storm安装部署

0x01 环境说明

OS: CentOS-6.5-x86_64
JDK: jdk-8u111-linux-x64
Hadoop: hadoop-2.6.5
Zookeeper: zookeeper-3.4.6
Storm: storm-0.9.7

0x02 安装配置

2.1 下载

可以在下载列表中选择你想要安装的版本，这里下载的是apache-storm-0.9.7.tar.gz

2.2 安装

$ tar -zxvf apache-storm-0.9.7.tar.gz
//重命名
$ mv apache-storm-0.9.7 storm-0.9.7
//建立软链
$ ln -s /home/hadoop/cloud/storm-0.9.7/ /home/hadoop/cloud/storm

2.3 配置环境变量

# set storm environment
export STORM_HOME=/home/hadoop/cloud/storm
export PATH=$PATH:$STORM_HOME/bin

使环境变量生效

# source /etc/profile
# su hadoop
$ source /etc/profile

2.4 配置storm.yaml

需要自己在storm下建立data文件夹。

########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
      - "master"
      - "slave1"
      - "slave2"


nimbus.host: "master"
storm.local.dir: "/home/hadoop/cloud/storm/data"
ui.port: 8888
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
# 
# 
# ##### These may optionally be filled in:
#    
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"

## Metrics Consumers
# topology.metrics.consumer.register:
#   - class: "backtype.storm.metric.LoggingMetricsConsumer"
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"

2.5 将文件分发到其它节点

scp -r /home/hadoop/cloud/storm-0.9.7 hadoop@slave1:~/cloud/
scp -r /home/hadoop/cloud/storm-0.9.7 hadoop@slave2:~/cloud/

注意：需要在所有slave节点建立软链并配置环境变量。

0x03 测试

3.1 启动服务

master节点：启动nimbus和ui服务。

nohup /home/hadoop/cloud/storm/bin/storm ui >/dev/null 2>&1 &
nohup /home/hadoop/cloud/storm/bin/storm nimbus >/dev/null 2>&1 &

先启动ui服务，否则nimbus服务会退出。

slave节点启动supervisor服务。

nohup /home/hadoop/cloud/storm/bin/storm supervisor >/dev/null 2>&1 &

3.2 jps查看进程

master节点

$ jps

2947 QuorumPeerMain
3828 Jps
3485 nimbus
3437 core

slave节点

$ jps

1896 Jps
1628 QuorumPeerMain
1726 supervisor

3.3 查看 UI 状态

http://192.168.32.100:8888/

0x04 参考链接

2017-10-21 11:20:21 星期六