CDH通过rpm包安装
author:wangxing
1. 配置yum源
使用局域网yum源
http://192.168.9.114/cloudera-cdh5.15
2. 安装规划
Hadoop主要包括HDFS和YARN两部分。
通过rpm安装时系统会创建hdfs用户和yarn用户来管理相应的部分，hdfs用户和yarn用户都属于hadoop用户组。
| componet | role | 系统用户 | 用户组 |
|---|---|---|---|
| HDFS | namenode,secondarynamenode,datanode | hdfs | hadoop |
| YARN | resourcemanager,nodemanager | yarn | hadoop |
主机信息及安装规划如下：
| ip | hostname | role |
|---|---|---|
| 192.168.9.111 | earth | namenode,resourcemanager |
| 192.168.9.112 | mercury | secondarynamenode,datanode,nodemanager |
| 192.168.9.113 | venus | datanode,nodemanager |
| 192.168.9.114 | mars | datanode,nodemanager |
| 192.168.9.115 | saturn | datanode,nodemanager |
3. 安装各个组件
- 192.168.9.111 earth
yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-namenode
yum install hadoop-yarn-resourcemanager- 192.168.9.112 mercury
yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-secondarynamenode
yum install hadoop-hdfs-datanode
yum install hadoop-yarn-nodemanager- 192.168.9.112 venus
- 192.168.9.112 mars
- 192.168.9.112 saturn
yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-datanode
yum install hadoop-yarn-nodemanager4. 配置
Hadoop的配置文件目录为/etc/hadoop/conf
hdfs-site.xml
| name | value |
|---|---|
| dfs.namenode.name.dir | file:///xwtec/dfs/nn |
| dfs.namenode.checkpoint.dir | file:///xwtec/dfs/snn |
| dfs.datanode.data.dir | file:///xwtec/dfs/dn |
4.1 启动HDFS
在earth上初始化hdfs并启动namenode
# 初始化HDFS
sudo -u hdfs hadoop namenode -format
# 启动namenode
service hadoop-hdfs-namenode start在mercury上启动secondarynamenode和datanode
service hadoop-hdfs-secondarynamenode start
service hadoop-hdfs-datanode start在venus, mars, saturn上启动datanode
service hadoop-hdfs-datanode start查看HDFS状态 http://192.168.9.111:50070
4.2 启动YARN
在earth上启动resourcemanager
service hadoop-yarn-resourcemanager start在mercury, venus, mars, saturn上启动nodemanager
service hadoop-yarn-nodemanager start查看yarn状态 http://192.168.9.111:8088