前置章节：hadoop集群namenode启动ssh免密登录（hadoop完全分布式五）|11
集群启动
配置workers(3.x之前是slaves)，
删除localhost，添加102/103/104
[shaozhiqi@hadoop102 hadoop]$ pwd
/opt/module/hadoop-3.1.2/etc/hadoop
[shaozhiqi@hadoop102 hadoop]$ vi workers
[shaozhiqi@hadoop102 hadoop]$ cat workers
hadoop102
hadoop103
hadoop104
[shaozhiqi@hadoop102 hadoop]$
注：workers文件结尾不能有空格，文件中不能有空行
同步workers到其他机器
[shaozhiqi@hadoop102 hadoop]$ testxsync workers
fname=workers
pdir=/opt/module/hadoop-3.1.2/etc/hadoop
-------hadoop103 -----------------------
sending incremental file list
workers
sent 113 bytes received 37 bytes 100.00 bytes/sec
total size is 40 speedup is 0.27
------hadoop104 -----------------------
sending incremental file list
workers
sent 113 bytes received 37 bytes 100.00 bytes/sec
total size is 40 speedup is 0.27
[shaozhiqi@hadoop102 hadoop]$
验证是否同步成功
[shaozhiqi@hadoop103 hadoop]$ cat workers
hadoop102
hadoop103
hadoop104
[shaozhiqi@hadoop103 hadoop]$
[shaozhiqi@hadoop104 hadoop]$ cat workers
hadoop102
hadoop103
hadoop104
[shaozhiqi@hadoop104 hadoop]$
停止上节启动的namenode和datanode
[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode
shaozhiqi@hadoop104 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode
shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop datanode
shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/hadoop-daemon.sh stop namenode
版本提示用这个命令：hdfs --daemon stop
启动hdfs
sbin/start-dfs.sh
(hdfs包括了namenode和datanode两个节点。具体可查看大数据及hadoop简要概念|6
[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/start-dfs.sh
Starting namenodes on [hadoop102]
hadoop102: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating.
Starting datanodes
hadoop103: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating.
hadoop104: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating.
Starting secondary namenodes [hadoop104]
[shaozhiqi@hadoop102 hadoop-3.1.2]$
Jps发现其他都正常，但是namenode就是没有启动。。。
删除data logs，格式化namenode
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs namenode –format
再次启动hdfs
[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/start-dfs.sh
Starting namenodes on [hadoop102]
Starting datanodes
hadoop104: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating.
hadoop103: WARNING: /opt/module/hadoop-3.1.2/logs does not exist. Creating.
Starting secondary namenodes [hadoop104]
[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps
7619 DataNode
7877 Jps
7494 NameNode
[shaozhiqi@hadoop102 hadoop-3.1.2]$
验证
102
[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps
7619 DataNode
7877 Jps
7494 NameNode
[shaozhiqi@hadoop102 hadoop-3.1.2]$
103
[shaozhiqi@hadoop103 hadoop-3.1.2]$ jps
2931 Jps
2847 DataNode
[shaozhiqi@hadoop103 hadoop-3.1.2]$
104
[shaozhiqi@hadoop104 hadoop-3.1.2]$ jps
5424 SecondaryNameNode
5467 Jps
5308 DataNode
[shaozhiqi@hadoop104 hadoop-3.1.2]$
参照我们的规划配置，具体查看hadoop集群配置同步（hadoop完全分布式四）|10
配置规划
验证通过
也可以看先我们的hdfs的namenodeweb,可以看到有三台机器ok
namenodeweb
启动yarn
根据我们在[**hadoop****集群配置同步[hadoop集群配置同步（hadoop完全分布式四）|10]
的集群分配原则，我们的yarn的recourceManger在103所以yarn的启动只能在103起，否则会报错
sbin/start-yarn.sh
[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/start-yarn.sh
Starting resourcemanager
Starting nodemanagers
[shaozhiqi@hadoop103 hadoop-3.1.2]$ jps
3154 ResourceManager
3271 NodeManager
3611 Jps
2847 DataNode
[shaozhiqi@hadoop103 hadoop-3.1.2]$
103ok
[shaozhiqi@hadoop104 hadoop-3.1.2]$ jps
5424 SecondaryNameNode
5782 Jps
5657 NodeManager
5308 DataNode
[shaozhiqi@hadoop104 hadoop-3.1.2]$
104ok
[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps
7619 DataNode
8052 NodeManager
8164 Jps
7494 NameNode
[shaozhiqi@hadoop102 hadoop-3.1.2]$
102ok
启动成功
下节测试我们的hadoop集群
由于要关闭电脑，所以停止hadoop集群
启动时先起hdfs,然后yarn
先停yarn 再停hdfs
停集群yarn
[shaozhiqi@hadoop103 hadoop-3.1.2]$ sbin/stop-yarn.sh
Stopping nodemanagers
Stopping resourcemanager
[shaozhiqi@hadoop103 hadoop-3.1.2]$ jps
4145 Jps
2847 DataNode
[shaozhiqi@hadoop103 hadoop-3.1.2]$
停集群hdsf
[shaozhiqi@hadoop102 hadoop-3.1.2]$ sbin/stop-dfs.sh
Stopping namenodes on [hadoop102]
Stopping datanodes
Stopping secondary namenodes [hadoop104]
[shaozhiqi@hadoop102 hadoop-3.1.2]$ jps
8941 Jps
[shaozhiqi@hadoop102 hadoop-3.1.2]$