基础工作
1. 修改主机名和用户名
2. 配置静态ip
3. 配置ssh无密码连接
4. 安装JDK
5. 配置Hadoop
6. 修改配置文件
* hdfs
* hadoop-env.sh
* core-site.xml
* hdfs-site.xml
* slaves
* yarn
* yarn-env.sh
* yarn-site.xml
* slaves
* mapredue
* mapred-env.sh
* mapred-site.xml
在伪分布式的基础上，只需要将slaves文件修改为
slave1
slave2
利用scp将安装文件夹分发到从节点的相同目录下
scp -r /opt/hadoop-2.6.5 hadoop@slave1:/opt
scp -r /opt/hadoop-2.6.5 hadoop@slave2:/opt
开始测试
格式化hdfs
bin/hdfs namenode -format
启动hdfs
sbin/start-dfs.sh
新建文件夹
bin/hdfs dfs -mkdir -p /user/hadoop01/tmp/conf
上传文件
bin/hdfs dfs -put etc/hadoop/*-site.xml /user/hadoop01/tmp/conf
查看文件
bin/hdfs dfs -text /user/hadoop01/tmp/conf/core-site.xml
启动yarn
sbin/start-yarn.sh
运行wordcount示例
bin/hdfs dfs -mkdir -p /user/hadoop01/mapreduce/wordcount/input
bin/hdfs dfs -put /opt/softwares/hadoop-2.6.5/wcinput/* /user/hadoop01/mapreduce/wordcount/input/
bin/hdfs dfs -ls /user/hadoop01/mapreduce/wordcount/input/
bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar wordcount /user/hadoop01/mapreduce/wordcount/input/ /user/hadoop01/mapreduce/wordcount/output/
查看运行结果
bin/hdfs dfs -text /user/hadoop01/mapreduce/wordcount/output/part*
hdfs界面
http://10.10.0.141:50070/dfshealth.html#tab-overview
yarn界面
http://10.10.0.141:8088/cluster
sbin/mr-jobhistory-daemon.sh start historyserver