一、core-site.xml:包含hadoop启动时的信息。
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/xxlong/myInstall/hadoop/tmp</value>
</property>
</configuration>
二、hdfs-site.xml:指定主机namenode和datanode的目录
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/xxlong/myInstall/hadoop/hdfs/name</value>
</property>
<property>
<name>dfs.namenode.data.dir</name>
<value>file:/home/xxlong/myInstall/hadoop/hdfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
三、mapred-site.xml:复制mapred.xml.template文件命名为mapred-site.xml，指定 MapReduce使用的框架。
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
四、yarn-site.xml:包含了MapReduce启动时的配置信息。
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
五、hadoop-env.sh
加入export JAVA_HOME=/home/xxlong/myInstall/jdk1.8.0_45
export HADOOP_LOG_DIR=/home/xxlong/myInstall/hadoop/logs
.profile
export HADOOP_HOME=/home/xxlong/myInstall/hadoop/hadoop-2.5.2
PATH:$HADOOP-HOME/bin
六、ssh免密码登陆
1、sudo apt-get install openssh-server
2、运行 ps -e | grep ssh，查看是否有sshd进程，如果没有/etc/init.d/ssh -start
3、ssh-keygen -t rsa，一路回车
4、cat id_rsa.pub>>authorized_keys
5、chmod 600 authorized_keys
6、如果是远程两个主机A和B
scp /home/A/.ssh/id_rsa.pub B@192.168.1.181:/home/B/.ssh/id_rsa.pub
七、格式化hdfs
hdfs namenode -format
八、启动hdfs
start-dfs.sh
或hadoop-daemon.sh start namenode
hadoop-daemon.sh start datanode
hadoop-daemon.sh start secondarynamenode
九、启动yarn
start-yarn.sh
或hadoop-daemon.sh start resourcemanager
hadoop-daemon.sh start nodemanager
十、hdfs管理界面：http://localhost:50070/
yarn管理界面：http://localhost:8088/cluster
十一、关闭hdfs
stop-dfs.sh
或hadoop-daemon.sh stop namenode
hadoop-daemon.sh stop datanode
hadoop-daemon.sh stop secondarynamenode
十二、stop-yarn.sh
或yarn-daemon.sh stop resourcemanager
yarn-daemon.sh stop nodemanager
十三、eclipse中安装hadoop插件
将hadoop.jar插件复制到eclipse安装目录下的plugins目录中。
1、Windows-->Preferences-->HadoopMap/Reduce-->Hadoop installation
direction:/home/xxlong/myInstall/hadoop/hadoop-2.5.2
2、windows-->showView-->Map/ReduceLocation
3、视图中右击-->New Hadoop location-->Map/Reduce(V2)Master Host:localhost Port:9001 Use M/R Master host Host:localhost Port:9000
4、先创建目录，不然报错。
hadoop fs -mkdir -p /user/xxlong/input
hadoop fs -mkdir -p /user/xxlong/output
十四、查看hadoop码源
新建一个java项目,将/hadoop-2.5.2/share/hadoop/mapreduce/sources下hadoop-mapreduce- examples-2.5.2-sources.jar解压后org包复制到新建项目src包下。
将hadoop-2.5.2-src包压缩成zip格式。eclipse中通过点击类按F3引入
十五、运行wordcount
Arguments:
hdfs://localhost:9000/user/xxlong/input/xxlong.txt hdfs://localhost:9000/user/xxlong/output
VM arguments:
Djava.library.path=/home/xxlong/myInstall/hadoop/hadoop-2.5.2/lib/native