hadoop安装

1、tar -zxvf hadoop-2.8.0.tar.gz

[root@localhost java-1.8.0]# useradd hadoop
[root@localhost java-1.8.0]# passwd hadoop

[root@localhost local]# chown hadoop:hadoop hadoop-2.8.0 -R

[root@localhost hadoop-2.8.0]# mkdir tmp

[hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hadoop-env.sh

export JAVA_HOME=/usr   //由于rpm装好 java已经分布在／usr/bin下面

vim etc/hadoop/core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop.master</value>
    </property>
        <property>
        <name>fs.default.name</name>
        <value>hdfs://hadoop.master:8020</value>
    </property>
</configuration>

[hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

./bin/hadoop namenode -format

./sbin/start-dfs.sh

./sbin/start-yarn.sh

网页浏览

http://127.0.0.1:50070/dfshealth.html#tab-overview

如果网页打不开，说明执行不成功，可能哪里配置错误了，start-dfs.sh 的时候看下输出log，进行排查错误

jps检查datanode 是否启动

[root@hadoop hadoop-2.8.0]# yum list | grep jdk-devel

[root@hadoop hadoop-2.8.0]# yum install java-1.8.0-openjdk-devel.x86_64

想在宿主机浏览器访问hadoop web页面，开启某个端口

[root@hadoop hadoop]# service iptables start
Redirecting to /bin/systemctl start iptables.service
Failed to start iptables.service: Unit not found.
[root@hadoop hadoop]# firewall-cmd --zone=public --add-port=50070/tcp --permanent 
success
[root@hadoop hadoop]#   service iptables restart
Redirecting to /bin/systemctl restart iptables.service
Failed to restart iptables.service: Unit not found.
[root@hadoop hadoop]#  
[root@hadoop hadoop]# firewall-cmd --reload
success

在CentOS 7或RHEL 7或Fedora中防火墙由firewalld来管理，

如果要添加范围例外端口如 1000-2000

语法命令如下：启用区域端口和协议组合

firewall-cmd [--zone=] --add-port=[-]/ [--timeout=]

此举将启用端口和协议的组合。端口可以是一个单独的端口或者是一个端口范围 - 。协议可以是 tcp 或 udp。

实际命令如下：

添加

firewall-cmd --zone=public --add-port=80/tcp --permanent （--permanent永久生效，没有此参数重启后失效）

firewall-cmd --zone=public --add-port=1000-2000/tcp --permanent

重新载入

firewall-cmd --reload

查看

firewall-cmd --zone= public --query-port=80/tcp

删除

firewall-cmd --zone= public --remove-port=80/tcp --permanent

重大bug，datanode服务连接不到namenode，查看namenode 8030端口是否有在监听

显示

127.0.0.1:8020 说明只能本台服务器本机请求

需要修改外部服务器可以请求这个接口

<property>
  <name>dfs.namenode.servicerpc-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual address the service RPC server will bind to. If this optional address is
    set, it overrides only the hostname portion of dfs.namenode.servicerpc-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node listen on all interfaces by
    setting it to 0.0.0.0.
  </description>
</property>

<property>
  <name>dfs.namenode.http-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual adress the HTTP server will bind to. If this optional address
    is set, it overrides only the hostname portion of dfs.namenode.http-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node HTTP server listen on all
    interfaces by setting it to 0.0.0.0.
  </description>
</property>

<property>
  <name>dfs.namenode.https-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual adress the HTTPS server will bind to. If this optional address
    is set, it overrides only the hostname portion of dfs.namenode.https-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node HTTPS server listen on all
    interfaces by setting it to 0.0.0.0.
  </description>
</property>

Datanode denied communication with namenode

dfs-site.xml - Need to add following line.

<property> <name>dfs.namenode.datanode.registration.ip-hostname-check</name> <value>false</value></property>