hadoop CDH5.1.0 配置kerberos

依照下面2个其中一个说明流程配置即可:

1.http://blog.leanote.com/post/54cb1b48f6d5405a8f000005

2.http://www.gfzj.us/series/kerberized_hadoop/2015/03/31/Kerberized_HDFS.html

每次重启机器后运行下面命令获取 kerberos

$ kinit -k -t /etc/hadoop.keytab hadoop/datanode1@hadoop

问题:

2015-08-10 14:40:03,150 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2015-08-10 14:40:03,554 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-08-10 14:40:04,135 INFO org.apache.hadoop.security.UserGroupInformation: Login successful for user hadoop/datanode1@HADOOP using keytab file /etc/hadoop.keytab
2015-08-10 14:40:04,347 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-08-10 14:40:04,385 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-08-10 14:40:04,385 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2015-08-10 14:40:04,387 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is datanode1
2015-08-10 14:40:04,387 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
        at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:737)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:292)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1893)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1780)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1827)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2003)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2027)
2015-08-10 14:40:04,390 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2015-08-10 14:40:04,391 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at datanode1/10.155.9.16
************************************************************/

解决方法:

<!--在hdfs-site.xml中增加如下配置项：忽略linux的secure认证，可以抛开jsvc服务-->
<property>
  <name>ignore.secure.ports.for.testing</name>
  <value>true</value>
 </property>

配置YARN kerberos

修改 yarn-site.xml，添加下面配置：

<property>
  <name>yarn.resourcemanager.keytab</name>
    <value>/etc/hadoop.keytab</value>
    </property>
    <property>
        <name>yarn.resourcemanager.principal</name>
            <value>hadoop/_HOST@HADOOP</value>
            </property>
            <property>
               <name>yarn.nodemanager.keytab</name>
                  <value>/etc/hadoop.keytab</value>
                  </property>
                  <property>
                     <name>yarn.nodemanager.principal</name>
                        <value>hadoop/_HOST@HADOOP</value>
                        </property>

配置mapred-site.xml

<property>
  <name>mapreduce.jobhistory.keytab</name>
    <value>/etc/hadoop.keytab</value>
    </property>
    <property>
      <name>mapreduce.jobhistory.principal</name>
        <value>hadoop/_HOST@HADOOP</value>
        </property>