Hadoop集群维护

HDFS小文件问题及解决方案:http://dongxicheng.org/mapreduce/hdfs-small-files-solution/

Hadoop升级方案(一):Hadoop 1.0内部版本升级(初稿):http://dongxicheng.org/mapreduce-nextgen/hadoop-upgrade-in-version-1/

Hadoop升级方案(二):从Hadoop 1.0升级到2.0:http://dongxicheng.org/mapreduce-nextgen/hadoop-upgrade-to-version-2/

Hadoop 2.2.0设计文档:http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/ClusterSetup.html

Hadoop版本选择探讨:http://dongxicheng.org/mapreduce-nextgen/how-to-select-hadoop-versions/

截至目前(2012年12月23日),Apache Hadoop版本分为两代,我们将第一代Hadoop称为Hadoop 1.0,第二代Hadoop称为Hadoop 2.0。第一代Hadoop包含三个大版本,分别是0.20.x,0.21.x和0.22.x,其中,0.20.x最后演化成1.0.x,变成了稳定版, 而0.21.x和0.22.x则NameNode HA等新的重大特性。第二代Hadoop包含两个版本,分别是0.23.x和2.x,它们完全不同于Hadoop 1.0,是一套全新的架构,均包含HDFS Federation和YARN两个系统,相比于0.23.x,2.x增加了NameNode HA和Wire-compatibility两个重大特性。

Upgrading from CDH3 to CDH4:http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_6_3.html

Upgrading from CDH4 to CDH5:https://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_cdh5_mapreduce.html#../CDH5-Installation-Guide/cdh5ig_to_cdh5_upgrade.html,便于安装相关Storm on YARN,因为Storm on YARN仅支持Hadoop2.1.0-beta和CDH5及以上版本

Note:The YARN APIs changed a lot after 4.3.  Storm YARN does not compile or run on CDH 4.4, 4.5, or 4.6.  It does at least compile on CDH5.0.0 snapshot With a small patch to the master branch. It should work, and has been tested on hadoop 2.1 based builds. Also the master branch will run/compile with just JDK1.6.

原文地址:https://www.cnblogs.com/likai198981/p/3708076.html