##########################################################################################################
##########################################################################################################
Oozie安装部署：
1、安装需求 System Requirements
Unix (tested in Linux and Mac OS X)
Java 1.6+
Hadoop
Apache Hadoop (tested with 1.0.0 & 0.23.1)
ExtJS library (optional, to enable Oozie webconsole)
ExtJS 2.2
The Java 1.6+ bin directory should be in the command path.
2、上传文件并且解压安装包：
[hadoop@db01 softwares]$ pwd
/opt/softwares
[hadoop@db01 softwares]$ tar -zxvf oozie-4.0.0-cdh5.3.6.tar.gz -C /opt/cdh-5.3.6/
3、在hadoop的core-site.xml配置文件中添加如下内容,重启hadoop集群：
<!-- OOZIE -->
<property>
<name>hadoop.proxyuser.[OOZIE_SERVER_USER].hosts</name>
<value>[OOZIE_SERVER_HOSTNAME]</value>
</property>
<property>
<name>hadoop.proxyuser.[OOZIE_SERVER_USER].groups</name>
<value>[USER_GROUPS_THAT_ALLOW_IMPERSONATION]</value>
</property>
--------------------------------------------------------
<property>
<name>hadoop.proxyuser.hadoop.hosts</name>
<value>db01</value>
</property>
<property>
<name>hadoop.proxyuser.hadoop.groups</name>
<value>*</value>
</property>
4、在oozie安装目录解压hadooplibs jar包：
[hadoop@db01 oozie-4.0.0]$ tar -zxvf oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz
5、创建libext目录
[hadoop@db01 oozie-4.0.0]$ pwd
/opt/cdh-5.3.6/oozie-4.0.0
[hadoop@db01 oozie-4.0.0]$ mkdir libext/
6、cp4步骤解压目录下相应hadooplibs下jar包到5步骤创建libext目录下：
[hadoop@db01 oozie-4.0.0]$ cp -r oozie-4.0.0-cdh5.3.6/hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6/* libext/
7、If using the ExtJS library copy the ZIP file to the libext/ directory.
[hadoop@db01 oozie-4.0.0]$ cp /opt/softwares/ext-2.2.zip libext/
8、打包操作
[hadoop@db01 oozie-4.0.0]$ bin/oozie-setup.sh prepare-war
9、启动hadoop服务
略
10、
[hadoop@db01 oozie-4.0.0]$ cp /opt/cdh-5.3.6/hadoop-2.5.0/etc/hadoop/core-site.xml /opt/cdh-5.3.6/hadoop-2.5.0/etc/hadoop/hdfs-site.xml /opt/cdh-5.3.6/oozie-4.0.0/conf/
11、
----------------------------
[hadoop@db01 oozie-4.0.0]$ bin/oozie-setup.sh sharelib create -fs hdfs://db01:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cdh-5.3.6/oozie-4.0.0/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cdh-5.3.6/oozie-4.0.0/libtools/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cdh-5.3.6/oozie-4.0.0/libext/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
the destination path for sharelib is: /user/hadoop/share/lib/lib_20170324165042
---------------------------------------------------------------------------------------
12、创建数据库
bin/ooziedb.sh create -sqlfile oozie.sql -run DB Connection
13、启动oozie
[hadoop@db01 oozie-4.0.0]$ bin/oozied.sh start
14、oozie-site.xml配置hdoop conf目录：
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/opt/cdh-5.3.6/hadoop-2.5.0/etc/hadoop</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
15、重新启动oozie
[hadoop@db01 oozie-4.0.0]$ bin/oozied.sh stop
[hadoop@db01 oozie-4.0.0]$ bin/oozied.sh start
http://db01:11000 登录console
[hadoop@db01 oozie-4.0.0]$ bin/oozie admin -oozie http://localhost:11000/oozie -status
System mode: NORMAL
16、oozie使用mysql数据库存储元数据
1)修改配置文件：
<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
<description>
JDBC driver class.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://db01:3306/oozie</value>
<description>
JDBC URL.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>root</value>
<description>
DB user name.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>mysql</value>
<description>
DB user password.
IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,
if empty Configuration assumes it is NULL.
</description>
</property>
2）cp mysql 驱动到libext/下：
cp /opt/cdh-5.3.6/hive-0.13.1/lib/mysql-connector-java-5.1.27-bin.jar /opt/cdh-5.3.6/oozie-4.0.0/libext/
3）穿件数据库：
bin/ooziedb.sh create -sqlfile oozie.sql -run DB Connection
4）打包上传hdfs
bin/oozie-setup.sh prepare-war
bin/oozie-setup.sh sharelib create -fs hdfs://db01:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
4）重启即可
[hadoop@db01 oozie-4.0.0]$ bin/oozied.sh stop
[hadoop@db01 oozie-4.0.0]$ bin/oozied.sh start
********************************************************************************************
Examples:
bin/oozie job -oozie http://chavin.king:11000/oozie -config examples/apps/map-reduce/job.properties -run
insert overwrite directory '/user/hadoop/hive/output'
select empno,ename,mgr,job,sal,comm,deptno from chavin.emp;
bin/sqoop import --connect jdbc:mysql://db01:3306/chavin --username root --password mysql --table emp --target-dir ${nameNode}/${oozieDataRoot}/${outputDir} --num-mappers 1 --as-parquetfile
regsvr32.exe C:Program Files (x86)IDM Computer SolutionsUltraEditwodFtpDLX.dll
import --connect jdbc:mysql://db01:3306/chavin
--username root --password mysql --table emp
--target-dir ${nameNode}/${oozieDataRoot}/${outputDir}
--num-mappers 1 --fields-terminated-by "/t"
export --connect jdbc:mysql://chavin.king:3306/chavin --username root --password mysql --table emp --num-mappers 1 --fields-terminated-by "/t" --export-dir /user/hadoop/oozie/datas/bi-select-emp/output
<file>db.hsqldb.properties#db.hsqldb.properties</file>
<file>db.hsqldb.script#db.hsqldb.script</file>
bin/sqoop export
--connect jdbc:mysql://db01:3306/chavin
--username root
--password mysql
--table emp01
--export-dir /user/hadoop/sqoop/import/emp
create table chavin.emp02
(
EMPNO int,
ENAME string,
JOB string,
) row format delimited fields terminated by ' ';
create table chavin.emp01
(
EMPNO int,
ENAME string,
JOB string
) row format delimited fields terminated by ' ';
#########################################################################################################