Hadoop生态圈-Azkaban实战之Command类型多job工作流flow
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
Azkaban内置的任务类型支持command、java。
一.Command类型单一job示例
1>.创建job描述文件
2>.将job资源文件打包成zip文件
3>.通过azkaban的web管理平台创建project并上传job压缩包（首先创建project）
4>.上传zip包
5>.启动执行该job
6>.设置周期任务并开始执行job任务
7>.工程变为绿色说明job执行成功
8>.查看job list
二.Command类型多job工作流flow
1>.创建有依赖关系的多个job描述
[yinzhengjie@s101 ~]$ more foo.job # foo.job type=command command=echo foo [yinzhengjie@s101 ~]$ [yinzhengjie@s101 ~]$ more bar.job # bar.job type=command dependencies=foo command=echo bar [yinzhengjie@s101 ~]$
2>.将所有job资源文件打到一个zip包中
[yinzhengjie@s101 ~]$ yum -y install zip Loaded plugins: fastestmirror You need to be root to perform this command. [yinzhengjie@s101 ~]$ [yinzhengjie@s101 ~]$ [yinzhengjie@s101 ~]$ sudo yum -y install zip [sudo] password for yinzhengjie: Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 epel/x86_64/metalink | 6.3 kB 00:00:00 epel | 3.2 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 epel/x86_64/primary FAILED 0% [ ] 0.0 B/s | 0 B --:--:-- ETA http://mirrors.aliyun.com/epel/7/x86_64/repodata/d214b3e89f48b6e4033ee71f60676d1c912c1a2726241f62de9181243dacaca1-primary.xml.gz: [Errno 14] HTTP Error 404 - Not Found ] 0.0 B/s | 0 B --:--:-- ETA Trying other mirror. To address this issue please refer to the below knowledge base article https://access.redhat.com/articles/1320623 If above article doesn't help to resolve this issue please create a bug on https://bugs.centos.org/ (1/3): epel/x86_64/updateinfo | 925 kB 00:00:07 (2/3): updates/7/x86_64/primary_db | 2.7 MB 00:00:23 (3/3): epel/x86_64/primary | 3.5 MB 00:00:21 Determining fastest mirrors * base: mirror.bit.edu.cn * epel: mirrors.tongji.edu.cn * extras: mirror.bit.edu.cn * updates: mirror.bit.edu.cn epel 12601/12601 Resolving Dependencies --> Running transaction check ---> Package zip.x86_64 0:3.0-11.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =========================================================================================================================================================================================================================================== Package Arch Version Repository Size =========================================================================================================================================================================================================================================== Installing: zip x86_64 3.0-11.el7 base 260 k Transaction Summary =========================================================================================================================================================================================================================================== Install 1 Package Total download size: 260 k Installed size: 796 k Downloading packages: zip-3.0-11.el7.x86_64.rpm | 260 kB 00:00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : zip-3.0-11.el7.x86_64 1/1 Verifying : zip-3.0-11.el7.x86_64 1/1 Installed: zip.x86_64 0:3.0-11.el7 Complete! [yinzhengjie@s101 ~]$
[yinzhengjie@s101 data]$ cat /home/yinzhengjie/bar.job # bar.job type=command dependencies=foo command=echo bar [yinzhengjie@s101 data]$ cat /home/yinzhengjie/foo.job # foo.job type=command command=echo foo [yinzhengjie@s101 data]$ [yinzhengjie@s101 data]$ zip yinzhengjie.zip /home/yinzhengjie/bar.job /home/yinzhengjie/foo.job adding: home/yinzhengjie/bar.job (deflated 9%) adding: home/yinzhengjie/foo.job (deflated 15%) [yinzhengjie@s101 data]$ ll | grep yinzhengjie.zip -rw-rw-r-- 1 yinzhengjie yinzhengjie 460 Jun 24 20:45 yinzhengjie.zip [yinzhengjie@s101 data]$ [yinzhengjie@s101 data]$ sz yinzhengjie.zip [yinzhengjie@s101 data]$
3>.在azkaban的web管理界面创建工程并上传zip包（过程和上面的上传步骤类似）
4>.在web和后台查看日志信息
