安装airflow

1. 设置环境变量

echo "export AIRFLOW_HOME=/opt/airflow" >> ~/.bashrc
  source ~/.bashrc

2.pip install apache-airflow==1.10.5

3.修改airflow.cfg 配置

注释默认的DB 配置，设置为mysql DB url，在MySQL中新建数据库 airflow

#executor = SequentialExecutor
executor = LocalExecutor

#sql_alchemy_conn = sqlite:////opt/airflow/airflow.db
sql_alchemy_conn = mysql://root:123456@localhost:3306/airflow

4. airflow initdb

如果报错 ModuleNotFoundError: No module named 'MySQLdb'

[2019-10-12 12:18:41,600] {settings.py:213} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=5466
Traceback (most recent call last):
  File "/opt/anaconda3/bin/airflow", line 21, in <module>
    from airflow import configuration
  File "/opt/anaconda3/lib/python3.7/site-packages/airflow/__init__.py", line 44, in <module>
    settings.initialize()
  File "/opt/anaconda3/lib/python3.7/site-packages/airflow/settings.py", line 338, in initialize
    configure_orm()
  File "/opt/anaconda3/lib/python3.7/site-packages/airflow/settings.py", line 225, in configure_orm
    engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
  File "/opt/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py", line 435, in create_engine
    return strategy.create(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/opt/anaconda3/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 118, in dbapi
    return __import__("MySQLdb")
ModuleNotFoundError: No module named 'MySQLdb'

解决方法：

 sudo apt-get install aptitude
sudo aptitude install  libmysqlclient-dev
pip install mysqlclient

Noted: **sudo apt-get install aptitude ** and sudo aptitude install libmysqlclient-dev 是用于解决

 ERROR: Complete output from command python setup.py egg_info:
    ERROR: /bin/sh: 1: mysql_config: not found
    /bin/sh: 1: mariadb_config: not found
    /bin/sh: 1: mysql_config: not found
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-1xi26n1k/mysqlclient/setup.py", line 16, in <module>
        metadata, options = get_config()
      File "/tmp/pip-install-1xi26n1k/mysqlclient/setup_posix.py", line 61, in get_config
        libs = mysql_config("libs")
      File "/tmp/pip-install-1xi26n1k/mysqlclient/setup_posix.py", line 29, in mysql_config
        raise EnvironmentError("%s not found" % (_mysql_config_path,))
    OSError: mysql_config not found
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-1xi26n1k/mysqlclient/

5.启动服务

airflow webserver -p 9888
airflow scheduler