spark HA

---

spark HA 的两种实现：

• 基于文件系统的单点恢复(Single-Node Recovery with Local File System)
• 基于zookeeper的Standby Masters(Standby Masters with ZooKeeper)

---

1.基于文件系统的单点恢复

在spark-env里对SPARK_DAEMON_JAVA_OPTS设置：

System property	Meaning
spark.deploy.recoveryMode	Set to FILESYSTEM to enable single-node recovery mode (default: NONE).（设成FILESYSTEM ， 缺省值为NONE）
spark.deploy.recoveryDirectory	The directory in which Spark will store recovery state, accessible from the Master's perspective.（Spark 保存恢复状态的目录）

在spark-env.sh 中添加property

export SPARK_DAEMON_JAVA_OPTS="
    -Dspark.deploy.recoveryMode=FILESYSTEM
    -Dspark.deploy.recoveryDirectory=/home/hadoop/spark-1.2.0-bin-2.2.0/recovery"