Spark HA模式访问Hadoop HA下的数据

首先是需要将hadoop的配置文件core-site,xml和hdfs-site.xml 拷贝到Spark conf目录下

然后启动提交即可

spark-submit 
--master spark://hadoop-namenode-01:7077,hadoop-namenode-02:7077 
--class org.apache.spark.examples.SparkPi 
--deploy-mode cluster 
--executor-memory 4G 
--total-executor-cores 10 
hdfs://bi/spark-examples-1.6.0-hadoop2.6.4.jar 
100