异常:System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-mem

报错信息:

System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-mem

spark连接hadoop集群

解决方法:

这主要是运行内存的问题,所以我们在运行的时候更改JVM的资源分配:

或者:

val sparkConf = new SparkConf() .set("spark.testing.memory","2147480000")
原文地址:https://www.cnblogs.com/chong-zuo3322/p/15476624.html