Problem Description:
java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error
SparkException: [Error 1] Topo check failed. Mapred tasks exceed
1000000000
Root Cause:
数据量太大导致,需要设小reduce个数
Solution:
通过设置参数set mapred.reduce.tasks=xx;
Problem Description:
java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error
SparkException: [Error 1] Topo check failed. Mapred tasks exceed
1000000000
Root Cause:
数据量太大导致,需要设小reduce个数
Solution:
通过设置参数set mapred.reduce.tasks=xx;