Hive on tez 执行作业时报错请求内存大于允许内存

1、问题描述

Status: Failed Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1597977573448_0003_1_00 [Map 9], Requested TaskResource=<memory:4096, vCores:1>, Cluster MaxContainerCapability=<memory:2048, vCores:2> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1597977573448_0003_1_00 [Map 9], Requested TaskResource=<memory:4096, vCores:1>, Cluster MaxContainerCapability=<memory:2048, vCores:2>

2、问题原因:

hive.tez.container.size设置了4096内存,超过了yarn的容器允许的最大内存,yarn的nodemanager.resource.memory-mb设置的过小,需要将调整改值。

或者调整hive.tez.container.size的值小于nodemanager.resource.memory-mb的值。

3、解决方法

将nodemanager.resource.memory-mb调整为6g或者将hive.tez.container.size的值改为3g。

原文地址:https://www.cnblogs.com/chhyan-dream/p/15703109.html