Problem:
java.lang.NumberFormatException at java.math.BigDecimal.<init>(BigDecimal.java:459) at java.math.BigDecimal.<init>(BigDecimal.java:728) at CDR_D_DETAIL_LUC.__loadFromFields(CDR_D_DETAIL_LUC.java:9803) at CDR_D_DETAIL_LUC.parse(CDR_D_DETAIL_LUC.java:9630) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:811) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1113) at org.apache.hadoop.mapred.Child.main(Child.java:260)
Analyisis:
根据at CDR_D_DETAIL_LUC.parse(CDR_D_DETAIL_LUC.java:9630)这一行,知道是那个字段出了问题,然后检查hive数据源,发现这个字段是\N,是hive中默认的空值。问题的根源在于sqoop将\N当作了一个数值,在解析的时候出现格式异常。
Solution:
在sqoop export中指定空串
sqoop export --connect jdbc:oracle:thin:@192.168.40.119:1521:SPTDI --username CDMA --password sptdi --table CDR_D_DETAIL_LUC --export-dir /user/hive/warehouse/pcmd_1x_hive_etl --input-fields-terminated-by ';' --input-null-string '\\N' --input-null-non-string '\\N' -m 1
注意,空串变成了\\N 而不是\N,原因在于sqoop会将''中的字符串按字面量传给java,而java中字符串'\N'是非法的,要将\进行转义,所以,应该再加上一个\