Sqoop export hive to oracle number format issue resolution

Problem:

java.lang.NumberFormatException
        at java.math.BigDecimal.<init>(BigDecimal.java:459)
        at java.math.BigDecimal.<init>(BigDecimal.java:728)
        at CDR_D_DETAIL_LUC.__loadFromFields(CDR_D_DETAIL_LUC.java:9803)
        at CDR_D_DETAIL_LUC.parse(CDR_D_DETAIL_LUC.java:9630)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:811)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1113)
        at org.apache.hadoop.mapred.Child.main(Child.java:260)

Analyisis:
根据at CDR_D_DETAIL_LUC.parse(CDR_D_DETAIL_LUC.java:9630)这一行,知道是那个字段出了问题,然后检查hive数据源,发现这个字段是\N,是hive中默认的空值。问题的根源在于sqoop将\N当作了一个数值,在解析的时候出现格式异常。

Solution:
在sqoop export中指定空串
sqoop export --connect jdbc:oracle:thin:@192.168.40.119:1521:SPTDI --username CDMA --password sptdi --table CDR_D_DETAIL_LUC --export-dir /user/hive/warehouse/pcmd_1x_hive_etl --input-fields-terminated-by ';' --input-null-string '\\N' --input-null-non-string '\\N' -m 1
注意,空串变成了\\N 而不是\N,原因在于sqoop会将''中的字符串按字面量传给java,而java中字符串'\N'是非法的,要将\进行转义,所以,应该再加上一个\

原文地址:https://www.cnblogs.com/littlesuccess/p/2670014.html