Error: java.io.IOException: Can't export data, please check failed map task logs

来源：互联网发布：数据分析师是青春饭吗编辑：程序博客网时间：2024/06/03 16:20

Error: java.io.IOException: Can't export data, please check failed map task logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.RuntimeException: Can't parse input data: '\N'
at sd_everyday_order_info.__loadFromFields(sd_everyday_order_info.java:464)
at sd_everyday_order_info.parse(sd_everyday_order_info.java:382)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
Caused by: java.lang.NumberFormatException: For input string: "\N"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:492)
at java.lang.Integer.valueOf(Integer.java:582)
at sd_everyday_order_info.__loadFromFields(sd_everyday_order_info.java:436)

... 12 more

——————————————-

改正：

sqoop export --connect "jdbc:mysql://192.168.0.197:3306/dac_lots?useUnicode=true&characterEncoding=utf-8" --username root --password root --table sd_everyday_order_info --export-dir '/user/hive/warehouse/totallots.db/sd_everyday_order_info' --input-null-string "\\\\N" --input-null-non-string "\\\\N" --input-fields-terminated-by "\001" --input-lines-terminated-by "\\n" -m 1

说明：

sqoop导入hive数据到MySql碰到hive表中列的值如果为null的情况，hive中为null的是以\N代替的，所以你在导入到MySql时，需要加上两个参数：--input-null-string '\\N' --input-null-non-string '\\N'，多加一个'\'，是为转义。

0 0