Hive导入sequencefile错误

来源：互联网发布：各国知乎编辑：程序博客网时间：2024/06/06 19:23

本地load data到hive表中，可能会由于一些表格式的问题或文本格式问题，导致上传失败。

总结原因：

1. 上传格式和建表格式不匹配

自己上传的为txt文本，而创建表指定的file format 是sequencefile。

hive> load data local inpath '/home/hadoop/ma_test.txt' into table sep26_ma_deposit_dim;Copying data from file:/home/hadoop/ma_test.txtCopying file: file:/home/hadoop/ma_test.txtLoading data to table dw.sep26_ma_deposit_dimFailed with exception Wrong file format. Please check the file's format.FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

思路：

先导入存为textfile,然后再执行MR，覆写这个表

hive> create table tempshengli_test (t1 int, t2 int, t3 int, t4 string) row format delimited fields terminated by '\t' stored as textfile;OKTime taken: 0.256 secondshive> desc tempshengli_test;OKt1      intt2      intt3      intt4      stringTime taken: 0.141 secondshive> load data local inpath '/home/hadoop/ma_test.txt' into table tempshengli_test;                                                      Copying data from file:/home/hadoop/ma_test.txtCopying file: file:/home/hadoop/ma_test.txtLoading data to table dw.tempshengli_testOKTime taken: 4.228 secondshive> insert overwrite into table sep26_ma_deposit_dim as select * from tempshengli_test;FAILED: Parse Error: line 1:17 cannot recognize input near 'into' 'table' 'sep26_ma_deposit_dim' in destination specificationhive> insert overwrite table sep26_ma_deposit_dim select * from tempshengli_test;   Total MapReduce jobs = 2Ended Job = job_201403301416_11274516 Rows loaded to sep26_ma_deposit_dimOKTime taken: 64.341 seconds

drop table tempshengli_test

2.文本格式不匹配

还要注意格式, Unix UTF-8和Windows上的ANSI上传上去不一样。

0 0