MapReduce——LongWritable cannot be cast to org.apache.hadoop.io.Text 错误原因

来源:互联网 发布:怎样关闭136端口 xp 编辑:程序博客网 时间:2024/06/08 02:06

            运行环境:虚拟机,Ubuntu16,Ubuntu Server 做Hadoop集群(一主两从),编程软件eclipse。

           运行时出现如下错误:

java.lang.Exception: java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text

(Hadoop的输入LongWritable不能转变为Text)


When you read a file with a M/R program, the input key of your mapper should be the index of the line in the file, while the input value will be the full line.So here what's happening is that you're trying to have the line index as a Text object which is wrong, and you need an LongWritable instead so that Hadoop doesn't complain about type.


        在stackoverflow上找到了答案:当你用MapReduce程序读入文件是,mapper的输入key值应该是文件当前行的索引值,文件当前行的所有内容会被当做mapper的输入value值。所以Mapper程序的第一个参数(Mapper类的输入Key值)不可以具体化为LongWritable,可以用Text或者Object类。


参考stackoverflow:

https://stackoverflow.com/questions/11784729/hadoop-java-lang-classcastexception-org-apache-hadoop-io-longwritable-cannot

阅读全文
1 0
原创粉丝点击