hadoop编译WORDCOUNT
来源:互联网 发布:单片机与usb连接电路图 编辑:程序博客网 时间:2024/06/05 17:09
hadoop fs -ls output
hadoop fs -ls
1,首先先配置好JDK2,并配置SSH免密码登陆
3,安装HADOOP,这里用的是版本hadoop-0.20.2.tar.gz,可上CSDN下载那里找到这个版本。
4,安装伪分布式。
可以把伪分布式的hadoop看做只有一个节点的集群,在这个集群中,这个节点即是master,也是slave;既是NameNode,也是DATANODE;既是jobtracker,也是tasktracker。
解压之后,需修改几个文件。
5,在hadoop的安装目录下,先讲Wordcount.java这个例子打包生成JAR文件。
[root@localhost hadoop-0.20.2]# mkdir FirstJar[root@localhost hadoop-0.20.2]# cp src/examples/org/apache/hadoop/examples/WordCount.java FirstJarjavac -classpath hadoop-0.20.2-core.jar WordCount.java Wordcount
假如是以上这个版本的话会报错,以为还缺少一个JAR
编译时需加入commons-cli-1.2.jar
命令如下由于编译时需注意包的问题,将WordCount.java放到一个文件下,编译时需注意包的问题
javac -classpath hadoop-0.20.2-core.jar:/root/hadoop-0.20.2/lib/commons-cli-1.2.jar -d FirstJar FirstJar/WordCount.java特别注意引入commons-cli-1.2.jar请使用绝对路径!!!!
会生成已文件org/apache/hadoop/examples/WordCount 的三个CLASS文件
打包JAR文件
[root@localhost hadoop-0.20.2]# jar -cvf Wordcount.jar -C FirstJar/ .
记住后面有个 . !!!!!!!!!!成功后的页面
added manifestadding: WordCount.java(in = 2395) (out= 813)(deflated 66%)adding: org/(in = 0) (out= 0)(stored 0%)adding: org/apache/(in = 0) (out= 0)(stored 0%)adding: org/apache/hadoop/(in = 0) (out= 0)(stored 0%)adding: org/apache/hadoop/examples/(in = 0) (out= 0)(stored 0%)adding: org/apache/hadoop/examples/WordCount$TokenizerMapper.class(in = 1790) (out= 765)(deflated 57%)adding: org/apache/hadoop/examples/WordCount.class(in = 1911) (out= 996)(deflated 47%)adding: org/apache/hadoop/examples/WordCount$IntSumReducer.class(in = 1789) (out= 746)(deflated 58%)
之后查看hadoop的目录,会发现生成了Wordcount.jar这个JAR接着上传输入文件(输入文件是file01,file02,存放在input)
[root@localhost hadoop-0.20.2]# bin/hadoop dfs -mkdir input
可能会报这个错误: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/hadoop/input. Name node is in safe mode.hadoop dfsadmin -safemode leave
即可解除安全模式,在分别创建两个file01,file02,内容自定
hadoop jar wordCount.jar org.apache.hadoop.examples.WordCount input output
注意类要加上包的路径。hadoop fs -lshadoop fs -ls output
hadoop fs -cat output/part-r-00000
嘿嘿,终于成功了!
- hadoop编译WORDCOUNT
- hadoop 2.2.0编译WordCount
- Hadoop不用eclipse去编译运行WordCount
- Hadoop示例程序WordCount编译运行
- hadoop mapreduce 之wordcount 编译执行
- hadoop 2.2.0 编译运行wordcount
- 打包编译hadoop之HelloWord(WordCount)
- hadoop搭建与编译运行wordcount例
- HADOOP编译WordCount.java文件报错
- hadoop wordcount
- hadoop wordcount
- hadoop-wordcount
- Hadoop WordCount
- hadoop-wordcount
- hadoop wordcount
- 运行hadoop的WordCount程序——编译,打包,运行
- Hadoop MapReduce示例程序WordCount.java手动编译运行解析
- Hadoop重新编译WordCount程序报错:cannot access Options
- MD5实现小应用实例
- struts2 监听器
- 【图论基础练习】 SCU 2013 Summer Traing 3 - Graph Theory (1) 解题报告
- container_of 理解
- 解决SQLSERVER在还原数据时出现的“FILESTREAM功能被禁用”问题
- hadoop编译WORDCOUNT
- poj 1269 Intersecting Lines
- linux下解压命令大全
- 节约空间的筛素数方法
- iOS 获取一个类的所有方法
- The import android.* cannot be resolved
- S3C2440 电阻式触摸屏
- java.lang.NoClassDefFoundError: org/hibernate/cfg/HbmBinder$SecondPass解决办法 .
- STL中map,set的基本用法示例