window下eclipse配制hadoop插件
来源:互联网 发布:西游记 女儿情 知乎 编辑:程序博客网 时间:2024/05/22 12:08
1.把插件hadoop-eclipse-plugin-2.6.2.jar拷贝到eclipse安装目录下的plugins中
2.重启一下Eclipse
配制hadoop
3.进入map/reduce视图模式
4.向hadoop分布式存储系统中存入数据
5.连接hadoop
6.创建hadoop工程
7.创建类MyWordCount.java
package com.yc.hadoop;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class MyWordCount {
public static class MyWordCountMapper extends Mapper<Object, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(Object key, Text value, Context context) throws IOException, InterruptedException {
String[] words = value.toString().split("\\s");
for (String w : words) {
word.set(w);
context.write(word, one);
}
}
}
public static class MyWordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
private IntWritable result = new IntWritable();
public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
result.set(sum);
context.write(key, result);
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "mywordcount");
job.setJarByClass(MyWordCount.class);
job.setMapperClass(MyWordCountMapper.class);
job.setCombinerClass(MyWordCountReducer.class);
job.setReducerClass(MyWordCountReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
8.运行hadoop项目
转自:http://user.qzone.qq.com/413670706?ptlang=2052
- window下eclipse配制hadoop插件
- window下eclipse配制hadoop插件 (四)
- eclipse在window环境下配置hadoop插件
- eclipse 插件&配制 内容收集
- Eclipse下配置Hadoop插件
- Eclipse下配置hadoop插件
- eclipse下安装Hadoop插件
- [大数据-hadoop基础]window下编译hadoop2.7.4-eclipse mar2插件
- window下编译hadoop-clipse-plugin插件
- eclipse下resin的配制
- eclipse下resin的配制
- eclipse下resin的配制
- eclipse下resin的配制
- window下eclipse安装python插件
- svn安装配制以及eclipse插件安装
- 编译Hadoop2.x Hadoop-eclipse-plugin插件(window)
- Eclipse下配置使用Hadoop插件
- Eclipse下配置使用Hadoop插件
- 来自一个前高三狗的分享(个人意见,不喜勿喷)
- 一个C++类的最小程序(用到.h文件)
- Myeclipse中js总是报错 解决方法
- hdu 5919 Sequence II(主席树,求区间不同数的个数)
- 人体头像面部的二维主成分分析(2D PCA)
- window下eclipse配制hadoop插件
- 状态压缩dp入门--------TSP问题
- GPUImage--美颜滤镜GPUImageBeautifyFilter
- [Android] 内存优化
- Java接口 详解(二)
- java 日期计算
- 配置本地yum源安装软件
- Java垃圾回收小结
- Spring之Bean在BeanFactory的生命周期