MapRecuce 结果输出

Combine过程和Reduce过程都可以省略，在缺少Combine过程与Reduce过程时，输出结果为Mapper过程的结果；只缺少Reduce过程时，输出Combine过程的结果。
combine过程与reduce过程处理代码的方式基本相同，一个在单机节点上执行，一个在传输后执行，一般可以共用一个类。

只设置Map类

import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Job;import org.apache.hadoop.mapreduce.Mapper;import org.apache.hadoop.mapreduce.Reducer;import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;import java.io.IOException;/** * Created by fly on 15-7-13. */public class MapReduceExperiment {    public static class MapperClass extends Mapper<Object, Text, Text, IntWritable> {        public void map(Object key, Text value, Context context) throws IOException, InterruptedException {            StringTokenizer itr = new StringTokenizer(value.toString());        //将每一行分解为多个单词            Text word = new Text();            while (itr.hasMoreTokens()) {                word.set(itr.nextToken());                context.write(word, new IntWritable(1));            }        }    }    public static void main(String[] args) throws Exception {        Configuration conf = new Configuration();        @SuppressWarnings("deprecation")        Job job = new Job(conf, "word count");        job.setJarByClass(MapReduceExperiment.class);        job.setMapperClass(MapperClass.class);  //      job.setCombinerClass(ReducerClass.class); //combine过程与reduce过程的执行函数相同    //    job.setReducerClass(ReducerClass.class);        job.setOutputKeyClass(Text.class);        job.setOutputValueClass(IntWritable.class);        FileInputFormat.addInputPath(job, new Path("hdfs://localhost:9000/testDataProcess/"));        FileOutputFormat.setOutputPath(job, new Path("hdfs://localhost:9000/output9"));        System.out.println("over");        System.exit(job.waitForCompletion(true) ? 0 : 1);    }}

当只设置Mapper过程时，从输入文件中读入内容，并将context.write()的内容直接写入结果中。也可以不使用context.write(),得到一个空的文件。

Mapper + Combiner

在类中添加以下combiner代码

    public static class CombineClass extends Reducer<Text,IntWritable,Text,IntWritable>{        private IntWritable result = new IntWritable();        public void reduce(Text key, Iterable<IntWritable> values, Context context)                throws IOException, InterruptedException {//          System.out.println(key);            int sum = 0;            for (IntWritable val : values) {                sum += val.get();            }            result.set(sum);            context.write(key, result);     //结果存入最后的output文件中        }    }

在main函数job设置中添加
job.setCombinerClass(CombineClass.class);

得到文件中为进行一次combine的结果，文件中的内容取决于CombineClass类中的context.write() 函数