从word count分析Map Reduce / Map Reduce的word count讲解
来源:互联网 发布:泰囧是网络电影吗 编辑:程序博客网 时间:2024/06/13 01:06
public void map(Object key, Text value, Context context ) throws IOException,InterruptedException {//每个mapper对于每个key-value,一次调用map读进一个key-value StringTokenizer itr = new StringTokenizer(value.toString()); while (itr.hasMoreTokens()) { word.set(itr.nextToken()); context.write(word, one);//key是word,key也是reducer读的key;写到‘中间层’ } }//中间过程,相同hash值的word/key分别汇集到一起,才有key-list的结构,供reducer读 public void reduce(Text key, Iterable<IntWritable> values, Context context ) throws IOException, InterruptedException {//每个reducer对于每个key,每次调用reduce读进一个key-values;上一步相同key的value成为一个list即values int sum = 0; for (IntWritable val : values) { sum += val.get();//即+1 } result.set(sum); context.write(key, result);//写到指定HDFS(一个)文件 }
https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordCount.java
0 0
- 从word count分析Map Reduce / Map Reduce的word count讲解
- Word Count (Map Reduce)
- Word Count (Map Reduce)
- Swift 函数Count,Filter,Map,Reduce
- 【c++系列】map容器实现的word count
- map reduce原理分析
- 通俗易懂的大白话讲解Map/Reduce原理
- Map-Reduce的过程
- mongoDB的map reduce
- map&reduce的用法
- javascript 的 map/reduce
- map/reduce
- map reduce
- Map/Reduce
- map reduce
- Map Reduce
- map reduce
- map-reduce
- 安卓学习路线
- Yii 良好的架构(schema)设计指南
- SMTP-发送邮件
- Java中的Comparable和Comparator的比较
- LightOJ 1175 (BFS)
- 从word count分析Map Reduce / Map Reduce的word count讲解
- 属性访问修饰符小实验
- ubuntu 无线网卡没有驱动
- Android 获取验证码倒计时封装
- Xcode LaunchImage 图片尺寸
- AppCan移动开发:仿口袋购物APP源码
- uva 10341 Solve it
- Android组件
- Redis学习记录之Server(二十二)