STORM入门之(TridentAPI,Each)
来源:互联网 发布:linux线程池 编辑:程序博客网 时间:2024/06/07 07:28
1.基础
基础Topology与TritentTopology是不同的,就相当于JDBC VS Hibernate ,Hibernate是基于JDBC实现的ORM架构,二者本质是相同的,但是用法截然不同,Trident会抽象一些,不过底层也是基于Topology的Spout,Bolt等基础来构建,并且最终提交任务时,TritentTopology是会转换成Topology。Each相当于把Spout分割成段,进行batch处理。Each方法包含过滤与函数方式进行流式处理,下面代码是针对filter与function进行编写
2.代码
下面是简单构建TritentTopology的代码
public static void main(String[] args){ TridentTopology topology = new TridentTopology(); FixedBatchSpout spout = new FixedBatchSpout(new Fields("sentence"), 1, new Values("the cow jumped over the moon"), new Values("the man went to the store and bought some candy"), new Values("four score and seven years ago"), new Values("how many apples can you eat")); spout.setCycle(true); topology.newStream("batch-spout",spout) .each(new Fields("sentence"), new Split(), new Fields("word")) .each(new Fields("sentence","word"), new WordFilter("the")); StormTopology stormTopology = topology.build(); LocalCluster cluster = new LocalCluster(); Config conf = new Config(); conf.setDebug(true); cluster.submitTopology("soc", conf,stormTopology); }
Function分割函数Split
package com.storm.trident;import org.apache.storm.trident.operation.BaseFunction;import org.apache.storm.trident.operation.TridentCollector;import org.apache.storm.trident.tuple.TridentTuple;import org.apache.storm.tuple.Values;public class Split extends BaseFunction { public void execute(TridentTuple tuple, TridentCollector collector) { String sentence = tuple.getString(0); for(String word: sentence.split(" ")) { System.out.println("emit word is :"+word); collector.emit(new Values(word)); } }}
Filter过滤
package com.storm.trident;import org.apache.storm.trident.operation.BaseFilter;import org.apache.storm.trident.tuple.TridentTuple;/** * Created with IntelliJ IDEA. * User: Administrator * Date: 17-9-1 * Time: 上午9:03 * To change this template use File | Settings | File Templates. */public class WordFilter extends BaseFilter { String actor; public WordFilter(String actor) { this.actor = actor; } @Override public boolean isKeep(TridentTuple tuple) { if(tuple.getString(1).equals(actor)){ System.out.println("Filter word is:"+tuple.getString(1) + " and return type is:"+tuple.getString(1).equals(actor)); } return tuple.getString(1).equals(actor); }}
运行结果
阅读全文
0 0
- STORM入门之(TridentAPI,Each)
- STORM入门之(TridentAPI,Aggregation)
- STORM入门之(TridentAPI,partition)
- STORM入门之(基本Shell命令)
- STORM入门之(Topology简易Demo)
- STORM入门之(集成KafkaBolt)
- STORM入门之(集成KafkaSpout)
- STORM入门之(集成ElasticSearch)
- STORM入门之(集成Redis)
- STORM入门之(TridentTopology集成Kafka)
- STORM入门之(yaml配置文件说明)
- STORM入门之(集成HDFS)
- Storm入门之第一章
- Storm入门之第一章
- Storm入门之第一章
- Storm入门之第一章
- Storm入门之附录A
- Storm入门之附录B
- Android中 Lottie库初步实践与应用场景分析
- bootstrap一点心得
- LINUX常用系统命令
- 上传绕过-实验吧
- 利用树莓派3 实现简单的人机交互程序
- STORM入门之(TridentAPI,Each)
- S/Jquery 中怎么定义存储一个类似map中key-value形式的变量,并且怎样动态赋值key和value?
- NS3 Tracing System
- 神奇的单项选择题和多项选择题答案js选项生成算法
- css 实现单行居中,多行居左省略
- c中scanf函数读取换行符的问题
- APK减少包体大小
- 让客户爱上你的24条建议
- springMVC ,put,delete方法失效