【Storm初探】wordcount demo

目前工作下需要接触storm，遂写个demo练练手。
代码主要借鉴的是storm官方的github项目：https://github.com/apache/storm
略做修改。
1.pom.xml引入

   <dependency>        <groupId>org.apache.storm</groupId>        <artifactId>storm-core</artifactId>        <type>jar</type>        <version>1.0.0</version>   </dependency>

2.Spout
Spout是strom里产生元数据（tuple）流的类，通常情况下Spout会读取外部数据，然后转换为Topology内部的元数据（tuple），主动角色。
nextTuple方法不断执行，数据在此方法内产生。
此处用在数组中随机获得句子作为元数据流做示例。

public class RandomSentenceSpout extends BaseRichSpout {  private static final Logger LOG = LoggerFactory.getLogger(RandomSentenceSpout.class);  SpoutOutputCollector _collector;  Random _rand;  @Override  public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {    _collector = collector;    _rand = new Random();  }  @Override  public void nextTuple() {    Utils.sleep(100);    String[] sentences = new String[]{sentence("the cow jumped over the moon"), sentence("an apple a day keeps the doctor away"),            sentence("four score and seven years ago"), sentence("snow white and the seven dwarfs"), sentence("i am at two with nature")};    final String sentence = sentences[_rand.nextInt(sentences.length)];    LOG.info("Emitting tuple: {}", sentence);    _collector.emit(new Values(sentence));  }  protected String sentence(String input) {    return input;  }  @Override  public void ack(Object id) {  }  @Override  public void fail(Object id) {  }  @Override  public void declareOutputFields(OutputFieldsDeclarer declarer) {    declarer.declare(new Fields("word"));  }}

3.Bolt
在一个Topology中接收数据并处理的组件，被动角色。
execute函数处理接收到的数据，并生成新元组转发（emit）出去。
此处定义了两个bolt，第一个blot先把得到Sentence分隔成word，然后下一个bolt对word计数。

public class SplitSentenceBlot extends BaseBasicBolt {    @Override    public void execute(Tuple tuple, BasicOutputCollector collector) {        String word = tuple.getString(0);        String[] words = word.split(" ");        for (String s : words) {            System.out.println("==========" + s);            collector.emit(new Values(s));        }    }    @Override    public void declareOutputFields(OutputFieldsDeclarer declarer) {        declarer.declare(new Fields("word"));    }}

public class WordCountBlot extends BaseBasicBolt {    Map<String, Integer> counts = new HashMap<String, Integer>();    @Override    public void execute(Tuple tuple, BasicOutputCollector collector) {        String word = tuple.getString(0);        Integer count = counts.get(word);        if (count == null)            count = 0;        count++;        counts.put(word, count);        System.out.println("==============[" + word + "]:" + count);        collector.emit(new Values(word, count));    }    @Override    public void declareOutputFields(OutputFieldsDeclarer declarer) {        declarer.declare(new Fields("word", "count"));    }}

4.Topology
拓扑，storm里的关键性概念，本质上就是在storm里运行的一个实时应用程序，也就是此处要定义的主类。
具体看注释即可…

public class SimpleTopology {    public static void main(String[] args) throws Exception{        // 实例化TopologyBuilder类。        TopologyBuilder topologyBuilder = new TopologyBuilder();        // 设置喷发节点并分配并发数，该并发数将会控制该对象在集群中的线程数。        topologyBuilder.setSpout("spout", new RandomSentenceSpout(), 1);        // 设置数据处理节点并分配并发数。指定该节点接收喷发节点的策略为随机方式。        topologyBuilder.setBolt("split", new SplitSentenceBlot(), 3).shuffleGrouping("spout");        topologyBuilder.setBolt("count", new WordCountBlot(), 3).fieldsGrouping("split", new Fields("word"));        String topologyName = "word-count";        Config config = new Config();//        config.setDebug(true);        if (args != null && args.length > 0) {            config.setNumWorkers(1);            StormSubmitter.submitTopology(args[0], config, topologyBuilder.createTopology());        } else {            // 这里是本地模式下运行的启动代码。            config.setMaxTaskParallelism(1);            LocalCluster cluster = new LocalCluster();            cluster.submitTopology(topologyName, config, topologyBuilder.createTopology());        }    }}