【多线程系列二】-master-woker模式：统计单次频率

序:

Master-Worker模式是常用的并行设计模式。核心思想是，系统由两个角色组成，Master和Worker，Master负责接收和分配任务，Worker负责处理子任务。任务处理过程中，Master还负责监督任务进展和Worker的健康状态；Master将接收Client提交的任务，并将任务的进展汇总反馈给Client。各角色关系如下图 

但是书上给出的demo比较麻烦。

我按照一个更简洁的方式：模拟统计单词词频的方式。

具体细节如上图，Master对任务进行切分，并放入任务队列；然后，触发Worker处理任务。实际操作中，任务的分配有多种形式，如Master主动拉起Workder进程池或线程池，并将任务分配给Worker；或者由Worker主动领取任务，这样的Worker一般是常驻进程；还有一种解耦的方式，即Master指做任务的接收、切分和结果统计，指定Worker的数量和性能指标，但不参与Worker的实际管理，而是交由第三方调度监控和调度Worker。

demo:如下：

public class Master {// 任务队列static BlockingQueue<String> blockingQueue = new ArrayBlockingQueue<>(1024);// 子任务处理结果集static Map<String, Integer> resultMap = new ConcurrentHashMap<String, Integer>();public static void main(String[] args) {for (int i = 0; i < 50; i++) {try {String tmp ="AAA";if(i%2==0){tmp ="BBB";}blockingQueue.put("hello world" + i+" "+tmp);} catch (InterruptedException e) {// TODO Auto-generated catch blocke.printStackTrace();}}ExecutorService exector = Executors.newFixedThreadPool(20);ArrayList<Future<Map<String, Integer>>> futures = new ArrayList<Future<Map<String, Integer>>>();for (int j = 0; j < 20; j++) {Future<Map<String, Integer>> futurej = exector.submit(new Task("" + j, blockingQueue));futures.add(futurej);}System.out.println("获取结果中...");for (Future<Map<String, Integer>> f : futures) {try {// if(f.isDone())Map<String, Integer> reMap = f.get();for (Entry<String, Integer> entry : reMap.entrySet()) {resultMap.put(entry.getKey(), resultMap.getOrDefault(entry.getKey(), 0) + entry.getValue());}} catch (Exception e) {e.printStackTrace();}}System.out.println("得到结果.");for (Entry<String, Integer> entry : resultMap.entrySet()) {System.out.print(entry.getKey() + ":" + entry.getValue()+" ");}// 关闭线程池。exector.shutdown();}}class Task implements Callable<Map<String, Integer>> {private String name;private BlockingQueue<String> blockingQueue;public Task(String name, BlockingQueue<String> blockingQueue) {this.name = name;this.blockingQueue = blockingQueue;}@Overridepublic Map<String, Integer> call() {System.out.println(System.currentTimeMillis() + " " + name + "run");Map<String, Integer> wordCount = new HashMap<>();while (true) {try {String input = blockingQueue.poll(1, TimeUnit.SECONDS);if (null != input && StringUtils.isNotBlank(String.valueOf(input))) {String[] words = StringUtils.split(String.valueOf(input), " "); // 分词for (String w : words) {if (StringUtils.isBlank(w)) {continue;}wordCount.put(w, wordCount.getOrDefault(w, 0) + 1);}}else{break;}} catch (InterruptedException e) {break;}}System.out.println(LocalTime.now() + " " + name + "runover");return wordCount;}}

因为使用了future跟线程池，大幅简化了代码。可以对比下相关的代码：网上好多就是照搬了葛一鸣原书上代码没改。
执行结果如下：

获取结果中...
20:24:19.991 worker6run
20:24:19.992 worker19run
20:24:19.993 worker3run
20:24:19.995 worker2run
20:24:19.996 worker18run
20:24:20 worker10run
20:24:20 worker5run
20:24:20.001 worker14run
20:24:20.001 worker11run
20:24:20.002 worker4run
20:24:20.002 worker1run
20:24:20.002 worker13run
20:24:20.002 worker15run
20:24:20.003 worker7run
20:24:20.002 worker8run
20:24:20.003 worker12run
20:24:20.003 worker0run
20:24:20.003 worker9run
20:24:20.003 worker17run
20:24:20.003 worker16run
20:24:21.010 worker3runover
20:24:21.011 worker18runover
20:24:21.011 worker2runover
20:24:21.011 worker6runover
20:24:21.011 worker14runover
20:24:21.012 worker8runover
20:24:21.012 worker16runover
20:24:21.012 worker12runover
20:24:21.012 worker7runover
20:24:21.012 worker11runover
20:24:21.012 worker4runover
20:24:21.011 worker0runover
20:24:21.011 worker15runover
20:24:21.011 worker19runover
20:24:21.013 worker10runover
20:24:21.013 worker5runover
20:24:21.014 worker1runover
20:24:21.014 worker17runover
20:24:21.017 worker13runover
20:24:21.017 worker9runover
得到结果.
world4:1 
world5:1 
world2:1 
BBB:25 
world3:1 
world8:1 
world9:1 
world6:1 
world30:1 
world7:1 
world39:1 
world31:1 
world32:1 
world33:1 
world34:1 
world35:1 
world36:1 
world37:1 
world38:1 
AAA:25 
world28:1 
world29:1 
world20:1 
world21:1 
world22:1 
hello:50 
world23:1 
world24:1 
world0:1 
world25:1 
world1:1 
world26:1 
world27:1 
world17:1 
world18:1 
world19:1 
world10:1 
world11:1 
world12:1 
world13:1 
world14:1 
world15:1 
world16:1 
world40:1 
world41:1 
world42:1 
world43:1 
world44:1 
world45:1 
world46:1 
world47:1 
world48:1 
world49:1 

当然，这里只是介绍这种思路，典型的还是要考虑master异常怎么办，任务线程的处理结果如何处理，存表还是发消息。还有线程安全问题。多个节点如何做分布式的。

参考：

http://blog.csdn.net/wendingzhulu/article/details/52708472