Hadoop Yarn事件处理框架源码分析

由于想在项目中使用类似yarn的事件处理机制，就看了实现。主要是由Dispatcher.java，EventHandler.java，Service.java这3个类撑起来的。

 在事件处理之前，先注册相应的事件处理handler，收到事件event后，由派发事件的Dispatcher进行派发，默认采用异步事件处理方式将事件放到事件队列(LinkedBlockingQueue)中，消费者会循环从队列中取出事件进行处理。

要使用事件处理，首先需要创建Dispatcher，示例代码如下：

 dispatcher = new AsyncDispatcher();//创建  addIfService(dispatcher);// 由于继承AbstractService，可以方便完成服务统一管理，比如初始化和资源释放等操作  dispatcher.register(EventType.class,new EventHandler());//注册对应的事件处理方法

然后通过AsyncDispatcher调用getEventHandler()返回的EventHandler的处理对应事件，AsyncDispatcher类的getEventHandler()方法如下：

@Override  public EventHandler getEventHandler() {    if (handlerInstance == null) {      handlerInstance = new GenericEventHandler();//如果没有注册生产事件处理，就走通用事件处理    }    return handlerInstance;  }

class GenericEventHandler implements EventHandler<Event> {    public void handle(Event event) {      if (blockNewEvents) {        return;      }      /* all this method does is enqueue all the events onto the queue */      int qSize = eventQueue.size();      if (qSize !=0 && qSize %1000 == 0) {        LOG.info("Size of event-queue is " + qSize);      }      int remCapacity = eventQueue.remainingCapacity();      if (remCapacity < 1000) {        LOG.warn("Very low remaining capacity in the event-queue: "            + remCapacity);      }      try {        eventQueue.put(event);//放进队列        drained = false;      } catch (InterruptedException e) {        if (!stopped) {          LOG.warn("AsyncDispatcher thread interrupted", e);        }        throw new RuntimeException(e);      }    };  }

上述完成生产，再看消费如下实现的：

@Overrideprotected void serviceStart() throws Exception {  //start all the components  super.serviceStart();  eventHandlingThread = new Thread(createThread()); // 调用创建消费eventQueue队列中事件的线程  eventHandlingThread.setName("AsyncDispatcher event handler");  eventHandlingThread.start();}

查看createThread()方法，如下所示：

Runnable createThread() {    return new Runnable() {      @Override      public void run() {        while (!stopped && !Thread.currentThread().isInterrupted()) {          drained = eventQueue.isEmpty();          // blockNewEvents is only set when dispatcher is draining to stop,          // adding this check is to avoid the overhead of acquiring the lock          // and calling notify every time in the normal run of the loop.          if (blockNewEvents) {            synchronized (waitForDrained) {              if (drained) {                waitForDrained.notify();              }            }          }          Event event;          try {            event = eventQueue.take();          } catch(InterruptedException ie) {            if (!stopped) {              LOG.warn("AsyncDispatcher thread interrupted", ie);            }            return;          }          if (event != null) {            dispatch(event);//分发事件          }        }      }    };  }

从eventQueue队列中取出Event，然后调用dispatch(event);来处理事件，看dispatch(event)方法，如下所示：

protected void dispatch(Event event) {  //all events go thru this loop  if (LOG.isDebugEnabled()) {    LOG.debug("Dispatching the event " + event.getClass().getName() + "."        + event.toString());  }  Class<? extends Enum> type = event.getType().getDeclaringClass();  try{    EventHandler handler = eventDispatchers.get(type); //通过event获取事件类型，根据事件类型得到注册的EventHandler    if(handler != null) {      handler.handle(event); //EventHandler处理事件event    } else {      throw new Exception("No handler for registered for " + type);    }  } catch (Throwable t) {    //TODO Maybe log the state of the queue    LOG.fatal("Error in dispatcher thread", t);    // If serviceStop is called, we should exit this thread gracefully.    if (exitOnDispatchException        && (ShutdownHookManager.get().isShutdownInProgress()) == false        && stopped == false) {      LOG.info("Exiting, bbye..");      System.exit(-1);    }  }}

整个过程使用生产--消费者模型，异步事件处理，整体实现起来还是很简单的！