第四章、flume源码解析之核心类（source）

flume的核心类分别为Source、Channel、Sink，下面结合源码进行分析
1、Source
source的源码：
`**
*
* A source generates {@plainlink Event events} and calls methods on the
* configured {@link ChannelProcessor} to persist those events into the
* configured {@linkplain Channel channels}.
* 译：A source（源） 生产一个events（事件），和调用方法ChannelProcessor（通道处理器），将这些
* events （事件）持久化到配置的Channel （通道）中
*
*
* Sources are associated with unique {@linkplain NamedComponent names} that can
* be used for separating configuration and working namespaces.
*
*
* No guarantees are given regarding thread safe access.
*译：不保证线程安全
*
* @see org.apache.flume.Channel
* @see org.apache.flume.Sink
*/
@InterfaceAudience.Public
@InterfaceStability.Stable
public interface Source extends LifecycleAware, NamedComponent {

/**
* Specifies which channel processor will handle this source’s events.
*
* @param channelProcessor
*/
public void setChannelProcessor(ChannelProcessor channelProcessor);

/**
* Returns the channel processor that will handle this source’s events.
*/
public ChannelProcessor getChannelProcessor();

}`
接口Source有很多实现类，如下图

下面就NetcatSource 进行分析
NetcatSource.java中的start()方法

  @Override  public void start() {    logger.info("Source starting");    counterGroup.incrementAndGet("open.attempts");    handlerService = Executors.newCachedThreadPool(new ThreadFactoryBuilder()        .setNameFormat("netcat-handler-%d").build());    try {      SocketAddress bindPoint = new InetSocketAddress(hostName, port);      serverSocket = ServerSocketChannel.open();      serverSocket.socket().setReuseAddress(true);      serverSocket.socket().bind(bindPoint);      logger.info("Created serverSocket:{}", serverSocket);    } catch (IOException e) {      counterGroup.incrementAndGet("open.errors");      logger.error("Unable to bind to socket. Exception follows.", e);      throw new FlumeException(e);    }    AcceptHandler acceptRunnable = new AcceptHandler(maxLineLength);    acceptThreadShouldStop.set(false);    acceptRunnable.counterGroup = counterGroup;    acceptRunnable.handlerService = handlerService;    acceptRunnable.shouldStop = acceptThreadShouldStop;    acceptRunnable.ackEveryEvent = ackEveryEvent;    acceptRunnable.source = this;    acceptRunnable.serverSocket = serverSocket;    acceptRunnable.sourceEncoding = sourceEncoding;    acceptThread = new Thread(acceptRunnable);    acceptThread.start();    logger.debug("Source started");    super.start();  }

和客户端套接字连接，并启动一个线程，一旦线程启动就进入AcceptHandler中run方法

   @Override    public void run() {      logger.debug("Starting accept handler");      while (!shouldStop.get()) {        try {          SocketChannel socketChannel = serverSocket.accept();          NetcatSocketHandler request = new NetcatSocketHandler(maxLineLength);          request.socketChannel = socketChannel;          request.counterGroup = counterGroup;          request.source = source;          request.ackEveryEvent = ackEveryEvent;          request.sourceEncoding = sourceEncoding;          handlerService.submit(request);          counterGroup.incrementAndGet("accept.succeeded");        } catch (ClosedByInterruptException e) {          // Parent is canceling us.        } catch (IOException e) {          logger.error("Unable to accept connection. Exception follows.", e);          counterGroup.incrementAndGet("accept.failed");        }      }      logger.debug("Accept handler exiting");    }  }

一旦请求被submit，就进入NetcatSocketHandler 的run方法中

    @Override    public void run() {      logger.debug("Starting connection handler");      Event event = null;      try {        Reader reader = Channels.newReader(socketChannel, sourceEncoding);        Writer writer = Channels.newWriter(socketChannel, sourceEncoding);        //字节缓冲区        CharBuffer buffer = CharBuffer.allocate(maxLineLength);        buffer.flip(); // flip() so fill() sees buffer as initially empty        while (true) {          // this method blocks until new data is available in the socket          int charsRead = fill(buffer, reader);          logger.debug("Chars read = {}", charsRead);          // attempt to process all the events in the buffer          //处理事件          int eventsProcessed = processEvents(buffer, writer);          logger.debug("Events processed = {}", eventsProcessed);          if (charsRead == -1) {            // if we received EOF before last event processing attempt, then we            // have done everything we can            break;          } else if (charsRead == 0 && eventsProcessed == 0) {            if (buffer.remaining() == buffer.capacity()) {              // If we get here it means:              // 1. Last time we called fill(), no new chars were buffered              // 2. After that, we failed to process any events => no newlines              // 3. The unread data in the buffer == the size of the buffer              // Therefore, we are stuck because the client sent a line longer              // than the size of the buffer. Response: Drop the connection.              logger.warn("Client sent event exceeding the maximum length");              counterGroup.incrementAndGet("events.failed");              writer.write("FAILED: Event exceeds the maximum length (" +                  buffer.capacity() + " chars, including newline)\n");              writer.flush();              break;            }          }        }        socketChannel.close();        counterGroup.incrementAndGet("sessions.completed");      } catch (IOException e) {        counterGroup.incrementAndGet("sessions.broken");        try {          socketChannel.close();        } catch (IOException ex) {          logger.error("Unable to close socket channel. Exception follows.", ex);        }      }      logger.debug("Connection handler exiting");    }

run方法中有一个处理事件的方法processEvents（）

  private int processEvents(CharBuffer buffer, Writer writer)        throws IOException {      int numProcessed = 0;      boolean foundNewLine = true;      while (foundNewLine) {        foundNewLine = false;        int limit = buffer.limit();        //读取字节流        for (int pos = buffer.position(); pos < limit; pos++) {          if (buffer.get(pos) == '\n') {            // parse event body bytes out of CharBuffer            buffer.limit(pos); // temporary limit            ByteBuffer bytes = Charsets.UTF_8.encode(buffer);            buffer.limit(limit); // restore limit            // build event object            //将字节转化成事件            byte[] body = new byte[bytes.remaining()];            bytes.get(body);            Event event = EventBuilder.withBody(body);            // process event            ChannelException ex = null;            try {            //通道处理器处理事件              source.getChannelProcessor().processEvent(event);              ex = chEx;            }            if (ex == null) {              counterGroup.incrementAndGet("events.processed");              numProcessed++;              if (true == ackEveryEvent) {                writer.write("OK\n");              }            } else {              counterGroup.incrementAndGet("events.failed");              logger.warn("Error processing event. Exception follows.", ex);              writer.write("FAILED: " + ex.getMessage() + "\n");            }            writer.flush();            // advance position after data is consumed            buffer.position(pos + 1); // skip newline            foundNewLine = true;            break;          }        }      }      return numProcessed;    }

processEvents中有一个通道处理器处理事件的方法processEvent（）

public void processEvent(Event event) {//拦截器链    event = interceptorChain.intercept(event);    if (event == null) {      return;    }    // Process required channels    //选择器得到所有必须的通道    List<Channel> requiredChannels = selector.getRequiredChannels(event);    //遍历通道    for (Channel reqChannel : requiredChannels) {    //得到事务      Transaction tx = reqChannel.getTransaction();      Preconditions.checkNotNull(tx, "Transaction object must not be null");      try {        tx.begin();    //把事件放到每个通道里        reqChannel.put(event);        tx.commit();      } catch (Throwable t) {        tx.rollback();        if (t instanceof Error) {          LOG.error("Error while writing to required channel: " + reqChannel, t);          throw (Error) t;        } else if (t instanceof ChannelException) {          throw (ChannelException) t;        } else {          throw new ChannelException("Unable to put event on required " +              "channel: " + reqChannel, t);        }      } finally {        if (tx != null) {          tx.close();        }      }    }    // Process optional channels    //通过选择器得到可选通道    List<Channel> optionalChannels = selector.getOptionalChannels(event);    for (Channel optChannel : optionalChannels) {      Transaction tx = null;      try {        tx = optChannel.getTransaction();        tx.begin();        optChannel.put(event);        tx.commit();      } catch (Throwable t) {        tx.rollback();        LOG.error("Unable to put event on optional channel: " + optChannel, t);        if (t instanceof Error) {          throw (Error) t;        }      } finally {        if (tx != null) {          tx.close();        }      }    }  }}

source流程：Source–>Event–>processEvents–>InterceptorChain–>Channel s