netty（二）:事件执行分析

上一篇中，重点是看下自己对各个组件的认识，数据流向的大致把握。

我记得，我经常问自己，事件到底是怎么被执行的呢？我怎么看事件被执行，被哪个线程执行的呢？

还有一个问题：vertx中，译者注如下：

每个 Verticle 在部署的时候都会被分配一个 Context（根据配置不同，可以是Event Loop Context 或者 Worker Context），之后此 Verticle 上所有的普通代码都会在此 Context 上执行（即对应的 Event Loop 或Worker 线程）。一个 Context 对应一个 Event Loop 线程（或 Worker 线程），但一个 Event Loop 可能对应多个 Context。

一直觉得一个Context对应一个EventLoop，自然一个EventLoop就对应一个Context。既然理解有偏差，肯定是netty的概念还存在混淆，于是。

1.EventLoopGroup与EventLoop的关系。

    先看一段服务端启动的代码：

// 配置服务端的NIO线程组EventLoopGroup bossGroup = new NioEventLoopGroup();EventLoopGroup workerGroup = new NioEventLoopGroup();try {    ServerBootstrap b = new ServerBootstrap();    b.group(bossGroup, workerGroup)            .channel(NioServerSocketChannel.class)            .option(ChannelOption.SO_BACKLOG, 1024)            .childHandler(new ChildChannelHandler());    // 绑定端口，同步等待成功    ChannelFuture f = b.bind(port).sync();    // 等待服务器坚挺端口关闭    f.channel().closeFuture().sync();} finally {    // 优雅退出，释放线程池资源    bossGroup.shutdownGracefully();    workerGroup.shutdownGracefully();}

    这段代码我们之前看过，

public NioEventLoopGroup() {    this(0);}

/** * Create a new instance. * * @param nThreads          the number of threads that will be used by this instance. * @param executor          the Executor to use, or {@code null} if the default should be used. * @param chooserFactory    the {@link EventExecutorChooserFactory} to use. * @param args              arguments which will passed to each {@link #newChild(Executor, Object...)} call */protected MultithreadEventExecutorGroup(int nThreads, Executor executor,                                        EventExecutorChooserFactory chooserFactory, Object... args) {    if (nThreads <= 0) {        throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));    }    if (executor == null) {        executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());    }    children = new EventExecutor[nThreads];    for (int i = 0; i < nThreads; i ++) {        boolean success = false;        try {            children[i] = newChild(executor, args);            success = true;        } catch (Exception e) {            // TODO: Think about if this is a good exception type            throw new IllegalStateException("failed to create a child event loop", e);        } finally {            if (!success) {                for (int j = 0; j < i; j ++) {                    children[j].shutdownGracefully();                }                for (int j = 0; j < i; j ++) {                    EventExecutor e = children[j];                    try {                        while (!e.isTerminated()) {                            e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);                        }                    } catch (InterruptedException interrupted) {                        // Let the caller handle the interruption.                        Thread.currentThread().interrupt();                        break;                    }                }            }        }    }    chooser = chooserFactory.newChooser(children);    final FutureListener<Object> terminationListener = new FutureListener<Object>() {        @Override        public void operationComplete(Future<Object> future) throws Exception {            if (terminatedChildren.incrementAndGet() == children.length) {                terminationFuture.setSuccess(null);            }        }    };    for (EventExecutor e: children) {        e.terminationFuture().addListener(terminationListener);    }    Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);    Collections.addAll(childrenSet, children);    readonlyChildren = Collections.unmodifiableSet(childrenSet);}

    对照上面的代码，我们知道NioEventLoopGroup在初始化的时候，executor默认用的是ThreadPerTaskExecutor

    这个任务执行器，初始化的时候传入了一个默认的线程工厂。这个先不说。

    NioEventLoopGroup继承自MultiThreadEventEexcutorGroup

private final EventExecutor[] children;

    MultiXXXGroup这个类中有EventExecutor数组，如果NioEventLoopGroup上管理EventLoop的，那么，这个数组就应该对应到EventLoop

children[i] = newChild(executor, args);

/** * Create a new EventExecutor which will later then accessible via the {@link #next()}  method. This method will be * called for each thread that will serve this {@link MultithreadEventExecutorGroup}. * */protected abstract EventExecutor newChild(Executor executor, Object... args) throws Exception;

    在该类中，初始化EventExecutor的newChild方法，并不是自己实现的，而是子类自己去实现，那好，我们跳回到NioEventLoopGroup中

@Overrideprotected EventLoop newChild(Executor executor, Object... args) throws Exception {    return new NioEventLoop(this, executor, (SelectorProvider) args[0],        ((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]);}

    确实，根据传入的执行器，生成新的NioEventLoop对象。

2.EventLoop与Thread的关系。

    上一篇我们分析过NioEventLoop对象，它继承自SingleThreadEventLoop，很明显，netty一直说一个NioEventLoop对应着一条线程，至此我们都没有看到线程Thread,

    在其父类SingleThreadEventExecutor中我们才看到Thread的影子：

private final Queue<Runnable> taskQueue;private volatile Thread thread;@SuppressWarnings("unused")private volatile ThreadProperties threadProperties;private final Executor executor;private volatile boolean interrupted;private final Semaphore threadLock = new Semaphore(0);private final Set<Runnable> shutdownHooks = new LinkedHashSet<Runnable>();private final boolean addTaskWakesUp;private final int maxPendingTasks;private final RejectedExecutionHandler rejectedExecutionHandler;

    这里截取了该类的部分成员变量。其中我关心的就是Executor, Thread, taskQueue.

    那么，Thread是在哪里被初始化的呢?

private void doStartThread() {    assert thread == null;    executor.execute(new Runnable() {        @Override        public void run() {            thread = Thread.currentThread();            if (interrupted) {                thread.interrupt();            }            boolean success = false;            updateLastExecutionTime();            try {                SingleThreadEventExecutor.this.run();                success = true;            } catch (Throwable t) {                logger.warn("Unexpected exception from an event executor: ", t);            } finally {                for (;;) {                    int oldState = state;                    if (oldState >= ST_SHUTTING_DOWN || STATE_UPDATER.compareAndSet(                            SingleThreadEventExecutor.this, oldState, ST_SHUTTING_DOWN)) {                        break;                    }                }

    在SingleThreadEventExecutor中跟踪thread变量，发现在doStartThread()方法中thread才被赋值，并且为当前线程。很恐怖。

    我们前面也多次提到executor变量，这个是由默认值提供生成的，在这里调用了execute()方法.那我们先去看executor到底是什么：

    

executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());

public final class ThreadPerTaskExecutor implements Executor {    private final ThreadFactory threadFactory;    public ThreadPerTaskExecutor(ThreadFactory threadFactory) {        if (threadFactory == null) {            throw new NullPointerException("threadFactory");        }        this.threadFactory = threadFactory;    }    @Override    public void execute(Runnable command) {        threadFactory.newThread(command).start();    }}

    哦，也就是在执行execute方法的时候，用现成工厂去生成一个新线程，并且启动线程，那就是直接运行了run方法，怪不得直接把thread赋值为当前线程。

    那么这个线程工厂是生成了一个怎么样的线程呢？DefaultThreadFactory没有什么奇怪的，只是设置了线程的优先级啊，名称啊什么的基本信息，真正生成方法是下面这个：

@Overridepublic Thread newThread(Runnable r) {    Thread t = newThread(new DefaultRunnableDecorator(r), prefix + nextId.incrementAndGet());    try {        if (t.isDaemon() != daemon) {            t.setDaemon(daemon);        }        if (t.getPriority() != priority) {            t.setPriority(priority);        }    } catch (Exception ignored) {        // Doesn't matter even if failed to set.    }    return t;}

protected Thread newThread(Runnable r, String name) {    return new FastThreadLocalThread(threadGroup, r, name);}

public FastThreadLocalThread(ThreadGroup group, Runnable target, String name) {    super(group, target, name);}

/** * A special {@link Thread} that provides fast access to {@link FastThreadLocal} variables. */public class FastThreadLocalThread extends Thread {
 

    看了这个调用链，发现生成的线程是在原Thread类上做了封装，看注释，是为了性能优化呢（还观察到将Runnable参数直接赋值给Thread的target变量了）。

    这样看来，一个NioEventLoop包含了
    Executor-> ThreadPerTaskExecutor执行器

    Thread   -> FastThreadLocalThread线程

3.事件与Thread的关系。

    假设SingleThreadEventExecutor的doStartThread()方法被调用了，这里生成新线程，并开始EventLoop的工作，但是工作在哪儿？我们看到

SingleThreadEventExecutor.this.run();

    在线程赋值后被执行了，进去

protected abstract void run();

    这里没有实现，那在上层找找

@Overrideprotected void run() {    for (;;) {        try {            switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {                case SelectStrategy.CONTINUE:                    continue;                case SelectStrategy.SELECT:                    select(wakenUp.getAndSet(false));                    // 'wakenUp.compareAndSet(false, true)' is always evaluated                    // before calling 'selector.wakeup()' to reduce the wake-up                    // overhead. (Selector.wakeup() is an expensive operation.)                    //                    // However, there is a race condition in this approach.                    // The race condition is triggered when 'wakenUp' is set to                    // true too early.                    //                    // 'wakenUp' is set to true too early if:                    // 1) Selector is waken up between 'wakenUp.set(false)' and                    //    'selector.select(...)'. (BAD)                    // 2) Selector is waken up between 'selector.select(...)' and                    //    'if (wakenUp.get()) { ... }'. (OK)                    //                    // In the first case, 'wakenUp' is set to true and the                    // following 'selector.select(...)' will wake up immediately.                    // Until 'wakenUp' is set to false again in the next round,                    // 'wakenUp.compareAndSet(false, true)' will fail, and therefore                    // any attempt to wake up the Selector will fail, too, causing                    // the following 'selector.select(...)' call to block                    // unnecessarily.                    //                    // To fix this problem, we wake up the selector again if wakenUp                    // is true immediately after selector.select(...).                    // It is inefficient in that it wakes up the selector for both                    // the first case (BAD - wake-up required) and the second case                    // (OK - no wake-up required).                    if (wakenUp.get()) {                        selector.wakeup();                    }                default:                    // fallthrough            }            cancelledKeys = 0;            needsToSelectAgain = false;            final int ioRatio = this.ioRatio;            if (ioRatio == 100) {                try {                    processSelectedKeys();                } finally {                    // Ensure we always run tasks.                    runAllTasks();                }            } else {                final long ioStartTime = System.nanoTime();                try {                    processSelectedKeys();                } finally {                    // Ensure we always run tasks.                    final long ioTime = System.nanoTime() - ioStartTime;                    runAllTasks(ioTime * (100 - ioRatio) / ioRatio);                }            }        } catch (Throwable t) {            handleLoopException(t);        }        // Always handle shutdown even if the loop processing threw an exception.        try {            if (isShuttingDown()) {                closeAll();                if (confirmShutdown()) {                    return;                }            }        } catch (Throwable t) {            handleLoopException(t);        }    }}

    果然在NioEventLoop中找到了，这段代码在上一篇我们也看到过，就是轮询多路复用器，处理i/o事件的，这样想来，EventLoop线程启动后，在无线循环中select()完成的         SelectionKeys，并封装成任务，被分别处理。

    处理被选中的key：processSelectedKeys()

private void processSelectedKeys() {    if (selectedKeys != null) {        processSelectedKeysOptimized();    } else {        processSelectedKeysPlain(selector.selectedKeys());    }}

  如果selectedKeys不为空就调用优化过的SelectionKeys的处理方法，不然就处理普通的SelectionKeys集合，普通的SelectionKey容器为Set,优化过的容器用数组存储，具体可    查代码。  

  

private void processSelectedKeysPlain(Set<SelectionKey> selectedKeys) {    // check if the set is empty and if so just return to not create garbage by    // creating a new Iterator every time even if there is nothing to process.    // See https://github.com/netty/netty/issues/597    if (selectedKeys.isEmpty()) {        return;    }    Iterator<SelectionKey> i = selectedKeys.iterator();    for (;;) {        final SelectionKey k = i.next();        final Object a = k.attachment();        i.remove();        if (a instanceof AbstractNioChannel) {            processSelectedKey(k, (AbstractNioChannel) a);        } else {            @SuppressWarnings("unchecked")            NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;            processSelectedKey(k, task);        }        if (!i.hasNext()) {            break;        }        if (needsToSelectAgain) {            selectAgain();            selectedKeys = selector.selectedKeys();            // Create the iterator again to avoid ConcurrentModificationException            if (selectedKeys.isEmpty()) {                break;            } else {                i = selectedKeys.iterator();            }        }    }}

    看下这里

NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;            processSelectedKey(k, task);

/** * An arbitrary task that can be executed by {@link NioEventLoop} when a {@link SelectableChannel} becomes ready. * * @see NioEventLoop#register(SelectableChannel, int, NioTask) */public interface NioTask<C extends SelectableChannel>

    注释说这种nioTask会在channel的ready状态被nioEventLoop execute().

    处理任务的方法：runAllTasks()

/** * Poll all tasks from the task queue and run them via {@link Runnable#run()} method. * * @return {@code true} if and only if at least one task was run */protected boolean runAllTasks() {    assert inEventLoop();    boolean fetchedAll;    boolean ranAtLeastOne = false;    do {        fetchedAll = fetchFromScheduledTaskQueue();        if (runAllTasksFrom(taskQueue)) {            ranAtLeastOne = true;        }    } while (!fetchedAll); // keep on processing until we fetched all scheduled tasks.    if (ranAtLeastOne) {        lastExecutionTime = ScheduledFutureTask.nanoTime();    }    afterRunningAllTasks();    return ranAtLeastOne;}

/** * Runs all tasks from the passed {@code taskQueue}. * * @param taskQueue To poll and execute all tasks. * * @return {@code true} if at least one task was executed. */protected final boolean runAllTasksFrom(Queue<Runnable> taskQueue) {    Runnable task = pollTaskFrom(taskQueue);    if (task == null) {        return false;    }    for (;;) {        safeExecute(task);        task = pollTaskFrom(taskQueue);        if (task == null) {            return true;        }    }}

/** * Try to execute the given {@link Runnable} and just log if it throws a {@link Throwable}. */protected static void safeExecute(Runnable task) {    try {        task.run();    } catch (Throwable t) {        logger.warn("A task raised an exception. Task: {}", task, t);    }}

    它只是循环执行taskQueue中。

    现在，我们以一个事件跟踪整个流程:

ServerBootstrap b = new ServerBootstrap();b.group(bossGroup, workerGroup)        .channel(NioServerSocketChannel.class)        .option(ChannelOption.SO_BACKLOG, 1024)        .childHandler(new ChildChannelHandler());// 绑定端口，同步等待成功ChannelFuture f = b.bind(port).sync();

    我们就个弄bind()事件的处理流程。

/** * Create a new {@link Channel} and bind it. */public ChannelFuture bind(int inetPort) {    return bind(new InetSocketAddress(inetPort));}

/** * Create a new {@link Channel} and bind it. */public ChannelFuture bind(SocketAddress localAddress) {    validate();    if (localAddress == null) {        throw new NullPointerException("localAddress");    }    return doBind(localAddress);}

private ChannelFuture doBind(final SocketAddress localAddress) {    final ChannelFuture regFuture = initAndRegister();    final Channel channel = regFuture.channel();    if (regFuture.cause() != null) {        return regFuture;    }    if (regFuture.isDone()) {        // At this point we know that the registration was complete and successful.        ChannelPromise promise = channel.newPromise();        doBind0(regFuture, channel, localAddress, promise);        return promise;    } else {        // Registration future is almost always fulfilled already, but just in case it's not.        final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);        regFuture.addListener(new ChannelFutureListener() {            @Override            public void operationComplete(ChannelFuture future) throws Exception {                Throwable cause = future.cause();                if (cause != null) {                    // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an                    // IllegalStateException once we try to access the EventLoop of the Channel.                    promise.setFailure(cause);                } else {                    // Registration was successful, so set the correct executor to use.                    // See https://github.com/netty/netty/issues/2586                    promise.registered();                    doBind0(regFuture, channel, localAddress, promise);                }            }        });        return promise;    }}

private static void doBind0(        final ChannelFuture regFuture, final Channel channel,        final SocketAddress localAddress, final ChannelPromise promise) {    // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up    // the pipeline in its channelRegistered() implementation.    channel.eventLoop().execute(new Runnable() {        @Override        public void run() {            if (regFuture.isSuccess()) {                channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);            } else {                promise.setFailure(regFuture.cause());            }        }    });}

    到这里，我们看到eventLoop().execute()这里会生成新的Thread,并执行bind方法。后面的bind方法就不追踪了，涉及到具体的channel和pipeline等。