ZooKeeper-ExpiryQueue详解

来源：互联网发布：西门子840d编程手册编辑：程序博客网时间：2024/06/16 23:56

ZooKeeper服务端管理客户端会话超时使用到ExpiryQueue，该数据结构在管理超时连接时的使用还比较巧妙，在这里还是研究一下，以供以后参考使用。

一、类图

图一

该队列的数据结构比较简单，包含两个Map，一个expirationInterval，一个nextExpirationTime，
一个私有方法，六个公共方法，一个构造函数。

二、数据模型

图二

ExpiryQueue根据expirationInterval将时间分段，将每段区间的时间放入对应的一个集合进行管理。如图二所示，时间段在1503556830000-1503556860000中的数据将会放到1503556860000对应的集合中，1503556860000-1503556890000中的数据将会放到1503556890000的集合中，以此类推。
在ExpiryQueue的数据结构中，图二中的集合由ConcurrentHashMap进行管理，其中的Key值为到期时间。
数据分段使用公式为:（当前时间(毫秒)/ expirationInterval + 1）* expirationInterval。该公式表示将当前时间按照expirationInterval间隔算份数，算完后再加一个份额，最后再乘以expirationInterval间隔，就得出了下一个到期时间。

三、源码分析

类名：org.apache.zookeeper.server.ExpiryQueue<E>

public class ExpiryQueue<E> {    private final ConcurrentHashMap<E, Long> elemMap = new ConcurrentHashMap<E, Long>();    /**     * The maximum number of buckets is equal to max timeout/expirationInterval,     * so the expirationInterval should not be too small compared to the     * max timeout that this expiry queue needs to maintain.     */    private final ConcurrentHashMap<Long, Set<E>> expiryMap = new ConcurrentHashMap<Long, Set<E>>();    private final AtomicLong nextExpirationTime = new AtomicLong();    private final int expirationInterval;    public ExpiryQueue(int expirationInterval) {        this.expirationInterval = expirationInterval;        nextExpirationTime.set(roundToNextInterval(Time.currentElapsedTime()));    }    private long roundToNextInterval(long time) {        return (time / expirationInterval + 1) * expirationInterval;    }    /**     * Removes element from the queue.     * @param elem  element to remove     * @return      time at which the element was set to expire, or null if     *              it wasn't present     */    public Long remove(E elem) {        Long expiryTime = elemMap.remove(elem);        if (expiryTime != null) {            Set<E> set = expiryMap.get(expiryTime);            if (set != null) {                set.remove(elem);                // We don't need to worry about removing empty sets,                // they'll eventually be removed when they expire.            }        }        return expiryTime;    }    /**     * Adds or updates expiration time for element in queue, rounding the     * timeout to the expiry interval bucketed used by this queue.     * @param elem     element to add/update     * @param timeout  timout in milliseconds     * @return         time at which the element is now set to expire if     *                 changed, or null if unchanged     */    public Long update(E elem, int timeout) {        Long prevExpiryTime = elemMap.get(elem);        long now = Time.currentElapsedTime();        Long newExpiryTime = roundToNextInterval(now + timeout); //计算对应元素的到期时间，该处加了timeout一般会增加一个到期时间间隔        if (newExpiryTime.equals(prevExpiryTime)) {            // No change, so nothing to update            return null;        }        // First add the elem to the new expiry time bucket in expiryMap.        Set<E> set = expiryMap.get(newExpiryTime);        if (set == null) {            // Construct a ConcurrentHashSet using a ConcurrentHashMap            set = Collections.newSetFromMap(                new ConcurrentHashMap<E, Boolean>());            // Put the new set in the map, but only if another thread            // hasn't beaten us to it            Set<E> existingSet = expiryMap.putIfAbsent(newExpiryTime, set);            if (existingSet != null) {                set = existingSet;            }        }        set.add(elem);        // Map the elem to the new expiry time. If a different previous        // mapping was present, clean up the previous expiry bucket.        prevExpiryTime = elemMap.put(elem, newExpiryTime);        if (prevExpiryTime != null && !newExpiryTime.equals(prevExpiryTime)) {            Set<E> prevSet = expiryMap.get(prevExpiryTime);            if (prevSet != null) {                prevSet.remove(elem);  //删除原来到期时间的数据            }        }        return newExpiryTime;    }    /**     * @return milliseconds until next expiration time, or 0 if has already past     */    public long getWaitTime() {        long now = Time.currentElapsedTime();        long expirationTime = nextExpirationTime.get();        return now < expirationTime ? (expirationTime - now) : 0L;    }    /**     * Remove the next expired set of elements from expireMap. This method needs     * to be called frequently enough by checking getWaitTime(), otherwise there     * will be a backlog of empty sets queued up in expiryMap.     *     * @return next set of expired elements, or an empty set if none are     *         ready     */    public Set<E> poll() {        long now = Time.currentElapsedTime();        long expirationTime = nextExpirationTime.get();        if (now < expirationTime) {            return Collections.emptySet();        }        Set<E> set = null;        long newExpirationTime = expirationTime + expirationInterval;        if (nextExpirationTime.compareAndSet(              expirationTime, newExpirationTime)) {            set = expiryMap.remove(expirationTime);        }        if (set == null) {            return Collections.emptySet();        }        return set;    }    public void dump(PrintWriter pwriter) {        pwriter.print("Sets (");        pwriter.print(expiryMap.size());        pwriter.print(")/(");        pwriter.print(elemMap.size());        pwriter.println("):");        ArrayList<Long> keys = new ArrayList<Long>(expiryMap.keySet());        Collections.sort(keys);        for (long time : keys) {            Set<E> set = expiryMap.get(time);            if (set != null) {                pwriter.print(set.size());                pwriter.print(" expire at ");                pwriter.print(Time.elapsedTimeToDate(time));                pwriter.println(":");                for (E elem : set) {                    pwriter.print("\t");                    pwriter.println(elem.toString());                }            }        }    }    /**     * Returns an unmodifiable view of the expiration time -> elements mapping.     */    public Map<Long, Set<E>> getExpiryMap() {        return Collections.unmodifiableMap(expiryMap);    }}

在update方法中，我们可以看到计算到期时间时，调用方法为roundToNextInterval(now + timeout)，在这里使用当前时间加一个超时时间，该方式计算出来的到期时间一般会增加一个时间间隔。
在update方法中，会执行以下三个步骤：
1)计算到期时间
2)将元素加入到对应的到期时间的Set集合中
3)判断元素原来的到期时间，并移除原到期时间Set集合中该元素。

poll方法的作用是弹出当前到期时间对应的Set集合。该方法会判断当前时间和到期时间的大小，如果当前时间小于到期时间，则返回空集合；如果当前时间大于或等于到期时间，则设置下个到期时间，并返回expiryMap中对应的到期时间的Set集合。

四、如何使用

在该对象的使用方式上，我们可以构建一个超时检测线程，将该队列作为成员变量，一直循环调用getWaitTime方法，如果该方法返回值大于0，则休眠相应时间，然后继续循环，如果返回值小于或等于0，则调用poll方法，对返回集合做对应的超时业务处理。

    public static void main(String[] args) {        int timeOut = 30 * 1000;        ExpiryQueue<Long> expiryQueue = new ExpiryQueue<Long>(timeOut);        ExpiryThread thread = new ExpiryThread(expiryQueue);        thread.setDaemon(true);        thread.start();    }    private static class ExpiryThread extends Thread {        private final ExpiryQueue<Long> expiryQueue;        ExpiryThread(ExpiryQueue<Long> expiryQueue) {            this.expiryQueue = expiryQueue;        }        @Override        public void run() {            while (true) {                try {                    long waitTime = expiryQueue.getWaitTime();                    if (waitTime > 0) {                        Thread.sleep(waitTime);                    }                    Set<Long> expirySet = expiryQueue.poll();                    for (Long expiry : expirySet) {                        System.out.println(expiry);                    }                } catch (Exception e) {                    e.printStackTrace();                }            }        }    }

阅读全文

1 0