Memcached Java客户端2.6.1发布

来源:互联网 发布:可视化图表制作软件 编辑:程序博客网 时间:2024/06/10 20:48

Memcached Java客户端2.6.1发布

    博客分类: 
  • memcached
memcachedJavaGmailITeyeperformance

Memcached是被广泛使用的分布式缓存技术。不同的语言有不同的Memcached客户端程序,对于Java客户端来说,首推Memcached Java Client(http://github.com/gwhalin/Memcached-Java-Client )。

这次,Memcached Java Client推出的2.6.1发布版是基于全新的performance分支,具有如下重大改进:

  1. 较之老版本,在性能上有300%左右的提升;
  2. 兼容老版本,用户无须修改自己的源代码;
  3. 支持多个memcached协议,包括text,udp和binary协议;
  4. 支持SASL认证机制;
  5. 重新实现的连接池,修复了之前的连接数太多所导致的OutOfMemory异常;
  6. 加入了slf4j logger支持,使得开发人员可以方便的记录日志;
  7. 支持自定义的对象序列化方法。

这个分支由Schooner Information Technology贡献,并由Schooner中国团队完成开发,开发者是:

  1. 王新根,new.root@gmail.com;
  2. 李蒙,jowett.lee@gmail.com,IT eye博客地址是:http://menglee.iteye.com

可以从这里下载二进制包:https://github.com/gwhalin/Memcached-Java-Client/downloads 

源代码在github上,http://github.com/gwhalin/Memcached-Java-Client ,然后选择performance分支。

下面是一些性能测试的数据,包括了当前流行的Memcached Java Client。其中,schooner指的是这个分支的text protocol, schooner_bin指的是binary protocol。





 

  • 查看图片附件
分享到:  
Axis2/C 基础入门 | Linux 0.11内核编译和bochs上的实验环境的 ...
  • 2011-06-25 18:11
  • 浏览 15775
  • 评论(18)
  • 论坛回复 / 浏览 (16 / 5265)
  • 分类:编程语言
  • 查看更多
相关资源推荐
  • unix系列系统镜像下载
  • SUN的UNIX操作系统
  • UNIX操作系统设计
  • 谈谈网络编程(基于C++)
  • C/C++ socket网络编程扫盲篇
  • C++基于TCP/IP简单的客户端、服务器通信程序实例
  • linux下C/C++网络编程基本:socket实现tcp和udp的例子
  • C/C++网络编程总结与ZeroMQ
  • C++网络编程
  • C++之网络编程
  • C++网络编程方面的开源项目
  • C++ P2P网络编程
  • C/C++实现基于UDP的网络编程
  • C++网络编程之客户端程序
  • C\C++网络编程总结
  • C++网络编程 卷1和卷2 (中文,pdf版)
  • DB2 v8.2,v9.1,v9.5,v9.7下载地址及详细安装图解
  • DB2全套学习资料
  • DB2数据库管理软件 - db2mon v1.0
  • IBM DB2 通用数据库 在64位平台上使用DB2 通用数据库.rar
参考知识库
Android知识库38653  关注 | 3162  收录
React知识库3831  关注 | 393  收录
人工智能基础知识库17720  关注 | 212  收录
Java 知识库37185  关注 | 3748  收录
评论
18 楼 747578076 2015-09-02  
您好,请问我用2.6.6版本,请问SockIOPool下的哪个方法 配置 所连接的Memcache的用户密码登陆?
17 楼 modowei 2013-07-30  
问下性能提升了300%上针对1.X版本吗?
我测了下2.5X和3.0的性能没什么差别
到是比1.X提升了3、4倍
16 楼 sunwine 2013-05-29  
继续吐槽,还有Bug
failover,failback开启时,性能问题
failback开启时,如果一个memcached挂掉,由于程序始终要做back尝试(没有详细阅读这块代码,猜测),导致任何一次访问挂掉memcached服务器上数据,都会尝试建立访问挂掉服务器的socket,导致每次访问都要消耗connectTimeout时间,返回非常慢

大体看了下whalin版本的
在selfMaint做维护时尝试访问挂掉服务器,如果不成功等待1秒,下次再尝试时还不成功等待1*2秒,以此类推,好像whalin版本中这个自维护还是有些价值的

楼主,要多做测试啊

还有个建议,增加一个动态增减memcached服务器的函数,这样子可靠性才好,否则一旦memcached内存不够用,只能停web服务啊,学习下xmemcached
15 楼 sunwine 2013-05-28  
正在试用这个版本,好像测试得不是很完整啊,有几个Bug啊:
storeCounter函数好像不正确,直接存储Long型数据到服务器,后续Incr/Decr就不正确了:
public boolean storeCounter(String key, Long counter, Date date, Integer hashCode) {  
  return set(key, counter, date, hashCode);
  此处counter没有转换为字符串再存储
}

getCounter函数好像也不正确,连接MemCache1.2.6(Windows版本)时,如果做Decr,一旦最后数值小于初始值长度时,服务器返回的数值字符串中会含有空格,此时getCounter获取会失败,返回-1,例如:初始计数器值为10,做一次Decr后值为9,此时从服务器获取的实际数据是“9 ”(9加一个空格),此时getCounter把该字符串转换为Long型会出错


14 楼 goohere 2013-02-05  
作者用ObjectTransCoder来实现对象的解码,你set的值也的相对应编码才行

25weiqin 写道
楼主好。在使用memcached client 2.5 和2.6版本时,都遇到如下的错误,还请帮忙帮看一下:
现象:从mysql memcached客户端插入一条记录,如:"test","test",在telnet 命令行和 memcached client 2.0.1版本(设置setPrimitiveAsString(true))都get获取正常。但是使用版本 2.5,2.5.1 和 2.6时都会抛出如下的错误
// exception thrown
log.error("++++ exception thrown while trying to get object from cache for key: " + key);

com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - ++++ exception thrown while trying to get object from cache for key: test117
com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - invalid stream header: 76616C75
java.io.StreamCorruptedException: invalid stream header: 76616C75
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.<init>(Unknown Source)
at com.schooner.MemCached.ObjectTransCoder.decode(ObjectTransCoder.java:57)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:761)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:612)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:608)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at TestM.main(TestM.java:56)

操作时通过设置可分布式环境的要求变量 client。setPrimitiveAsString(true),和 pool.setHashingAlg(2);还请楼主帮忙看一下是怎么回事?

13 楼 jade88 2013-01-11  
楼主,你好,如何关闭连接,shutdown不管用。
12 楼 bukebuhao 2012-09-05  
Memcached Java Client2.5.2,线上出现java.lang.OutOfMemoryError 
at sun.misc.Unsafe.allocateMemory(Native Method) 
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:126) 
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) 
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.<init>(Unknown Source)。 
最新的2.6.1已修复这个bug,不过,我本地模拟大并发量,为何2.5.2也不出现内存溢出。 
能不能给我一段测试代码,重现bug,好确认是不是这个问题。谢谢!!配置参数最好给个参考实例
11 楼 287854442 2012-08-06  
请问有没有2.6.3版本的api文档??
10 楼 sankby 2012-04-13  
为什么我用binary比ascii的慢一倍,需要怎么设置?有测试过的吗
9 楼 rainmanz 2012-03-06  
楼主有没有试过当set的值过大的时候会出现下面的错误
[2012-03-05 22:58:29.872]-[ERROR] com.danga.MemCached.MemCachedClient ++++ exception thrown while writing bytes to server on get
[2012-03-05 22:58:29.874]-[ERROR] com.danga.MemCached.MemCachedClient An existing connection was forcibly closed by the remote host
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:33)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:26)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:1547)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:1482)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:738)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:734)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:917)
at com.eshore.crm.memcache.client.SimpleTest$1.run(SimpleTest.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
null

测试代码如下:

import java.util.UUID;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import com.danga.MemCached.MemCachedClient;
import com.danga.MemCached.SockIOPool;

public class SimpleTest {

// create a static client as most installs only need
// a single instance
protected static MemCachedClient mcc;

// set up connection pool once at class load
static {

    // server list and weights
    String[] servers = { "10.17.13.49:11212" };

    Integer[] weights = { 3, 3, 2 };

    // grab an instance of our connection pool
    SockIOPool pool = SockIOPool.getInstance();

    // set the servers and the weights
    pool.setServers(servers);
    pool.setWeights(weights);

    // set some basic pool settings
    // 5 initial, 5 min, and 250 max conns
    // and set the max idle time for a conn
    // to 6 hours
    pool.setInitConn(5);
    pool.setMinConn(5);
    pool.setMaxConn(250);
    pool.setMaxIdle(1000 * 60 * 60 * 6);

    // set the sleep for the maint thread
    // it will wake up every x seconds and
    // maintain the pool size
    pool.setMaintSleep(30);

    // set some TCP settings
    // disable nagle
    // set the read timeout to 3 secs
    // and don’t set a connect timeout
    pool.setNagle(false);
    pool.setSocketTO(3000);
    pool.setSocketConnectTO(0);

    pool.setBufferSize(30000000);

    // initialize the connection pool
    pool.initialize();

    // lets set some compression on for the client
    // compress anything larger than 64k
    //mcc.setCompressEnable(true);
    //mcc.setCompressThreshold(64 * 1024);
    mcc = new MemCachedClient(true);
}

public static void main(String args[]){
    int k=1;
    String[] values= new String[8000];
    int j = 0;
    for (int i = 0; i < keys.length; i++) {
        keys[j] = "3";
        j++;
    }
    BlockingQueue<Runnable> startStopQueue = new LinkedBlockingQueue<Runnable>();
    ThreadPoolExecutor taskExecutor = new ThreadPoolExecutor(10,
            10, 10, TimeUnit.SECONDS,
            startStopQueue);
    taskExecutor.allowCoreThreadTimeOut(true);
    final String[] vs=values;
    final MemCachedClient mc = mcc;
    while(k>0){
        //mcc.delete("foo");
        taskExecutor.submit(new Runnable(){

            @Override
            public void run() {
                    //mcc = new MemCachedClient(true);
                    String key = "foo"+UUID.randomUUID();
                    mcc.set(key, vs);
                    Object obj = mcc.get(key);
                    System.out.println(obj);
            }

        });
        try {
            TimeUnit.MICROSECONDS.sleep(100);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        k--;
    }

    taskExecutor.shutdown();
    while(!taskExecutor.isTerminated()){
        try {
            TimeUnit.SECONDS.sleep(1);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    System.exit(1);
}
}

服务端日志:

<30 new auto-negotiating client connection
30: Client using the binary protocol
<30 Read binary protocol data:
<30 0x80 0x01 0x00 0x27
<30 0x08 0x00 0x00 0x00
<30 0x00 0x00 0x62 0x02
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 SET foo5eb1b171-68e6-4d82-af89-ba9fed2f368e Value len is 25043
Failed to read, and not due to blocking:
errno: 0 Error 0 
rcurr=57ccac4a ritem=5a4272d5 rbuf=57cca2a0 rlbytes=22640 rsize=4096
<30 connection closed.
8 楼 cmysunny 2012-02-24  
原来项目用2.5.*,put到cache中的对象要实现可序列化接口。项目升级jar包使用2.6.3版本遇到下面这个问题,请楼主百忙之中抽空回复。
google之后,说是新版本不需要序列化,可是使用hibernate框架,entity必须实现序列化接口,请问新的版本有没有提供可配置方式?
下面是异常信息,偶现概率在20%左右
2012-02-24 13:14:39 ERROR[resin-http-:8080-6]com.schooner.MemCached.AscIIClient.get(?:?)- ++++ exception thrown while trying to get object from cache for key: zjwapKeyPefixFrontip_white_list
2012-02-24 13:14:39 ERROR[resin-http-:8080-6]com.schooner.MemCached.AscIIClient.get(?:?)- cn.m8book.cmread.entity.AccessAuthority
java.io.IOException: cn.m8book.cmread.entity.AccessAuthority
at com.schooner.MemCached.ObjectTransCoder.decode(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.danga.MemCached.MemCachedClient.get(Unknown Source)
at com.index_china.common.cache.memcache.MemCachedManager.getFromCache(MemCachedManager.java:250)
at com.index.cmread.filter.MobileSystemContextFilter.doFilter(MobileSystemContextFilter.java:151)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:76)
at com.index.cmread.filter.AccessAuthorityFilter.doFilter(AccessAuthorityFilter.java:62)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:76)
at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:76)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:76)
at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:158)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:178)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:241)
at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:268)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:586)
at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:690)
at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:612)
at java.lang.Thread.run(Thread.java:619)
7 楼 zogwei 2011-10-27  
具有随机性,不是每次都出现
6 楼 zogwei 2011-10-27  
我使用2.6.1版本遇到这问题,楼主能帮忙看看吗?
2011-10-27 19:50:48,777 ERROR (com.danga.MemCached.MemCachedClient:443) - ++++ exception thrown while writing bytes to server on set
2011-10-27 19:50:48,793 ERROR (com.danga.MemCached.MemCachedClient:444) - An established connection was aborted by the software in your host machine
java.io.IOException: An established connection was aborted by the software in your host machine
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:33)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:60)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:1551)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:427)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:240)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:646)
5 楼 25weiqin 2011-08-17  
楼主好。在使用memcached client 2.5 和2.6版本时,都遇到如下的错误,还请帮忙帮看一下:
现象:从mysql memcached客户端插入一条记录,如:"test","test",在telnet 命令行和 memcached client 2.0.1版本(设置setPrimitiveAsString(true))都get获取正常。但是使用版本 2.5,2.5.1 和 2.6时都会抛出如下的错误
// exception thrown
log.error("++++ exception thrown while trying to get object from cache for key: " + key);

com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - ++++ exception thrown while trying to get object from cache for key: test117
com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - invalid stream header: 76616C75
java.io.StreamCorruptedException: invalid stream header: 76616C75
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.<init>(Unknown Source)
at com.schooner.MemCached.ObjectTransCoder.decode(ObjectTransCoder.java:57)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:761)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:612)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:608)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at TestM.main(TestM.java:56)

操作时通过设置可分布式环境的要求变量 client。setPrimitiveAsString(true),和 pool.setHashingAlg(2);还请楼主帮忙看一下是怎么回事?
4 楼 ypf3027 2011-06-30  
2.6.1版的MemCachedClient删除了2.5.1的所有deprecated方法, 
其中setCompressEnable和setCompressThreshold方法是实现数据压缩的设置,删除了这两个方法,是自动实现压缩设置了吗?
3 楼 jsczxy2 2011-06-27  
用了这个 Memcached Java Client 不错的 效率是越来越高了
2 楼 MengLee 2011-06-26  
hxhnarkissos 写道
看到楼主头像想到韩寒···· 
直接拿你的客户端测试下负载均衡中session情况试试

欢迎试用!
1 楼 hxhnarkissos 2011-06-26  
看到楼主头像想到韩寒···· 
直接拿你的客户端测试下负载均衡中session情况试试
原创粉丝点击