HttpClient大并发下Timeout waiting for connection from pool优化方案
来源:互联网 发布:isp网络数量 编辑:程序博客网 时间:2024/05/21 05:08
(本文的优化方案是由同事给出,我就对此做一下整理,记录分享给大家)
当前用的httpclient jar 包是 httpclient-4.3.6,相对应之前的版本是有一些不同的,用起来也会出现一些问题,但是可以通过配置参数解决我遇到的问题
HttpClient一些正常所需的参数:
//设置连接超时时间
private static int socketTimeout = 2000;private static int connectTimeout = 2000;private static int connectionRequestTimeout = 10000;
在做性能测试时,tps=50时,绝大部分的请求都是正常的,只有少数请求抛出了如下异常:
2015-12-23 11:50:45.716 INFO [[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'] AssetCrowdFundQueryServiceImpl line35 - AssetCrowdFundQueryServiceImpl queryCrowdFunding is SUCCESS, Exception{}java.io.IOException: org.apache.http.conn.ConnectionPoolTimeoutException: <span style="color:#ff0000;">Timeout waiting for connection from pool</span> at com.pingan.toa.asset.common.utils.http.HttpClientUtil.post(HttpClientUtil.java:180) ~[asset-common-1.0.0-SNAPSHOT.jar:na] at com.pingan.toa.asset.common.utils.http.HttpClientUtil.post(HttpClientUtil.java:158) ~[asset-common-1.0.0-SNAPSHOT.jar:na] at com.pingan.toa.asset.service.crowdfunding.impl.AssetCrowdFundQueryServiceImpl.queryCrowdFunding(AssetCrowdFundQueryServiceImpl.java:32) ~[asset-service-1.0.0-SNAPSHOT.jar:na] at com.pingan.toa.asset.facade.crowdfunding.impl.CrowdFundingAssetFacadeImpl.getCrowdFundInfo(CrowdFundingAssetFacadeImpl.java:173) ~[asset-facade-1.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_37] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_37] at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:188) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:104) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:204) ~[cxf-rt-frontend-jaxrs-2.7.17.jar:2.7.17] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:101) ~[cxf-rt-frontend-jaxrs-2.7.17.jar:2.7.17] at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:94) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[cxf-api-2.7.17.jar:2.7.17] at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:249) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:154) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:290) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:210) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) ~[javax.servlet_1.0.0.0_2-5.jar:2.5] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:265) ~[cxf-rt-transports-http-2.7.17.jar:2.7.17] at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227) ~[weblogic.jar:10.3.6.0] at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125) ~[weblogic.jar:10.3.6.0] at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301) ~[weblogic.jar:10.3.6.0] at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:27) ~[weblogic.jar:10.3.6.0] at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:61) ~[BUG20181997_1036011.jar:10.3.6.0] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:89) ~[spring-web-3.2.14.RELEASE.jar:3.2.14.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106) ~[spring-web-3.2.14.RELEASE.jar:3.2.14.RELEASE] at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:61) ~[BUG20181997_1036011.jar:10.3.6.0] at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3748) ~[BUG20181997_1036011.jar:10.3.6.0] at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3714) ~[BUG20181997_1036011.jar:10.3.6.0] at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) ~[com.bea.core.weblogic.security.identity_1.2.0.0.jar:1.2.0.0] at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120) [com.bea.core.weblogic.security.wls_1.0.0.0_6-2-0-0.jar:6.2.0.0] at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2283) [BUG20181997_1036011.jar:10.3.6.0] at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2182) [BUG20181997_1036011.jar:10.3.6.0] at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491) [BUG20181997_1036011.jar:10.3.6.0] at weblogic.work.ExecuteThread.execute(ExecuteThread.java:263) [BUG20181997_1036011.jar:1.11.0.0] at weblogic.work.ExecuteThread.run(ExecuteThread.java:221) [BUG20181997_1036011.jar:1.11.0.0]Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:254) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:231) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:173) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.3.6.jar:4.3.6] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106) ~[httpclient-4.3.6.jar:4.3.6] at com.pingan.toa.asset.common.utils.http.HttpClientUtil.post(HttpClientUtil.java:174) ~[asset-common-1.0.0-SNAPSHOT.jar:na] ... 40 common frames omitted
HttpClient在并发量高的时候,可能会出现连接池不够用的情况,可以通过配置总体最大连接池(maxConnTotal)和单个路由连接最大数(maxConnPerRoute),默认是(20,2)
maxConnTotal 和 maxConnPerRoute 的区别?
maxConnTotal是整个连接池的大小,根据自己的业务需求进行设置
maxConnPerRoute是单个路由连接的最大数,可以根据自己的业务需求进行设置
比如maxConnTotal =200,maxConnPerRoute =100,那么,如果只有一个路由的话,那么最大连接数也就是100了;如果有两个路由的话,那么它们分别最大的连接数是
100,总数不能超过200
private static CloseableHttpClient httpClient; private static int socketTimeout = 2000; private static int connectTimeout = 2000; private static int connectionRequestTimeout = 10000; private static int maxConnTotal = 200; //最大不要超过1000 private static int maxConnPerRoute = 100;//实际的单个连接池大小,如tps定为50,那就配置50 public static void init() { RequestConfig config = RequestConfig.custom() .setSocketTimeout(socketTimeout) .setConnectTimeout(connectTimeout) .setConnectionRequestTimeout(connectionRequestTimeout).build(); httpClient = HttpClients.custom().setDefaultRequestConfig(config) .setMaxConnTotal(maxConnTotal) .setMaxConnPerRoute(maxConnPerRoute).build(); } public static int getMaxConnTotal() { return maxConnTotal; } public static void setMaxConnTotal(int maxConnTotal) { HttpClientUtil.maxConnTotal = maxConnTotal; } public static int getMaxConnPerRoute() { return maxConnPerRoute; } public static void setMaxConnPerRoute(int maxConnPerRoute) { HttpClientUtil.maxConnPerRoute = maxConnPerRoute; }
当然,这些参数都是可以在配置文件里设置的。
1 0
- HttpClient大并发下Timeout waiting for connection from pool优化方案
- HttpClient大并发下Timeout waiting for connection from pool 问题解决方案
- ConnectionPoolTimeoutException:Timeout waiting for connection from pool
- 解决Httpclient 4 偶尔报错ConnectionPoolTimeoutException: Timeout waiting for connection from pool
- HttpClient连接池出现连续ConnectionPoolTimeoutException:Timeout waiting for connection from pool异常
- httpsclient 报错 timeout waiting for connection from pool
- http4.x报错Timeout waiting for connection from pool
- HttpClient ConnectionPoolTimeoutException: Timeout waiting for connection
- HttpClient ConnectionPoolTimeoutException: Timeout waiting for connection
- HttpClient连接池抛出大量ConnectionPoolTimeoutException: Timeout waiting for connection
- AxisFault: Timeout waiting for connection
- AxisFault: Timeout waiting for connection
- AxisFault: Timeout waiting for connection
- ConnectionPoolTimeoutException: Timeout waiting for connection
- Cannot get a connection, pool error Timeout waiting for idle object
- Cannot get a connection, pool error Timeout waiting for idle object
- Cannot get a connection, pool error Timeout waiting for idle object
- Cannot get a connection, pool error Timeout waiting for idle object解决方法
- 【mongodb】 log4j输出信息到mongodb
- javascript 对象类型
- Struts2中Result类型配置详解
- 过河卒
- Android权限 uses-permission
- HttpClient大并发下Timeout waiting for connection from pool优化方案
- 【oracle调优】优化全表扫---降低高水位
- C#中HttpWebRequest的用法详解
- .gitignore 与 Change-Id 的生成
- Git常用命令
- 重磅教程!帮你全面彻底搞定MATERIAL DESIGN的学习笔记
- 修改mybatis-generator-1.3.2源码实现自定义代码生成详解
- 【Java并发编程】深入分析ThreadLocal(八)
- JSP自定义简单标签入门之带有属性