求救SSH系统性能问题-数据库连接泄漏问题[由于帖子不能贴图片，暂时先把问题放这了]

     这段时间一直被一个SSH系统性能问题困扰，项目在开发过程中出现Tomcat假死的问题。为了查清系统问题的原因，我找了个SIW软件用来监控系统占用端口，发现数据库连接数时常达到连接池设置的最大连接数。我使用的C3P0连接池，我一开始认为是连接池设置问题，可是问题没有解决。然后开发小组开始修改SSH配置文件，貌似系统偶尔能释放部分数据库连接，可是问题还是没有解决。
    后来请教了一技术牛人，我们修改了连接池，把C3P0连接池换成DBCP连接池，并且把连接池配置由hibernate.cfg.xml配置文件中迁移到了Spring配置文件中（为了节省时间，我也就不贴连接池配置的代码了），这下系统能释放大部分连接。可是通过监控发现，保持的数据库连接数还是多于连接池配置的空闲连接数，而且时间一久，系统还是会出现卡死的问题，估计就是连接池保持的连接已经失效导致的。
    今天下午，我又上网查了这方面的系统问题的介绍资料，偶尔发现网上介绍JConsole监控JVM和开启线程的相关技术（这里我也不介绍JConsole技术，因为网上实在太多了），了解到JVM设置和数据库连接泄漏都会导致系统问题。因为我对JConsole监控信息是如何反映系统问题不是很了解，所以就将感觉能反映系统性能问题的监控信息贴出来，希望大牛们能明白。

图一 JVM监控信息

图2  内存监控

图3 Tomcat线程监控

    Tomcat线程监控中发现【http-8080-XX】线程很有相似性，现在随机贴出两个线程堆栈追踪信息:

名称： http-8080-1
状态：WAITING 在 com.mchange.v2.resourcepool.BasicResourcePool@10d3f0d 上
阻塞总数：9  等待总数： 6

堆栈追踪：
java.lang.Object.wait(Native Method)
com.mchange.v2.resourcepool.BasicResourcePool.awaitAcquire(BasicResourcePool.java:966)
com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:208)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:232)
com.mchange.v2.c3p0.PoolBackedDataSource.getConnection(PoolBackedDataSource.java:94)
org.hibernate.connection.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:35)
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:417)
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:144)
org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:494)
org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:315)
org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:257)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:102)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
$Proxy1.queryDiskList(Unknown Source)
com.iwork.docmgr.action.DiskAction.readDiskInfo(DiskAction.java:59)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:440)
com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:279)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:242)
com.util.LoginInterceptor.intercept(LoginInterceptor.java:24)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept(DefaultWorkflowInterceptor.java:163)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(ValidationInterceptor.java:249)
org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.doIntercept(AnnotationValidationInterceptor.java:68)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.intercept(ConversionErrorInterceptor.java:122)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:148)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.CheckboxInterceptor.intercept(CheckboxInterceptor.java:93)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:235)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:89)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ScopedModelDrivenInterceptor.intercept(ScopedModelDrivenInterceptor.java:128)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.ProfilingActivationInterceptor.intercept(ProfilingActivationInterceptor.java:104)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.debugging.DebuggingInterceptor.intercept(DebuggingInterceptor.java:267)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:126)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:138)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:148)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:164)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.AliasInterceptor.intercept(AliasInterceptor.java:128)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:176)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:52)
org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:468)
org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:77)
org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:76)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:78)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:216)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445)
java.lang.Thread.run(Thread.java:619)

 

名称： http-8080-23
状态：WAITING 在 com.mchange.v2.resourcepool.BasicResourcePool@10d3f0d 上
阻塞总数：0  等待总数： 2

堆栈追踪：
java.lang.Object.wait(Native Method)
com.mchange.v2.resourcepool.BasicResourcePool.awaitAcquire(BasicResourcePool.java:966)
com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:208)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:232)
com.mchange.v2.c3p0.PoolBackedDataSource.getConnection(PoolBackedDataSource.java:94)
org.hibernate.connection.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:35)
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:417)
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:144)
org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:494)
org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:315)
org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:257)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:102)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:209)
$Proxy26.userLogin(Unknown Source)
com.iwork.basic.action.LoginAction.login(LoginAction.java:78)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:440)
com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:279)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:242)
com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept(DefaultWorkflowInterceptor.java:163)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(ValidationInterceptor.java:249)
org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.doIntercept(AnnotationValidationInterceptor.java:68)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.intercept(ConversionErrorInterceptor.java:122)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:148)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.CheckboxInterceptor.intercept(CheckboxInterceptor.java:93)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:235)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:89)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ScopedModelDrivenInterceptor.intercept(ScopedModelDrivenInterceptor.java:128)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.ProfilingActivationInterceptor.intercept(ProfilingActivationInterceptor.java:104)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.debugging.DebuggingInterceptor.intercept(DebuggingInterceptor.java:267)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:126)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:138)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:148)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:164)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.AliasInterceptor.intercept(AliasInterceptor.java:128)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:176)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:236)
org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:52)
org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:468)
org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:77)
org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:76)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:78)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:216)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445)
java.lang.Thread.run(Thread.java:619)

 

请求大牛们给出紧急援助！！！