17 记一次 NOAUTH Authentication required

redisNotAuth 的问题

 

大概是上周四的时候吧, 把项目打包之后上传到测试服务器, 然后启动之后 出现了此异常

问题日志 [省略若干]

[12-08 17:52:27.579 INFO ][SingletonRedisRegionFactory.java:41] - Starting SingletonRedisRegionFactory...[12-08 17:52:27.584 INFO ][JedisTool.java:129] - Loading cache properties...path=[properties/redis.properties][12-08 17:52:27.586 WARN ][JedisTool.java:140] - Fail to load cache properties.cachePath=properties/redis.propertiesjava.lang.NullPointerException         atjava.util.Properties$LineReader.readLine(Properties.java:434)         atjava.util.Properties.load0(Properties.java:353)         atjava.util.Properties.load(Properties.java:341)         atorg.hibernate.cache.redis.util.JedisTool.loadCacheProperties(JedisTool.java:138)         atorg.hibernate.cache.redis.SingletonRedisRegionFactory.start(SingletonRedisRegionFactory.java:45)         atorg.hibernate.internal.CacheImpl.<init>(CacheImpl.java:70)         atorg.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:40)         atorg.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:35)         atorg.hibernate.service.internal.SessionFactoryServiceRegistryImpl.initiateService(SessionFactoryServiceRegistryImpl.java:91)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:251)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:225)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:206)         atorg.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:273)         atorg.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1859)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850)         atorg.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849)         atorg.hibernate.jpa.HibernatePersistenceProvider.createContainerEntityManagerFactory(HibernatePersistenceProvider.java:152)         atorg.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:343)         atorg.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:318)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)         atorg.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)         atorg.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)         atorg.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)         atorg.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)         atorg.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1054)         atorg.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:829)         atorg.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:538)         atorg.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)         atorg.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)         atorg.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)         atorg.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5068)         atorg.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5584)         atorg.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)         atorg.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)         atorg.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)         atorg.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)         atorg.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1092)         atorg.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1984)         atjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         atjava.util.concurrent.FutureTask.run(FutureTask.java:266)         atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         atjava.lang.Thread.run(Thread.java:745)[12-08 17:52:27.590 INFO ][JedisTool.java:59] - Creating JedisClient.[12-08 17:52:27.595 INFO ][JedisTool.java:86] - Creating JedisPool. host=[localhost], port=[6379],timeout=[2000], database=[0][12-08 17:52:27.779 INFO ][SingletonRedisRegionFactory.java:47] - Started SingletonRedisRegionFactory[12-08 17:52:27.782 INFO ][UpdateTimestampsCache.java:72] - HHH000250: Starting update timestamps cacheat region: org.hibernate.cache.spi.UpdateTimestampsCache[12-08 17:52:27.789 INFO ][JedisTool.java:129] - Loading cache properties...path=[properties/redis.properties][12-08 17:52:27.790 WARN ][JedisTool.java:140] - Fail to load cache properties.cachePath=properties/redis.propertiesjava.lang.NullPointerException         atjava.util.Properties$LineReader.readLine(Properties.java:434)         atjava.util.Properties.load0(Properties.java:353)         atjava.util.Properties.load(Properties.java:341)         atorg.hibernate.cache.redis.util.JedisTool.loadCacheProperties(JedisTool.java:138)         atorg.hibernate.cache.redis.AbstractRedisRegionFactory.loadCacheProperties(AbstractRedisRegionFactory.java:110)         atorg.hibernate.cache.redis.AbstractRedisRegionFactory.buildTimestampsRegion(AbstractRedisRegionFactory.java:173)        atorg.hibernate.cache.redis.SingletonRedisRegionFactory.buildTimestampsRegion(SingletonRedisRegionFactory.java:34)         atorg.hibernate.cache.spi.UpdateTimestampsCache.<init>(UpdateTimestampsCache.java:73)         atorg.hibernate.internal.CacheImpl.<init>(CacheImpl.java:72)         atorg.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:40)         atorg.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:35)         atorg.hibernate.service.internal.SessionFactoryServiceRegistryImpl.initiateService(SessionFactoryServiceRegistryImpl.java:91)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:251)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:225)         atorg.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:206)         atorg.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:273)         atorg.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1859)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850)         atorg.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425)         atorg.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849)         atorg.hibernate.jpa.HibernatePersistenceProvider.createContainerEntityManagerFactory(HibernatePersistenceProvider.java:152)         atorg.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:343)         atorg.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:318)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)         atorg.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)         atorg.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)         atorg.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)         atorg.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)         atorg.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)         atorg.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1054)         atorg.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:829)         atorg.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:538)         atorg.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)         atorg.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)         atorg.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)         atorg.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5068)         atorg.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5584)         atorg.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)         atorg.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)         atorg.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)         atorg.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)         atorg.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1092)         atorg.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1984)         atjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         atjava.util.concurrent.FutureTask.run(FutureTask.java:266)         atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         atjava.lang.Thread.run(Thread.java:745)省略若干和上面重复的日志 [12-08 17:52:28.952 WARN ][JedisClient.java:374] - Error in Cache Expiration Method.redis.clients.jedis.exceptions.JedisDataException:NOAUTH Authentication required.         atredis.clients.jedis.Protocol.processError(Protocol.java:117)         atredis.clients.jedis.Protocol.process(Protocol.java:151)         atredis.clients.jedis.Protocol.read(Protocol.java:205)         atredis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)         atredis.clients.jedis.Connection.getBinaryMultiBulkReply(Connection.java:233)         atredis.clients.jedis.BinaryJedis.zrangeByScore(BinaryJedis.java:2099)         atredis.clients.jedis.BinaryJedis.zrangeByScore(BinaryJedis.java:2093)         atorg.hibernate.cache.redis.jedis.JedisClient$13.execute(JedisClient.java:354)         atorg.hibernate.cache.redis.jedis.JedisClient$13.execute(JedisClient.java:351)         atorg.hibernate.cache.redis.jedis.JedisClient.run(JedisClient.java:612)         atorg.hibernate.cache.redis.jedis.JedisClient.expire(JedisClient.java:351)         atorg.hibernate.cache.redis.AbstractRedisRegionFactory$1.run(AbstractRedisRegionFactory.java:207)         atjava.lang.Thread.run(Thread.java:745)[12-08 17:52:28.954 WARN ][JedisClient.java:374] - Error in Cache Expiration Method.redis.clients.jedis.exceptions.JedisDataException:NOAUTH Authentication required.         atredis.clients.jedis.Protocol.processError(Protocol.java:117)         atredis.clients.jedis.Protocol.process(Protocol.java:151)         atredis.clients.jedis.Protocol.read(Protocol.java:205)         atredis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)         atredis.clients.jedis.Connection.getBinaryMultiBulkReply(Connection.java:233)         atredis.clients.jedis.BinaryJedis.zrangeByScore(BinaryJedis.java:2099)         atredis.clients.jedis.BinaryJedis.zrangeByScore(BinaryJedis.java:2093)         atorg.hibernate.cache.redis.jedis.JedisClient$13.execute(JedisClient.java:354)         atorg.hibernate.cache.redis.jedis.JedisClient$13.execute(JedisClient.java:351)         atorg.hibernate.cache.redis.jedis.JedisClient.run(JedisClient.java:612)         atorg.hibernate.cache.redis.jedis.JedisClient.expire(JedisClient.java:351)         atorg.hibernate.cache.redis.AbstractRedisRegionFactory$1.run(AbstractRedisRegionFactory.java:207)         atjava.lang.Thread.run(Thread.java:745)省略若干和上面重复的日志

 

 

当时碰到这个问题的时候, 第一反应是下载对应的源码跟一根  然而无奈下载的太慢, 下班了

 

项目中, 配置文件中指定"properties/redis.properties"的地方 大概是有两处, 第一处是spring指定占位符配置的位置, 另外一处是hibernate配置redis缓存的配置的位置

 

我一直以为是配置占位符配置的配置的地方出现的异常

然后回家之后 想了一下, 大概是 想了三种思路

1. 在我自己的虚拟机上面发布下项目看看有什么问题

2. 跟踪JedisTool的代码, 尝试解决问题

3. 既然是Properties.load报了NPE, 那么 应该是没有找到对应的文件吧, 因此 使用绝对路径应该是可以解决的, 但是 这样不好, 作为 实在解决不了问题的一种临时思路

 

 

然后第二天来到公司, 准备弄一弄这个问题, 然后首先是下载了源码, 然后跟了一下JedisTool的相关代码

 

第一个框是获取"hibernate.cache.provider_configuration_file_resource_path"的配置, 也就是这里的"properties/redis.properties"

第二个框, 根据特定的配置格式执行不同的加载方法, 一种是从文件系统加载, 另外一种使用classloader进行加载

第三个框就是我们报异常的地方, 走得是classloader加载配置, 然后获取到的inputStream为null, 然后 load的时候就报了此异常

 

 

至此, 我才发现, 原来报异常的的配置是hibernate的缓存配置, 而不是spring的占位符的配置文件

然后因此 我昨天思考的三个思路 就只能走走代码了,,

 

但是这里出现了一个使用绝对路径解决此问题的一个更加明显的解决方式, 也就是这个使用文件系统加载配置文件的方式

因此我最开始就是使用的这个来临时解决此问题

 

 

 

然后我跟了一下代码, 三个环境

1. 本地使用idea的发布web项目的方式

Jdk1.8 + apache-tomcat-8.0.37

 

2. 测试服务器 : jdk1.8 + tomcat 7.0.69.0

 

3. 我自己的虚拟机 : jdk 1.8 + tomcat8.5.6

 

我本地以及vm环境和测试环境还有一个不同在于, 前者的所有依赖的包存在于war下面的WEB-INF/lib下面, 然后测试环境的依赖包除了部分会时常变动的包[我们的common], 之外其他的包存在于$CATALINA_HOME/lib下面 [注意这个区别]

 

 

然后  看到这三个不同的classloader,我就蒙蔽了, 然后就没有想这个问题了, 然后周末的时候 无意间想到了tomcat 的类加载器的体系, 然后  画了一下, 但是 还是没有怎么想通

 

我顺着各个服务器的环境下面的classloader的”父子关系”往上走过

发现测试环境下面的的URLClassLoader其实就是tomcat中的CommonClassloader, 当然从上面的图中也能够看见

然后其他两个环境下面的WebAppClassLoader就是各个webapp对应的WebAppClassloader

但是当时 还不知所以然, 就没多想 然后 之后就到了今天下午

 

此图来自于 : http://www.cnblogs.com/jiaan-geng/p/4860432.html

 

 

然后直到今天 我们test1环境也出现了这个异常

然后我做了两次尝试

1. 我比较了一下test1, test2的项目war包，发现除了基本的配置文件存在差异 以及少量的版本提交的差异之外还有一个文件存在差异 log4j.xml, 然后我想起了之前几天leader将log4j的minLogLevel从error更新为了info, 查了一下提交日志, 问题好像就是在更新了之后出现的诶

我就怀疑是否是这个可能导致加载JedisTool的类加载器出现了问题，

然后我将console的info更新回error, 然后启动项目  果然就没有  再报这个错误了

然后我猛的发觉过来，之前没有报这个错误是不是因为输出级别是error所以  才没有把异常打印出来， 然后 我打开 文件形式记录的日志[输出级别为info]，刚才的启动过程果然也出现了这个异常,, 我擦嘞看来我们服务器是一直在报这个错误，只不过是我们将其日志输出关了。。然后因为这个配置是hibernate的缓存配置，因此没有对业务产生影响，因此就被忽略了

 

2. 将JedisTool所在的包[hiebernate-redis-xx.jar]移动到当前项目的/WEB-INF/lib下面, 然后 启动项目就好了

 

 

这里主要是是因为拿到“hibernate.cache.provider_configuration_file_resource_path”, 去读取数据依赖于加载JedisTools的classloader

由于在测试服务器环境下面, 所有的依赖的包都放在了 $CATALINA_HOME/lib下面, 由CommonClassloader加载了, 然后由CommonClassloader来找”properties/redis.properties”找不到, 因此 gg了

然后其他两个环境下面加载JedisTool是由当前项目的WebAppClassloader加载的, 然后使用该classloader寻找“properties/redis.properties”能够找到对应的资源, 完成了正常加载

 

 

对了最初的时候，我还怀疑过是不是环境的问题，然后我把vm上的环境复制了一份然后按照test环境部署了一下，然后  之后似乎启动项目的时候报了log4j的异常吧，然后 项目没有启动起来，然后 有不太想去检查环境 就算了

 

完