solr中 Could not connect to ZooKeeper
来源:互联网 发布:php手游平台源码 编辑:程序博客网 时间:2024/05/19 13:09
这是在通过写MapReduce读取HBase的数据,给指定的列加上索引到solr发现的
zookeeper.out日志:
2015-11-20 16:59:20,603 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 602015-11-20 16:59:20,619 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60
2015-11-20 16:59:20,762 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60
2015-11-20 16:59:20,774 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60
2015-11-20 16:59:20,915 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60
2015-11-20 16:59:20,973 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60
2015-11-20 16:59:21,219 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60
2015-11-20 16:59:22,058 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60
MapReduce执行报错:
15/11/20 16:59:41 INFO mapreduce.Job: Task Id : attempt_1448000031899_0003_m_000012_2, Status : FAILED
Error: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181 within 10000 ms
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:164)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:115)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:105)
at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:212)
at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:241)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:524)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at com.yiban.mapreduce.HBaseToSolrIndex$HBaseToSolrIndexMapper.map(HBaseToSolrIndex.java:60)
at com.yiban.mapreduce.HBaseToSolrIndex$HBaseToSolrIndexMapper.map(HBaseToSolrIndex.java:29)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181 within 10000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:156)
... 18 more
查看zookeeper 2181端口的连接数:使用netstat -nap | grep 2181 | wc -l查看
正常连接数
[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l
11
[root@slave02 lib]# netstat -nap | grep 2181 | wc -l
11
[root@slave03 lib]# netstat -nap | grep 2181 | wc -l
15
运行MapReduce连接数,太多不正常
[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l
801
[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l
826
[root@slave02 lib]# netstat -nap | grep 2181 | wc -l
566
[root@slave02 lib]# netstat -nap | grep 2181 | wc -l
641
[root@slave03 lib]# netstat -nap | grep 2181 | wc -l
196
[root@slave03 lib]# netstat -nap | grep 2181 | wc -l
529
[root@slave03 lib]# netstat -nap | grep 2181 | wc -l
563
[root@slave03 lib]# ^C
[root@slave03 lib]# netstat -nap | grep 2181 | wc -l
546
有问题代码:
写在MapReduce里的map方法里面(这里是局部变量,每调用一次map方法就会创建一个zookeeper连接,导致连接数直线上升)
CloudSolrServer solrServer = new CloudSolrServer("slave01:2181,slave02:2181,slave03:2181");//获取zookeeper地址
正常代码:
写在加载时的静态块里面(整个MapReduce只创建一个zookeeper连接)
private static CloudSolrServer solrServer;
static{
solrServer = new CloudSolrServer("slave01:2181,slave02:2181,slave03:2181");//获取zookeeper地址
}
总结:不管是什么连接,能用全局创建尽量用全局创建,这样能避免连接数过多问题
阅读全文
0 0
- solr中 Could not connect to ZooKeeper
- React-Native开发中Could not connect to development server
- Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181
- mplayer: could not connect to socket mplayer
- ideviceinstaller Could not connect to lockdownd. Exiting.
- Could not connect to lockdownd. Exiting.
- SVN的could not connect to server
- Could not connect to lockdownd. Exiting.
- Could not connect to lockdownd. Exiting.
- Warning: Could not connect to Nexus.解决方案
- 解决 dropdb: could not connect to database postgres: could not connect to server:
- Eclipse中使用SVN连接Google Code 报could not connect to server错误解决方法
- centos中apache连接postgresql提示SQLSTATE[08006] [7] could not connect to server: Permission denied
- 关于react-native 运行项目中遇到could not connect to development server的问题
- 解决qt安装过程中QXcbConnection: Could not connect to display问题
- Could not connect to Redis No route to host问题解决
- Could not connect to Redis No route to host问题解决
- javamail javax.mail.MessagingException: Could not connect to SMTP host:
- android 收不到ACTION_MEDIA_MOUNTED 广播
- webapi 开发日记
- java.io.IOException: Unable to open sync connection!
- 光环中国·2017敏捷千人峰会早鸟票
- 查看日志常用命令
- solr中 Could not connect to ZooKeeper
- Adb Push apk简单命令
- junit基础
- CodeForces
- html angular+jquery:判断密码并改变标签样式+新增提示内容
- OMP 正交匹配追踪
- 算术赋值运算符与自增自减运算符
- 给定一个m×n矩阵,如果一个元素为0,则将其整行和列的值设置为0.
- ASM正常冗余下模拟磁盘故障的恢复