解决java连接Hbase程序卡住执行也不报错的问题

来源:互联网 发布:lightroom mac版百度云 编辑:程序博客网 时间:2024/05/21 01:31

问题描述:

最近在hadoop集群上搭建好Hbase,用hbase shell进行增删改查操作都没有问题,但是用java API来对hbase进行操作却出现一个很奇怪的问题,就是执行java程序后没有任何反应,程序也不报错。

首先贴出我的测试代码:

import java.io.IOException;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.hbase.HBaseConfiguration;import org.apache.hadoop.hbase.client.Get;import org.apache.hadoop.hbase.client.HTable;import org.apache.hadoop.hbase.client.Result;import org.apache.hadoop.hbase.client.ResultScanner;import org.apache.hadoop.hbase.client.Scan;import org.apache.hadoop.hbase.util.Bytes;public class Test {private static Configuration conf=HBaseConfiguration.create();static {conf=HBaseConfiguration.create();//注释部分如果在本地测试不需要加,要是远程连接Hbase需要制定hbase地址//conf.set("hbase.master", "master:60000");//conf.set("hbase.zookeeper.quorum","slave,slave2,slave3");System.out.println(conf.get("hbase.master"));}public static void main(String[] args) throws IOException {// TODO Auto-generated method stubHTable table = new HTable(conf,"test1");Scan s = new Scan();ResultScanner rs = table.getScanner(s);for(Result res:rs){System.out.println(res);}table.close();}}

执行程序,发现程序并没有输出查到的表内的值,奇怪的是程序不报任何异常,也不产生输出,所以应该是卡住了,调试程序发现,程序会卡在
ResultScanner rs = table.getScanner(s);

这句话,应该是调用HTable的方法问题,查看zookeeper日志后发现,程序一直在尝试连接hbase,但是timeout,也就是连接超时,程序不断尝试,所以会出现卡住的现象。

而且查看Hbase-master日志发现没有任何写入,说明程序根本还没运行到要用到hbase的时候就已经卡住了,也侧面印证了我的Hbase配置是没有问题的。所以我判断是java连接Hbase过程出了问题。

接下来的几天我一直在查,但是一直没有解决,但是又一次我执行程序卡住之后,没退出程序就走了,等很久我回来时,发现程序报错了!(应该是长时间尝试连接,达到了设定的timeout最大值)

报错如下:

log4j:WARN No appenders could be found for logger (org.apache.hadoop.security.Groups).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.Exception in thread "main" java.net.SocketTimeoutException: callTimeout=1200000, callDuration=19813388: row '' on table 'TEST1' at nullat org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:143)at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:91)at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:289)at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:190)at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:185)at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:111)at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:780)at hbasetest.Test.main(Test.java:33)Caused by: java.io.IOException: com.google.protobuf.ServiceException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gaugeat org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:300)at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1564)at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1540)at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1337)at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1179)at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1136)at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:977)at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:74)at org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:125)at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)... 7 moreCaused by: com.google.protobuf.ServiceException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gaugeat org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1709)at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1750)at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:31612)at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1560)... 15 moreCaused by: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gaugeat org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1689)... 18 moreCaused by: java.lang.ClassNotFoundException: com.yammer.metrics.core.Gaugeat java.net.URLClassLoader.findClass(URLClassLoader.java:381)at java.lang.ClassLoader.loadClass(ClassLoader.java:424)at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)at java.lang.ClassLoader.loadClass(ClassLoader.java:357)... 19 more

发现是找不到包的异常,因为我之前都是按照网上很多帖子去导的jar包,很多帖子写的依赖包都大同小异,没什么差别,后来通过大量搜索,找到了问题,是程序缺少了一个叫:“metrics-core-2.2.0.jar”的包(此jar包可在$HBASE_HOME/lib下找到),build path该包,在运行程序,运行成功:

master:60000log4j:WARN No appenders could be found for logger (org.apache.hadoop.security.Groups).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.keyvalues={1/personinfo:name/1462411628085/Put/vlen=6/mvcc=0, 1/personinfo:sex/1462411628085/Put/vlen=3/mvcc=0}keyvalues={2/personinfo:name/1462411628085/Put/vlen=6/mvcc=0, 2/personinfo:sex/1462411628085/Put/vlen=3/mvcc=0}keyvalues={3/personinfo:name/1462411625239/Put/vlen=6/mvcc=0}keyvalues={4/personinfo:name/1462411623526/Put/vlen=6/mvcc=0, 4/personinfo:sex/1462411623526/Put/vlen=3/mvcc=0}keyvalues={5/personinfo:name/1462411628403/Put/vlen=6/mvcc=0, 5/personinfo:sex/1462411628403/Put/vlen=3/mvcc=0}keyvalues={6/personinfo:name/1462411628403/Put/vlen=6/mvcc=0}





0 0
原创粉丝点击