程序博客网 > 三工序雕刻机编程软件

HBase Client Get 源码分析

来源：互联网 发布：三工序雕刻机编程软件 编辑：程序博客网 时间：2024/05/18 16:18

1 入口

HTable.java

private Result get(Get get, final boolean checkExistenceOnly) throws IOException {    // if we are changing settings to the get, clone it.    if (get.isCheckExistenceOnly() != checkExistenceOnly || get.getConsistency() == null) {      get = ReflectionUtils.newInstance(get.getClass(), get);      get.setCheckExistenceOnly(checkExistenceOnly);      if (get.getConsistency() == null){        get.setConsistency(defaultConsistency);      }    }    if (get.getConsistency() == Consistency.STRONG) {      // Good old call.      final Get getReq = get;      RegionServerCallable<Result> callable = new RegionServerCallable<Result>(this.connection,          getName(), get.getRow()) {        @Override        public Result call(int callTimeout) throws IOException {          ClientProtos.GetRequest request =            RequestConverter.buildGetRequest(getLocation().getRegionInfo().getRegionName(), getReq);          PayloadCarryingRpcController controller = rpcControllerFactory.newController();          controller.setPriority(tableName);          controller.setCallTimeout(callTimeout);          try {            ClientProtos.GetResponse response = getStub().get(controller, request);            if (response == null) return null;            return ProtobufUtil.toResult(response.getResult());          } catch (ServiceException se) {            throw ProtobufUtil.getRemoteException(se);          }        }      };      return rpcCallerFactory.<Result>newCaller().callWithRetries(callable, this.operationTimeout);    }

这里注意两个地方：

1.1 callable的类型是RegionServerCallable<Result>, 同时覆盖了call函数，这里后面需要回来看。

1.2 最后一行callWithRetries


2 继续跟踪代码

public T callWithRetries(RetryingCallable<T> callable, int callTimeout)  throws IOException, RuntimeException {    List<RetriesExhaustedException.ThrowableWithExtraContext> exceptions =      new ArrayList<RetriesExhaustedException.ThrowableWithExtraContext>();    this.globalStartTime = EnvironmentEdgeManager.currentTime();    context.clear();    for (int tries = 0;; tries++) {      long expectedSleep;      try {        callable.prepare(tries != 0); // if called with false, check table status on ZK        interceptor.intercept(context.prepare(callable, tries));        return callable.call(getRemainingTime(callTimeout));      } catch (PreemptiveFastFailException e) {        throw e;      } catch (Throwable t) {        ExceptionUtil.rethrowIfInterrupt(t);        if (tries > startLogErrorsCnt) {          LOG.info("Call exception, tries=" + tries + ", retries=" + retries + ", started=" +              (EnvironmentEdgeManager.currentTime() - this.globalStartTime) + " ms ago, "              + "cancelled=" + cancelled.get() + ", msg="              + callable.getExceptionMessageAdditionalDetail());        }        // translateException throws exception when should not retry: i.e. when request is bad.        interceptor.handleFailure(context, t);        t = translateException(t);        callable.throwable(t, retries != 1);        RetriesExhaustedException.ThrowableWithExtraContext qt =            new RetriesExhaustedException.ThrowableWithExtraContext(t,                EnvironmentEdgeManager.currentTime(), toString());        exceptions.add(qt);        if (tries >= retries - 1) {          throw new RetriesExhaustedException(tries, exceptions);        }        // If the server is dead, we need to wait a little before retrying, to give        //  a chance to the regions to be        // tries hasn't been bumped up yet so we use "tries + 1" to get right pause time        expectedSleep = callable.sleep(pause, tries + 1);        // If, after the planned sleep, there won't be enough time left, we stop now.        long duration = singleCallDuration(expectedSleep);        if (duration > callTimeout) {          String msg = "callTimeout=" + callTimeout + ", callDuration=" + duration +              ": " + callable.getExceptionMessageAdditionalDetail();          throw (SocketTimeoutException)(new SocketTimeoutException(msg).initCause(t));        }      } finally {        interceptor.updateFailureInfo(context);      }      try {        if (expectedSleep > 0) {          synchronized (cancelled) {            if (cancelled.get()) return null;            cancelled.wait(expectedSleep);          }        }        if (cancelled.get()) return null;      } catch (InterruptedException e) {        throw new InterruptedIOException("Interrupted after " + tries + " tries  on " + retries);      }    }  }

重点观察两个地方：

2.1 callable.prepare, 这里要注意callable的类型，见上面一步, 

2.2 callable.call 


2.1 RegionServerCallable.java

  public void prepare(final boolean reload) throws IOException {    try (RegionLocator regionLocator = connection.getRegionLocator(tableName)) {      this.location = regionLocator.getRegionLocation(row, reload);    }    if (this.location == null) {      throw new IOException("Failed to find location, tableName=" + tableName +        ", row=" + Bytes.toString(row) + ", reload=" + reload);    }    setStub(getConnection().getClient(this.location.getServerName()));  }

2.1.1 setStub函数会找到合适的RPC client， 注意返回的client的类型

HConnection.java

ClientService.BlockingInterface getClient(final ServerName serverName) throws IOException;


2.2 call函数是被覆盖的，所以观察 1 中的call函数


ClientProtos.GetResponse response = getStub().get(controller, request);

继续跟踪

ClientProtos.java

    public interface BlockingInterface {      public org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse get(          com.google.protobuf.RpcController controller,          org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetRequest request)          throws com.google.protobuf.ServiceException;

这里的接口方法的实现类

ClientProtos.java  －－》 BlockingStub

    private static final class BlockingStub implements BlockingInterface {      private BlockingStub(com.google.protobuf.BlockingRpcChannel channel) {        this.channel = channel;      }      private final com.google.protobuf.BlockingRpcChannel channel;      public org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse get(          com.google.protobuf.RpcController controller,          org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetRequest request)          throws com.google.protobuf.ServiceException {        return (org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse) channel.callBlockingMethod(          getDescriptor().getMethods().get(0),          controller,          request,          org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse.getDefaultInstance());      }

这里的重点是channel.callBlockingMethod


public interface BlockingRpcChannel {    Message callBlockingMethod(MethodDescriptor var1, RpcController var2, Message var3, Message var4) throws ServiceException;}

实现类

AbstractRpcClient.java

public static class BlockingRpcChannelImplementation implements BlockingRpcChannel {    private final InetSocketAddress isa;    private final AbstractRpcClient rpcClient;    private final User ticket;    private final int channelOperationTimeout;    /**     * @param channelOperationTimeout - the default timeout when no timeout is given     */    protected BlockingRpcChannelImplementation(final AbstractRpcClient rpcClient,        final ServerName sn, final User ticket, int channelOperationTimeout) {      this.isa = new InetSocketAddress(sn.getHostname(), sn.getPort());      this.rpcClient = rpcClient;      this.ticket = ticket;      this.channelOperationTimeout = channelOperationTimeout;    }    @Override    public Message callBlockingMethod(Descriptors.MethodDescriptor md, RpcController controller,        Message param, Message returnType) throws ServiceException {      PayloadCarryingRpcController pcrc;      if (controller != null) {        pcrc = (PayloadCarryingRpcController) controller;        if (!pcrc.hasCallTimeout()) {          pcrc.setCallTimeout(channelOperationTimeout);        }      } else {        pcrc = new PayloadCarryingRpcController();        pcrc.setCallTimeout(channelOperationTimeout);      }      return this.rpcClient.callBlockingMethod(md, pcrc, param, returnType, this.ticket, this.isa);    }  }

rpcClient.callBlockingMethod的实现

Message callBlockingMethod(Descriptors.MethodDescriptor md, PayloadCarryingRpcController pcrc,      Message param, Message returnType, final User ticket, final InetSocketAddress isa)      throws ServiceException {    if (pcrc == null) {      pcrc = new PayloadCarryingRpcController();    }    long startTime = 0;    if (LOG.isTraceEnabled()) {      startTime = EnvironmentEdgeManager.currentTime();    }    Pair<Message, CellScanner> val;    try {      val = call(pcrc, md, param, returnType, ticket, isa);      // Shove the results into controller so can be carried across the proxy/pb service void.      pcrc.setCellScanner(val.getSecond());      if (LOG.isTraceEnabled()) {        long callTime = EnvironmentEdgeManager.currentTime() - startTime;        LOG.trace("Call: " + md.getName() + ", callTime: " + callTime + "ms");      }      return val.getFirst();    } catch (Throwable e) {      throw new ServiceException(e);    }  }

call函数

  protected abstract Pair<Message, CellScanner> call(PayloadCarryingRpcController pcrc,      Descriptors.MethodDescriptor md, Message param, Message returnType, User ticket,      InetSocketAddress isa) throws IOException, InterruptedException;

这里的call函数有两个实现

AsyncRpcClient.java

RpcClientImpl.java


目前Get请求使用的是RpcClientImpl


  protected Pair<Message, CellScanner> call(PayloadCarryingRpcController pcrc, MethodDescriptor md,      Message param, Message returnType, User ticket, InetSocketAddress addr)      throws IOException, InterruptedException {    if (pcrc == null) {      pcrc = new PayloadCarryingRpcController();    }    CellScanner cells = pcrc.cellScanner();    final Call call = new Call(this.callIdCnt.getAndIncrement(), md, param, cells, returnType,        pcrc.getCallTimeout());    final Connection connection = getConnection(ticket, call, addr);    final CallFuture cts;    if (connection.callSender != null) {      cts = connection.callSender.sendCall(call, pcrc.getPriority(), Trace.currentSpan());        pcrc.notifyOnCancel(new RpcCallback<Object>() {          @Override          public void run(Object parameter) {            connection.callSender.remove(cts);          }        });        if (pcrc.isCanceled()) {          // To finish if the call was cancelled before we set the notification (race condition)          call.callComplete();          return new Pair<Message, CellScanner>(call.response, call.cells);        }    } else {      cts = null;      connection.tracedWriteRequest(call, pcrc.getPriority(), Trace.currentSpan());    }    while (!call.done) {      if (call.checkAndSetTimeout()) {        if (cts != null) connection.callSender.remove(cts);        break;      }      if (connection.shouldCloseConnection.get()) {        throw new ConnectionClosingException("Call id=" + call.id +            " on server " + addr + " aborted: connection is closing");      }      try {        synchronized (call) {          if (call.done) break;          call.wait(Math.min(call.remainingTime(), 1000) + 1);        }      } catch (InterruptedException e) {        call.setException(new InterruptedIOException());        if (cts != null) connection.callSender.remove(cts);        throw e;      }    }    if (call.error != null) {      if (call.error instanceof RemoteException) {        call.error.fillInStackTrace();        throw call.error;      }      // local exception      throw wrapException(addr, call.error);    }    return new Pair<Message, CellScanner>(call.response, call.cells);  }

如果Get请求超时20分钟，call.checkAndSetTimeout函数会抛出exception


java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1209076: row '/dict/BPE_CUSTOMER_CONTACT.CUSTOMER_CONTACT_DATA_SUMMARY_V2/CONTACT_AREA/b0372936-96c6-45e4-9927-bce9060d55db.dict' on table 'tableX' at region=tableX,/cube_statistics/v_clsfd_ga_trffc_src_7D_v1/1fb0f3d2-a3ce-4027-990c-3549b2594006.seq,1451389955012.9a10b3032bb086d76f6037d17c16c3e2., hostname=phx9dn0486.phx.xx.com,60020,1476310334343, seqNum=35251· Caused by: java.io.IOException: Call to 
phx9dn0486.phx.xx.com/10.115.11.16:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=418960885, waitTime=1200001, operationTimeout=1200000 expired.
· at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1262)· at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1230)· at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)· at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)· at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:32627)· at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:881)· at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:872)· at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)· ... 19 more· Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=418960885, waitTime=1200001, operationTimeout=1200000 expired.· at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)· at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1204)· ... 25 more
上面RpcClientImpl的call方法中真正发生作用的是下面这句话。这里callSender其实是一个Thread子类，sendcall只是把call放到其中的blockingqueue中，
cts = connection.callSender.sendCall(call, pcrc.getPriority(), Trace.currentSpan());

如下只是offer到queue里面。
private class CallSender extends Thread implements Closeable {      protected final BlockingQueue<CallFuture> callsToWrite;      public CallFuture sendCall(Call call, int priority, Span span)          throws InterruptedException, IOException {        CallFuture cts = new CallFuture(call, priority, span);        if (!callsToWrite.offer(cts)) {          throw new IOException("Can't add the call " + call.id +              " to the write queue. callsToWrite.size()=" + callsToWrite.size());        }        checkIsOpen(); // We check after the put, to be sure that the call we added won't stay                       //  in the list while the cleanup was already done.        return cts;      }

查看一下这个blockingqueue的数据是如何被执行的，如下，可以开始看到根底层socket相关的线索。
      public void run() {        while (!shouldCloseConnection.get()) {          CallFuture cts = null;          try {            cts = callsToWrite.take();          } catch (InterruptedException e) {            markClosed(new InterruptedIOException());          }          if (cts == null || cts == CallFuture.DEATH_PILL) {            assert shouldCloseConnection.get();            break;          }          if (cts.call.done) {            continue;          }          if (cts.call.checkAndSetTimeout()) {            continue;          }          try {            Connection.this.tracedWriteRequest(cts.call, cts.priority, cts.span);          } catch (IOException e) {            if (LOG.isDebugEnabled()) {              LOG.debug("call write error for call #" + cts.call.id                + ", message =" + e.getMessage());            }            cts.call.setException(e);            markClosed(e);          }        }        cleanup();      }


tracedWriteRequest

  private void writeRequest(Call call, final int priority, Span span) throws IOException {      RequestHeader.Builder builder = RequestHeader.newBuilder();      builder.setCallId(call.id);      if (span != null) {        builder.setTraceInfo(            RPCTInfo.newBuilder().setParentId(span.getSpanId()).setTraceId(span.getTraceId()));      }      builder.setMethodName(call.md.getName());      builder.setRequestParam(call.param != null);      ByteBuffer cellBlock = ipcUtil.buildCellBlock(this.codec, this.compressor, call.cells);      if (cellBlock != null) {        CellBlockMeta.Builder cellBlockBuilder = CellBlockMeta.newBuilder();        cellBlockBuilder.setLength(cellBlock.limit());        builder.setCellBlockMeta(cellBlockBuilder.build());      }      // Only pass priority if there one.  Let zero be same as no priority.      if (priority != 0) builder.setPriority(priority);      RequestHeader header = builder.build();      setupIOstreams();

根据HBASE 官方的jira可以知道，1.1版本的hbase中，默认依然采用的是rpcclientimpl版本
AbstractRpcClient.java的子类有两个
asyncrpcclient 
rpcclientimpl
https://issues.apache.org/jira/browse/HBASE-12684
                                             
        0        0           
	
				
		









 三工序雕刻机编程软件
    		三工序雕刻机编程软件



原创粉丝点击



		

热门IT博客


剑灵人族刺客捏脸数据剑灵人族刺客捏脸数据




双休的网络销售靠谱吗双休的网络销售靠谱吗




部落冲突毒药法术数据




程序员名词




java设计模式详解pdf




双肩包推荐 不貴 知乎




网络语买买提的意思




淘宝拍衣服用什么相机




ds数据精灵下载




er软件




旅游入境数据统计




鲱鱼罐头淘宝评价




sqlserver查询分析器




宿舍收纳整理知乎




淘宝网道具道士服




大数据运用




阿里云大数据竞赛




表格里数据自动更新




offcf2007修复软件




变速箱 知乎



		

热门问题
    老师的惩罚
    人脸识别
    我在镇武司摸鱼那些年
    重生之率土为王
    我在大康的咸鱼生活
    盘龙之生命进化
    天生仙种
    凡人之先天五行
    春回大明朝
    姑娘不必设防，我是瞎子
    贵州省毕节市黔西县
    毕节市有多少个县
    毕节市黔西县
    毕节市火车站
    毕节市属于哪个省
    毕节市有几个县
    毕节市人民法院
    毕节市简介
    毕节市区号
    毕节市人民医院
    毕节市第三人民医院
    毕节市是哪个省
    毕节市大方县
    贵州省毕节市区号
    贵州毕节市织金县
    贵州的市
    毕节2019高铁站开通时间
    毕节火车站通车时间
    毕节火车站通车时间12月30日
    毕节火车站时刻表查询
    知网毕设
    毕金仪个人资料
    毕鑫业
    毕铭鑫
    毕飞宇
    毕飞宇作品
    毕飞宇文集
    青衣毕飞宇
    毕马汇
    毕马威招聘
    毕马威中国
    毕马威网申答案
    毕马威校园招聘
    毕马威网申
    毕马威2018工资待遇
    毕马威2018校园招聘
    毕鹏
    毗卢寺
    比沙门天
    毗沙门天威娜
    毗湿奴


	










程序博客网，程序员的互联网技术博客家园。csdn论坛精品 msdn技术资料都在这里