DataXceiverServer

来源:互联网 发布:淘宝怎样添加多个客服 编辑:程序博客网 时间:2024/05/30 07:12

       DataXceiverServer类是DataNode的辅助类,它最主要是用来实现客户端或其他DataNode与当前节点通信,并负责接收/发送数据块。在DataNode的run方法中,有dataXceiverServer.start();这个类的创建是为了监听来自客户端或其他数据节点的请求。 它的实现通信是用jdk本身的ServerSocket。

    这个类有两个重要的成员变量,ServerSocket ss,DataNode datanode,ss是datanode负责接受socket连接的对象。  BlockBalanceThrottler balanceThrottler,带宽节流器,用于协调块运输所耗费的带宽。一个块传输过程中,如果已经使用了超过预期的带宽就令其等待wait一段时间,使得不会因为某个块传输而带宽耗尽。
    通信用的方式是ServerSocket,并且采用线程的方式,因此实现类Runnable接口。

 

public void run() {    while (datanode.shouldRun) {      try {      //侦听并接受来自客户端或其他服务器的连接请求,ss为执行当前方法的数据节点        Socket s = ss.accept();        s.setTcpNoDelay(true); //不延迟        new Daemon(datanode.threadGroup,             new DataXceiver(s, datanode, this)).start();      } catch (SocketTimeoutException ignored) {        // wake up to see if should continue to run      } catch (AsynchronousCloseException ace) {          LOG.warn(datanode.dnRegistration + ":DataXceiveServer:"                  + StringUtils.stringifyException(ace));          datanode.shouldRun = false;      } catch (IOException ie) {        LOG.warn(datanode.dnRegistration + ":DataXceiveServer: IOException due to:"                                 + StringUtils.stringifyException(ie));      } catch (Throwable te) {        LOG.error(datanode.dnRegistration + ":DataXceiveServer: Exiting due to:"                                  + StringUtils.stringifyException(te));        datanode.shouldRun = false;      }    }    try {      ss.close();    } catch (IOException ie) {      LOG.warn(datanode.dnRegistration + ":DataXceiveServer: Close exception due to: "                               + StringUtils.stringifyException(ie));    }    LOG.info("Exiting DataXceiveServer");  }

      从上面的代码中,我们可以看到DataXceiverServer每接受一个socket连接就会启动一个DataXceiver线程处理这个socket,那么DataXceiverServer主要干接受任务和分配任务,而具体的事情让DataXceiver去做。同时DataXceiverServer控制进行的块传输请求数(同一时刻的传输数不能超过maxXceiverCount)和带宽耗费情况(块传输时带宽耗费带宽不能超过预定值BlockTransferThrottler.bytesPerPeriod)。系统关闭时,会关闭用于监听的连接的ServerSocket同时将DataXceiver所产生的线程关闭,使得DataXceiver因为出现错误而退出。      

      DataXceiver依赖两个类:BlockSender和BlockReceiver。既然DataXceiver是一个线程类,那么我们就重点看它的run方法:

public void run() {    DataInputStream in=null;     try {      in = new DataInputStream(          new BufferedInputStream(NetUtils.getInputStream(s),                                   SMALL_BUFFER_SIZE));      short version = in.readShort();      if ( version != DataTransferProtocol.DATA_TRANSFER_VERSION ) {        throw new IOException( "Version Mismatch" );      }      boolean local = s.getInetAddress().equals(s.getLocalAddress());      byte op = in.readByte();      // Make sure the xciver count is not exceeded      int curXceiverCount = datanode.getXceiverCount();      if (curXceiverCount > dataXceiverServer.maxXceiverCount) {        throw new IOException("xceiverCount " + curXceiverCount                              + " exceeds the limit of concurrent xcievers "                              + dataXceiverServer.maxXceiverCount);      }      long startTime = DataNode.now();      switch ( op ) {      //读数据块         case DataTransferProtocol.OP_READ_BLOCK:        readBlock( in );        datanode.myMetrics.addReadBlockOp(DataNode.now() - startTime);        if (local)          datanode.myMetrics.incrReadsFromLocalClient();        else          datanode.myMetrics.incrReadsFromRemoteClient();        break;       //写数据块        case DataTransferProtocol.OP_WRITE_BLOCK:        writeBlock( in );        datanode.myMetrics.addWriteBlockOp(DataNode.now() - startTime);        if (local)          datanode.myMetrics.incrWritesFromLocalClient();        else          datanode.myMetrics.incrWritesFromRemoteClient();        break;      //替换数据块         case DataTransferProtocol.OP_REPLACE_BLOCK: // for balancing purpose; send to a destination        replaceBlock(in);        datanode.myMetrics.addReplaceBlockOp(DataNode.now() - startTime);        break;      //拷贝数据块         case DataTransferProtocol.OP_COPY_BLOCK:            // for balancing purpose; send to a proxy source        copyBlock(in);        datanode.myMetrics.addCopyBlockOp(DataNode.now() - startTime);        break;       //读取数据块校验码         case DataTransferProtocol.OP_BLOCK_CHECKSUM: //get the checksum of a block        getBlockChecksum(in);        datanode.myMetrics.addBlockChecksumOp(DataNode.now() - startTime);        break;      default:        throw new IOException("Unknown opcode " + op + " in data stream");      }    } catch (Throwable t) {      LOG.error(datanode.dnRegistration + ":DataXceiver",t);    } finally {      LOG.debug(datanode.dnRegistration + ":Number of active connections is: "                               + datanode.getXceiverCount());      IOUtils.closeStream(in);      IOUtils.closeSocket(s);      dataXceiverServer.childSockets.remove(s);    }  }

        DataXceiver处理Client或DataNode的五种请求(DataTransferProtocol接口定义)。
原创粉丝点击