HBase源码分析 -- HBase Region 拆分(split)

来源:互联网 发布:零基础自学编程 编辑:程序博客网 时间:2024/09/21 06:17

代码版本:hbase-1.2.6

工程:hbase-server

类:org.apache.hadoop.hbase.regionserver.HRegion


需要解决的问题:

1、什么时候触发拆分?

2、拆分的策略是什么?



1、判断是否需要切分

方法: checkSplit

返回值: splitpoint

做了一些判断后,其实是调用:

byte[] ret = splitPolicy.getSplitPoint();

2、切分策略

org.apache.hadoop.hbase.regionserver.RegionSplitPolicy

/**   * @return the key at which the region should be split, or null   * if it cannot be split. This will only be called if shouldSplit   * previously returned true.   */  protected byte[] getSplitPoint() {    byte[] explicitSplitPoint = this.region.getExplicitSplitPoint();    if (explicitSplitPoint != null) {      return explicitSplitPoint;    }    List<Store> stores = region.getStores();    byte[] splitPointFromLargestStore = null;    long largestStoreSize = 0;    for (Store s : stores) {      byte[] splitPoint = s.getSplitPoint();      long storeSize = s.getSize();      if (splitPoint != null && largestStoreSize < storeSize) {        splitPointFromLargestStore = splitPoint;        largestStoreSize = storeSize;      }    }    return splitPointFromLargestStore;  }

从上边代码看如果explicitSplitPoint不为空,则使用这个,再往上查是forceSplit赋值的

如果explicitSplitPoint为空,则region.getStores() ,根据storeSize找到splitPoint

调用的是HStoregetSplitPoint方法:

@Override  public byte[] getSplitPoint() {    this.lock.readLock().lock();    try {      // Should already be enforced by the split policy!      assert !this.getRegionInfo().isMetaRegion();      // Not split-able if we find a reference store file present in the store.      if (hasReferences()) {        return null;      }      return this.storeEngine.getStoreFileManager().getSplitPoint();    } catch(IOException e) {      LOG.warn("Failed getting store size for " + this, e);    } finally {      this.lock.readLock().unlock();    }    return null;  }

DefaultStoreFileManager

@Override  public final byte[] getSplitPoint() throws IOException {    if (this.storefiles.isEmpty()) {      return null;    }    return StoreUtils.getLargestFile(this.storefiles).getFileSplitPoint(this.kvComparator);  }

然后到SotreFile的:

 /**   * Gets the approximate mid-point of this file that is optimal for use in splitting it.   * @param comparator Comparator used to compare KVs.   * @return The split point row, or null if splitting is not possible, or reader is null.   */  @SuppressWarnings("deprecation")  byte[] getFileSplitPoint(KVComparator comparator) throws IOException {    if (this.reader == null) {      LOG.warn("Storefile " + this + " Reader is null; cannot get split point");      return null;    }    // Get first, last, and mid keys.  Midkey is the key that starts block    // in middle of hfile.  Has column and timestamp.  Need to return just    // the row we want to split on as midkey.    byte [] midkey = this.reader.midkey();    if (midkey != null) {      KeyValue mk = KeyValue.createKeyValueFromKey(midkey, 0, midkey.length);      byte [] fk = this.reader.getFirstKey();      KeyValue firstKey = KeyValue.createKeyValueFromKey(fk, 0, fk.length);      byte [] lk = this.reader.getLastKey();      KeyValue lastKey = KeyValue.createKeyValueFromKey(lk, 0, lk.length);      // if the midkey is the same as the first or last keys, we cannot (ever) split this region.      if (comparator.compareRows(mk, firstKey) == 0 || comparator.compareRows(mk, lastKey) == 0) {        if (LOG.isDebugEnabled()) {          LOG.debug("cannot split because midkey is the same as first or last row");        }        return null;      }      return mk.getRow();    }    return null;  }




原创粉丝点击