HBase源码分析 -- HBase Region 拆分(split)
来源:互联网 发布:零基础自学编程 编辑:程序博客网 时间:2024/09/21 06:17
代码版本:hbase-1.2.6
工程:hbase-server
类:org.apache.hadoop.hbase.regionserver.HRegion
需要解决的问题:
1、什么时候触发拆分?
2、拆分的策略是什么?
1、判断是否需要切分
方法: checkSplit
返回值: splitpoint
做了一些判断后,其实是调用:
byte[] ret = splitPolicy.getSplitPoint();
2、切分策略
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy
/** * @return the key at which the region should be split, or null * if it cannot be split. This will only be called if shouldSplit * previously returned true. */ protected byte[] getSplitPoint() { byte[] explicitSplitPoint = this.region.getExplicitSplitPoint(); if (explicitSplitPoint != null) { return explicitSplitPoint; } List<Store> stores = region.getStores(); byte[] splitPointFromLargestStore = null; long largestStoreSize = 0; for (Store s : stores) { byte[] splitPoint = s.getSplitPoint(); long storeSize = s.getSize(); if (splitPoint != null && largestStoreSize < storeSize) { splitPointFromLargestStore = splitPoint; largestStoreSize = storeSize; } } return splitPointFromLargestStore; }
从上边代码看如果explicitSplitPoint不为空,则使用这个,再往上查是forceSplit赋值的
如果explicitSplitPoint为空,则region.getStores() ,根据storeSize找到splitPoint
调用的是HStore的getSplitPoint方法:
@Override public byte[] getSplitPoint() { this.lock.readLock().lock(); try { // Should already be enforced by the split policy! assert !this.getRegionInfo().isMetaRegion(); // Not split-able if we find a reference store file present in the store. if (hasReferences()) { return null; } return this.storeEngine.getStoreFileManager().getSplitPoint(); } catch(IOException e) { LOG.warn("Failed getting store size for " + this, e); } finally { this.lock.readLock().unlock(); } return null; }
DefaultStoreFileManager
@Override public final byte[] getSplitPoint() throws IOException { if (this.storefiles.isEmpty()) { return null; } return StoreUtils.getLargestFile(this.storefiles).getFileSplitPoint(this.kvComparator); }
然后到SotreFile的:
/** * Gets the approximate mid-point of this file that is optimal for use in splitting it. * @param comparator Comparator used to compare KVs. * @return The split point row, or null if splitting is not possible, or reader is null. */ @SuppressWarnings("deprecation") byte[] getFileSplitPoint(KVComparator comparator) throws IOException { if (this.reader == null) { LOG.warn("Storefile " + this + " Reader is null; cannot get split point"); return null; } // Get first, last, and mid keys. Midkey is the key that starts block // in middle of hfile. Has column and timestamp. Need to return just // the row we want to split on as midkey. byte [] midkey = this.reader.midkey(); if (midkey != null) { KeyValue mk = KeyValue.createKeyValueFromKey(midkey, 0, midkey.length); byte [] fk = this.reader.getFirstKey(); KeyValue firstKey = KeyValue.createKeyValueFromKey(fk, 0, fk.length); byte [] lk = this.reader.getLastKey(); KeyValue lastKey = KeyValue.createKeyValueFromKey(lk, 0, lk.length); // if the midkey is the same as the first or last keys, we cannot (ever) split this region. if (comparator.compareRows(mk, firstKey) == 0 || comparator.compareRows(mk, lastKey) == 0) { if (LOG.isDebugEnabled()) { LOG.debug("cannot split because midkey is the same as first or last row"); } return null; } return mk.getRow(); } return null; }
阅读全文
0 0
- HBase源码分析 -- HBase Region 拆分(split)
- hbase region split 源码分析
- HBase region预拆分
- Apache HBase region拆分
- hbase region split策略
- HBase Region split 策略
- hbase region 手动 split
- HBase Split流程源码分析
- hbase源码分析-是否split
- HBase笔记:Region拆分策略
- HBase源码分析之Region定位
- HBase源码分析之Region合并merge
- HBase源码分析之Region上线
- HBase源码分析之Region下线
- HBASE-region的SPLIT策略
- HBase 0.94.8 split 源码分析
- HBase Region的路由、分配与拆分
- Hbase Region的拆分和合并
- 【MyEclipse】代码调试
- [线性基 树链剖分 线段树 || ST表 || 点分治] BZOJ 4568 [Scoi2016]幸运数字
- 对acm一点点改变性的认识
- JS实现归并排序
- ubuntu下安装python3.5(系统默认自带的是python2.7)
- HBase源码分析 -- HBase Region 拆分(split)
- Java垃圾回收机制
- (转载)性能测试
- C++中的集合和字典(unordered_set, unordered_map)
- leetcode:去除石头
- 网络IO之阻塞、非阻塞、同步、异步总结
- 第二场个人训练赛
- Spring-Data-JPA概述
- 最大子列和问题