hbase中compact、split相应配置参数分析

1,
<property>
  <name>hbase.hregion.majorcompaction</name>
  <value>86400000</value>
  <description>The time (in miliseconds) between 'major' compactions of all
  HStoreFiles in a region.  Default: 1 day.
  Set to 0 to disable automated major compactions.
  </description>
</property>
说明：
合并所有HStoreFile(列簇文件夹下的文件),清理历史版本和删除记录
将value设置成0可以取消compact，可在hbase shell中通过命令：major_compact 'tablename' 来对某个进行手动compcat  


        
2,Column family   MemStore+Storefiles
说明：
  但是同一个table的所有Column Family的memstore都会同时split，同时flush 
  从写角度看一个table不要设置太多的cf，他们之间有太多的紧耦合性
  从读角度看，读取频繁的column放在一个具有较少column的cf中比较合适


3,
  <property>
    <name>hfile.block.cache.size</name>
    <value>0.2</value>
    <description>
        Percentage of maximum heap (-Xmx setting) to allocate to block cache
        used by HFile/StoreFile. Default of 0.2 means allocate 20%.
        Set to 0 to disable.
    </description>
</property>  
说明：  
读cache大小，当memstore读取不到时，读取cache,cache读取不到时读取文件。
当cache超过设置值的85%？时就进行eviction
当清理到75%？时可查看LRU Stats等状态 
    
2011-10-19 11:37:46,328 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 121.78 MB of total=1.01 GB
2011-10-19 11:37:46,337 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=121.78 MB, total=913.24 MB, single=175.3 MB, multi=849.67 MB, memory=2.38 KB
2011-10-19 11:51:16,121 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=919.5 MB, free=298.07 MB, max=1.19 GB, blocks=14486, accesses=19167618, hits=19040864, hitRatio=99.33%%, cachingAccesses=19151550, cachingHits=19036235, cachingHitsRatio=99.39%%, evictions=52, evicted=100829, evictedPerRun=1939.019287109375
  
4,
<property>
    <name>hbase.hregion.max.filesize</name>
    <value>268435456</value>
    <description>
    Maximum HStoreFile size. If any one of a column families' HStoreFiles has
    grown to exceed this value, the hosting HRegion is split in two.
    Default: 256M.
    </description>
</property>
说明：HStoreFile文件到达多大阀值时进行split  


5,
  <property>
    <name>hbase.hregion.memstore.flush.size</name>
    <value>6435456</value>
    <description>
    </description>
  <property>     
说明：一个HResion中“所有”MemStore的值的“和”的阈值,当所有MemStore达到这个阀值时就进行flush
这是一种是有目的的flush，即指定flush某个HRegion中的MemStore


6,
  <property>
    <name>hbase.hstore.compactionThreshold</name>
    <value>Integer.MAX_VALUE</value>
    <description>
    If more than this number of HStoreFiles in any one HStore
    (one HStoreFile is written per flush of memstore) then a compaction
    is run to rewrite all HStoreFiles files as one.  Larger numbers
    put off compaction but when it runs, it takes longer to complete.
    During a compaction, updates cannot be flushed to disk.  Long
    compactions require memory sufficient to carry the logging of
    all updates across the duration of the compaction.
    If too large, clients timeout during compaction.
    </description>
  </property>
说明：当每个hstore中的hstorefiles达到几个时进行文件合并，设置成Integer.MAX_VALUE即不进行合并  
 这是一种是无目的的flush，扫描所有HRegion
  
  7
   <property>
    <name>hbase.hstore.blockingStoreFiles</name>
    <value>Integer.MAX_VALUE</value>
    <description>
    If more than this number of StoreFiles in any one Store
    (one StoreFile is written per flush of MemStore) then updates are
    blocked for this HRegion until a compaction is completed, or
    until hbase.hstore.blockingWaitTime has been exceeded.
    </description>
  </property>
  说明：任何一个HStore（列簇）的文件数大于该阈值时，该HRegion的update会被阻塞，直到过多的store文件在compact中被合并或者超时