HRegionServer Flush操作源码分析

Flush操作是将HBase中的数据存到硬盘上的过程，具体的flush的流程图如下，本文主要简要分析flush的过程相关源码。

Flush 任务提交

每当HRegion完成数据插入的操作的时候，就会进行检查此时是否需要进行一次flush,flush是将HRegion缓存的数据存储到磁盘的过程：

long addedSize = doMiniBatchMutation(batchOp);long newSize = this.addAndGetGlobalMemstoreSize(addedSize);if (isFlushSize(newSize)) {    requestFlush();}

本文主要分析flush的过程以及涉及到得相关数据结构，在requestFlush内部调用：
this.rsServices.getFlushRequester().requestFlush(this);实际是调用了MemStoreFlusher具体执行flush的操作:

  public void requestFlush(HRegion r) {    synchronized (regionsInQueue) {      if (!regionsInQueue.containsKey(r)) {        // This entry has no delay so it will be added at the top of the flush        // queue.  It'll come out near immediately.        FlushRegionEntry fqe = new FlushRegionEntry(r);        this.regionsInQueue.put(r, fqe);        this.flushQueue.add(fqe);      }    }  }

MemStoreFlushRequeter有两个数据结构管理者需要flush的任务，private BlockingQueue<FlushQueueEntry>flushQueue Map<HRegion, FlushRegionEntry> regionsInQueue flushQueue相当于需要flush的工作队列，而regionsInQueue则是于保存已经在队列中的region的信息，上面的一段代码表示当当前请求flush的region没有记录在flushQueue中的时候则加入。其中FlushRegionEntry是一个flushQueue的单元数据结构
到这里flush request的请求就提交结束了，接下来等待MemStore中的FlushHander线程取出region并执行flush的任务。

Flush的任务执行前期准备

1.FlushHandler从flushQueue中取出FlushRegionEntry并执行
flushRegion(final FlushRegionEntry fqe)
这里首先判断当前region中是否含有过多的storefile的文件，如果是的话，需要首先进行storefile的合并操作(这里有必要解释一下HRegion中的数据组织)，然后重新加入队列，否则的话直接对region执行flush操作：

isTooManyStoreFiles(region)this.server.compactSplitThread.requestSystemCompaction(                  region, Thread.currentThread().getName());                        this.flushQueue.add(fqe.requeue(this.blockingWaitTime / 100));elsereturn flushRegion(region, false);

2.flushRegion函数内部的主要执行逻辑如下，首先notifyFlushRequest只是进行一些flush线程数量的统计，region.flashcache具体负责flush的工作。执行完之后会根据返回值进行相关的辅助操作

 notifyFlushRequest(region, emergencyFlush); HRegion.FlushResult flushResult = region.flushcache(); boolean shouldCompact = flushResult.isCompactionNeeded();      // We just want to check the size boolean shouldSplit = region.checkSplit() != null; if (shouldSplit) {    this.server.compactSplitThread.requestSplit(region); } else if (shouldCompact) {     server.compactSplitThread.requestSystemCompaction(            region, Thread.currentThread().getName()); }if (flushResult.isFlushSucceeded()) {   long endTime = EnvironmentEdgeManager.currentTime();   server.metricsRegionServer.updateFlushTime(endTime - startTime);}

Flush的任务执行过程

flushcahe内部调用 FlushResult fs = internalFlushcache(status);实际执行flush操作，StoreFlushContext的实现为StoreFlusherImpl，为每个HStore建一个StoreFlusherImpl，它为对应的HStore执行着具体非flush的操作。flush的具体实现包括三个步骤：
1.快照

 public void prepare() {      this.snapshot = memstore.snapshot();      this.cacheFlushCount = snapshot.getCellsCount();      this.cacheFlushSize = snapshot.getSize();      committedFiles = new ArrayList<Path>(1);    }

2.将memestore中的数据写入.tmp文件中

   public void flushCache(MonitoredTask status) throws IOException {      tempFiles = HStore.this.flushCache(cacheFlushSeqNum, snapshot, status);    }

3.将.tmp文件写入对应的cf下面的对应的文件中去，并用StoreFile保存相应的HFile的文件信息

    public boolean commit(MonitoredTask status) throws IOException {      if (this.tempFiles == null || this.tempFiles.isEmpty()) {        return false;      }      List<StoreFile> storeFiles = new ArrayList<StoreFile>(this.tempFiles.size());      for (Path storeFilePath : tempFiles) {        try {          storeFiles.add(HStore.this.commitFile(storeFilePath, cacheFlushSeqNum, status));        } catch (IOException ex) {          LOG.error("Failed to commit store file " + storeFilePath, ex);          // Try to delete the files we have committed before.          for (StoreFile sf : storeFiles) {            Path pathToDelete = sf.getPath();            try {              sf.deleteReader();            } catch (IOException deleteEx) {              LOG.fatal("Failed to delete store file we committed, halting " + pathToDelete, ex);              Runtime.getRuntime().halt(1);            }          }          throw new IOException("Failed to commit the flush", ex);        }      }      for (StoreFile sf : storeFiles) {        if (HStore.this.getCoprocessorHost() != null) {          HStore.this.getCoprocessorHost().postFlush(HStore.this, sf);        }        committedFiles.add(sf.getPath());      }      HStore.this.flushedCellsCount += cacheFlushCount;      HStore.this.flushedCellsSize += cacheFlushSize;      // Add new file to store files.  Clear snapshot too while we have the Store write lock.      return HStore.this.updateStorefiles(storeFiles, snapshot.getId());    }

至此HBase的flush的操作过程就完成了。