hbase 原代码分析（11） WAL 写日志过程

来源：互联网发布：fx2da模块数据编辑：程序博客网时间：2024/06/08 05:58

在put和delete过程的时候都会有WAL。这一章主要分析这个过程。

重点：

1） WAL在创建region的时候创建。

2) WAL 存储在HDFS，主要类FSHLog

3）在put 和delete写入内存之前都会写入WAL，主要是防止机器崩溃时能恢复。

4）用户在协处理器里可以自己调用WALEdit。

写入过程

调用disrupter框架。

1）创建HRegion.java 类，具体创建过程可以参考第10节。

 public static HRegion createHRegion(final HRegionInfo info, final Path rootDir,
      final Path tableDir, final Configuration conf, final HTableDescriptor hTableDescriptor,
      final WAL wal, final boolean initialize, final boolean ignoreWAL)
  throws IOException {
    HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info);
    WAL effectiveWAL = wal;
    if (wal == null && !ignoreWAL) {
      Configuration confForWAL = new Configuration(conf);
      confForWAL.set(HConstants.HBASE_DIR, rootDir.toString());
      effectiveWAL = (new WALFactory(confForWAL,
          Collections.<WALActionsListener>singletonList(new MetricsWAL()),
          "hregion-" + RandomStringUtils.randomNumeric(8))).
            getWAL(info.getEncodedNameAsBytes(), info.getTable().getNamespace());
    }
  }

在工程类里面最重要的。这里的defaultProvider(DefaultWALProvider.class),

provider = getProvider(WAL_PROVIDER, DEFAULT_WAL_PROVIDER, listeners, null);

这个版本1.2.3和1.3.1 有点不同，这里以1.3.1为准

  @Override
  public WAL getWAL(final byte[] identifier, byte[] namespace) throws IOException {
    if (log == null) {
      // only lock when need to create wal, and need to lock since
      // creating hlog on fs is time consuming
      synchronized (walCreateLock) {
        if (log == null) {
          log = new FSHLog(FileSystem.get(conf), FSUtils.getRootDir(conf),
              getWALDirectoryName(factory.factoryId), HConstants.HREGION_OLDLOGDIR_NAME, conf,
              listeners, true, logPrefix,
              META_WAL_PROVIDER_ID.equals(providerId) ? META_WAL_PROVIDER_ID : null);
        }
      }
    }
    return log;
  }

所以Log 的目录在：有点混淆。

hdfs = root/WALs/WALFactory.encode/oldWALs/WALFactory/wal.

root=hbase

后缀：.meta 或者为null

这样HLog的文件位置，名字都有了，并且写入Region中。

这个FSHLog里面定义很多东西，最重要的。

 public FSHLog(final FileSystem fs, final Path rootDir, final String logDir,
      final String archiveDir, final Configuration conf,
      final List<WALActionsListener> listeners,
      final boolean failIfWALExists, final String prefix, final String suffix)
      throws IOException {
    //HLog文件写流   
    // rollWriter sets this.hdfs_out if it can.
    rollWriter();
  
    this.disruptor =
      new Disruptor<RingBufferTruck>(RingBufferTruck.EVENT_FACTORY, preallocatedEventCount,
        this.appendExecutor, ProducerType.MULTI, new BlockingWaitStrategy());
    this.disruptor.getRingBuffer().next();
    this.ringBufferEventHandler =
      new RingBufferEventHandler(conf.getInt("hbase.regionserver.hlog.syncer.count", 5),
        maxHandlersCount);
    this.disruptor.handleExceptionsWith(new RingBufferExceptionHandler());
    this.disruptor.handleEventsWith(new RingBufferEventHandler [] {this.ringBufferEventHandler});
    // Presize our map of SyncFutures by handler objects.
    this.syncFuturesByHandler = new ConcurrentHashMap<Thread, SyncFuture>(maxHandlersCount);
    this.disruptor.start();
  }

这个FSHLog 里面最要的几个东西

rollWriter（），定义了一个FSDataOutputStream这个可以往hlog写文件。 rollWriter 里实现了ProtobufLogWriter 类，在该类的init方法里定义了write 与文件的关系。

FSDataOutputStream nextHdfsOut = null;if (nextWriter instanceof ProtobufLogWriter) {  nextHdfsOut = ((ProtobufLogWriter)nextWriter).getStream();

disruptor这个是一个队列工具。采用生产者与消费者模式。当有时间append时，会调用RingBufferEventHandler的onEven方法。

RingBufferEventHandler new里面定义很多同步的runner 这个主要是写write过程。


    RingBufferEventHandler(final int syncRunnerCount, final int maxHandlersCount) {
      this.syncFutures = new SyncFuture[maxHandlersCount];
      this.syncRunners = new SyncRunner[syncRunnerCount];
      for (int i = 0; i < syncRunnerCount; i++) {
        this.syncRunners[i] = new SyncRunner("sync." + i, maxHandlersCount);
      }
    }

2）记录数据。

这个主要是put过程了。具体put过程可以参考第七章

主要是HRegin类，因为每一个region都有一个WAL。WALEdit 是处理Wal的工具。

put过程主要哪些地方要写WAL。

1）rowProcessor 的钩子，用户可以定义

2）preBatchMutate

3）写入store 内存时

4）MVCC

5）结束的钩子

这里最重要的写入内存之前

@Override
  public void processRowsWithLocks(RowProcessor<?,?> processor, long timeout,
      long nonceGroup, long nonce) throws IOException {
    //new
    WALEdit walEdit = new WALEdit();、
    // 1. Run pre-process hook
    processor.preProcess(this, walEdit);
    MultiVersionConcurrencyControl.WriteEntry writeEntry = null;
    WALKey walKey = null;
          // 5. Call the preBatchMutate hook
          processor.preBatchMutate(this, walEdit);
          long txid = 0;
          // 6. Append no sync
          if (!walEdit.isEmpty()) {
            // we use HLogKey here instead of WALKey directly to support legacy coprocessors.
            walKey = new HLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
              this.htableDescriptor.getTableName(), WALKey.NO_SEQUENCE_ID, now,
              processor.getClusterIds(), nonceGroup, nonce, mvcc);
            txid = this.wal.append(this.htableDescriptor, this.getRegionInfo(),
                walKey, walEdit, true);
          }
          if(walKey == null){
            // since we use wal sequence Id as mvcc, for SKIP_WAL changes we need a "faked" WALEdit
            // to get a sequence id assigned which is done by FSWALEntry#stampRegionSequenceId
            walKey = this.appendEmptyEdit(this.wal);
          }
          // 7. Start mvcc transaction
          writeEntry = walKey.getWriteEntry();
          mvccNum = walKey.getSequenceId();
          // 8. Apply to memstore
           Store store = getStore(cell);
           addedSize += store.add(cell);
          
          }
         // 11. Sync edit log
          if (txid != 0) {
            syncOrDefer(txid, getEffectiveDurability(processor.useDurability()));
          }
          walSyncSuccessful = true;
  
      } finally {
        if (!mutations.isEmpty() && !walSyncSuccessful) {
 
          if (writeEntry != null) {
            mvcc.complete(writeEntry);
            writeEntry = null;
          }
        }
        // 13. Roll mvcc forward
        if (writeEntry != null) {
          mvcc.completeAndWait(writeEntry);
        }
      }
      // 14. Run post-process hook
      processor.postProcess(this, walEdit, walSyncSuccessful);
    }
  }

这个里面都简化了。留下来与Wal 有关的。里面最重要的已经标红。

第一个。

walKey= newHLogKey(this.getRegionInfo().getEncodedNameAsBytes(),
this.htableDescriptor.getTableName(),WALKey.NO_SEQUENCE_ID, now,
processor.getClusterIds(), nonceGroup, nonce, mvcc);
//txid 好比快递号，实时查询状态。
txid= this.wal.append(this.htableDescriptor, this.getRegionInfo(),
walKey, walEdit, true);

第二个

syncOrDefer(txid, getEffectiveDurability(processor.useDurability()));

FSHlog类

  @Override
  public long append(final HTableDescriptor htd, final HRegionInfo hri, final WALKey key,
      final WALEdit edits, final boolean inMemstore) throws IOException {
    if (this.closed) throw new IOException("Cannot append; log is closed");
    // Make a trace scope for the append.  It is closed on other side of the ring buffer by the
    // single consuming thread.  Don't have to worry about it.
    TraceScope scope = Trace.startSpan("FSHLog.append");
    FSWALEntry entry = null;
    long sequence = this.disruptor.getRingBuffer().next();
    try {
        //获得车厢。
      RingBufferTruck truck = this.disruptor.getRingBuffer().get(sequence);
        //定义实体
      entry = new FSWALEntry(sequence, key, edits, htd, hri, inMemstore);
        //放入车厢
      truck.loadPayload(entry, scope.detach());
    } finally {
    //告诉车可以走了。
      this.disruptor.getRingBuffer().publish(sequence);
    }
    return sequence;
  }

  @Override 
    //这是一个同步方法，如果车厢已经空了，说明已经结束。否则等待。
  public void sync(long txid) throws IOException {
    if (this.highestSyncedSequence.get() >= txid){
      // Already sync'd.
    //车厢已经空了。
      return;
    }
    TraceScope scope = Trace.startSpan("FSHLog.sync");
    try {//这里会等待空。
      scope = Trace.continueSpan(publishSyncThenBlockOnCompletion(scope.detach()));
    } finally {
      assert scope == NullScope.INSTANCE || !scope.isDetached();
      scope.close();
    }
  }

这个RingBufferEventHandler类的onEven 方法：不解释了，太晚了。这个主要是将下面车厢里的东西拿出来，也就是收货了，然后写到write里面。最后调用SysRunner里的run（这个方法无线循环）方法。run里的write.syn() .写到HDFS。

@Override// We can set endOfBatch in the below method if at end of our this.syncFutures arraypublic void onEvent(final RingBufferTruck truck, final long sequence, boolean endOfBatch)throws Exception {  try {    if (truck.hasSyncFuturePayload()) {      this.syncFutures[this.syncFuturesCount++] = truck.unloadSyncFuturePayload();      // Force flush of syncs if we are carrying a full complement of syncFutures.      if (this.syncFuturesCount == this.syncFutures.length) endOfBatch = true;    } else if (truck.hasFSWALEntryPayload()) {      TraceScope scope = Trace.continueSpan(truck.unloadSpanPayload());      try {        FSWALEntry entry = truck.unloadFSWALEntryPayload();        if (this.exception != null) {          entry.stampRegionSequenceId();          // Return to keep processing events coming off the ringbuffer          return;        }        append(entry);      } catch (Exception e) {        // Failed append. Record the exception.        this.exception = e;        // invoking cleanupOutstandingSyncsOnException when append failed with exception,        // it will cleanup existing sync requests recorded in syncFutures but not offered to SyncRunner yet,        // so there won't be any sync future left over if no further truck published to disruptor.        cleanupOutstandingSyncsOnException(sequence,            this.exception instanceof DamagedWALException ? this.exception                : new DamagedWALException("On sync", this.exception));        // Return to keep processing events coming off the ringbuffer        return;      } finally {        assert scope == NullScope.INSTANCE || !scope.isDetached();        scope.close(); // append scope is complete      }    } else {      // What is this if not an append or sync. Fail all up to this!!!      cleanupOutstandingSyncsOnException(sequence,        new IllegalStateException("Neither append nor sync"));      // Return to keep processing.      return;    }    // TODO: Check size and if big go ahead and call a sync if we have enough data.    // This is a sync. If existing exception, fall through. Else look to see if batch.    if (this.exception == null) {      // If not a batch, return to consume more events from the ring buffer before proceeding;      // we want to get up a batch of syncs and appends before we go do a filesystem sync.      if (!endOfBatch || this.syncFuturesCount <= 0) return;      // syncRunnerIndex is bound to the range [0, Integer.MAX_INT - 1] as follows:      //   * The maximum value possible for syncRunners.length is Integer.MAX_INT      //   * syncRunnerIndex starts at 0 and is incremented only here      //   * after the increment, the value is bounded by the '%' operator to [0, syncRunners.length),      //     presuming the value was positive prior to the '%' operator.      //   * after being bound to [0, Integer.MAX_INT - 1], the new value is stored in syncRunnerIndex      //     ensuring that it can't grow without bound and overflow.      //   * note that the value after the increment must be positive, because the most it could have      //     been prior was Integer.MAX_INT - 1 and we only increment by 1.      this.syncRunnerIndex = (this.syncRunnerIndex + 1) % this.syncRunners.length;      try {        // Below expects that the offer 'transfers' responsibility for the outstanding syncs to        // the syncRunner. We should never get an exception in here.        this.syncRunners[this.syncRunnerIndex].offer(sequence, this.syncFutures,          this.syncFuturesCount);      } catch (Exception e) {        // Should NEVER get here.        requestLogRoll();        this.exception = new DamagedWALException("Failed offering sync", e);      }    }    // We may have picked up an exception above trying to offer sync    if (this.exception != null) {      cleanupOutstandingSyncsOnException(sequence,        this.exception instanceof DamagedWALException?          this.exception:          new DamagedWALException("On sync", this.exception));    }    attainSafePoint(sequence);    this.syncFuturesCount = 0;  } catch (Throwable t) {    LOG.error("UNEXPECTED!!! syncFutures.length=" + this.syncFutures.length, t);  }}

到此WAL写日志过程全部解释清楚。里面还有一个回滚。不过FSHlog有个度数据流。所以就可以搞定了。

如果有什么没说清楚的，可以留言。欢迎交流。