HBase实战 2.2.3 背景知识： HBase写路径



2.2.3 Under the hood: the HBase write path
Whether you use Put to record a new row in HBase or to modify an existing row, the
internal process is the same. HBase receives the command and persists the change, or
throws an exception if the write fails.

无论你使用Put插入一个新行或者修改已有行，内部的过程是一样的。HBase收到命令并持久化变更，或者抛出异常，如果写入失败。

When a write is made, by default, it goes into two places: the write-ahead log (WAL), also referred to as the HLog, and the MemStore(figure 2.1). The default behavior of HBase recording the write in both places is in order to maintain data durability. Only after the change is written to and confirmed in both places is the write considered complete.

写入的时候，缺省的，会操作两个地方：前置写入日志（WAL），也叫做HLog，还有MemStore。HBase把数据写到两个地方的原因是为了数据的持久性。仅当数据写入到这两个地方并且被确认后，写入才会完成。

The MemStore is a write buffer where HBase accumulates data in memory before a
permanent write. Its contents are flushed to disk to form an HFile when the MemStore
fills up. It doesn’t write to an existing HFile but instead forms a new file on every flush.

MemStore是写缓冲区，在数据写入到永久存储区前，Hbase把数据暂存在其中。当MemStore写满的时候，MemStore暂存的数据会被刷入磁盘，形成一个HFile。每次刷写并不会写入到已经存在的HFile，而是每次都会生成一个新的HFile。

The HFile is the underlying storage format for HBase. HFiles belong to a column family,and a column family can have multiple HFiles. But a single HFile can’t have data for multiple column families. There is one MemStore per column family.

HFile是HBase的底层存储格式。HFiles属于列簇，一个列簇可以拥有多个HFile,但是一个HFile只能属于一个列簇。一个列簇有一个MemStore。

Failures are common in large distributed systems, and HBase is no exception.
Imagine that the server hosting a MemStore that has not yet been flushed crashes.
You’ll lose the data that was in memory but not yet persisted. HBase safeguards against that by writing to the WAL before the write completes. Every server that’s part of the HBase cluster keeps a WAL to record changes as they happen. The WAL is a file on the underlying file system. A write isn’t considered successful until the new WAL entry is successfully written. This guarantee makes HBase as durable as the file system backing it. Most of the time, HBase is backed by the Hadoop Distributed Filesystem (HDFS).

大型分布式系统发生故障再正常不过了，HBase也不例外。想象一下，一个服务器宕机了，该服务器的MemStore还没有刷写到硬盘。内存中没有持久化的数据将会丢失。HBase通过写入数据时，先把数据写入WAL防止类似事件发生。HBase集群的每一个服务器维持一个WAL记录变化。WAL是一个底层文件系统文件。在WAL成功写入前，一个写入不会认为成功。这使得HBase拥有其底部文件系统同样的持久性。大部分情况下，HBase的底层文件系统是HDFS。

If HBase goes down, the data that was not yet flushed from the MemStore to the
HFile can be recovered by replaying the WAL. You don’t have to do this manually. It’s
all handled under the hood by HBase as a part of the recovery process. There is a single WAL per HBase server, shared by all tables (and their column families) served from that server.

如果HBase崩溃，MemStore中没来得及写入HFile的数据将会通过回放WAL恢复。你无需手动干预，这是HBase恢复过程的一部分。每台HBase服务器一个WAL，由该服务器上所有的表（和列簇）共享。