HBase源码系列(三)HMaster启动过程

来源:互联网 发布:知乎 量化分析师 招聘 编辑:程序博客网 时间:2024/05/21 07:10

这是HBase server端开始的第一章。

在HMaster的构造函数,从方法注释上看到,主要做了两件事情:
1、初始化本地的HRegionServer;
2、启动ActiveMasterManager;

public HMaster(final Configuration conf, CoordinatedStateManager csm)        throws IOException, KeeperException, InterruptedException {    super(conf, csm);    ...    this.metricsMaster = new MetricsMaster(new MetricsMasterWrapperImpl(this));    ...    // Some unit tests don't need a cluster, so no zookeeper at all    if (!conf.getBoolean("hbase.testing.nocluster", false)) {        activeMasterManager = new ActiveMasterManager(zooKeeper, this.serverName, this);        int infoPort = putUpJettyServer();        startActiveMasterManager(infoPort);    } else {        activeMasterManager = null;    }}

HMaster是HRegionServer的子类,HRegionServer又是抽象类HasThread的子类,HasThread实现了Runnable接口。

HRegionServer的构造中,启动rpc通信。

public HRegionServer(Configuration conf, CoordinatedStateManager csm) throws IOException, InterruptedException {    rpcServices = createRpcServices();    rpcServices.start();}
/** Starts the service.  Must be called before any calls will be handled. */@Overridepublic synchronized void start() {  if (started) return;  authTokenSecretMgr = createSecretManager();  if (authTokenSecretMgr != null) {    setSecretManager(authTokenSecretMgr);    authTokenSecretMgr.start();  }  this.authManager = new ServiceAuthorizationManager();  HBasePolicyProvider.init(conf, authManager);  responder.start();  listener.start();  scheduler.start();  started = true;}

在启动了HMaster之后,从finishActiveMasterInitialization(MonitoredTask status)方法的注释上,看到完成HMaster的初始化过程做了许多工作:
1、初始化master的组件,如file system manager, server manager,assignment manager, region server tracker等;
2、启动必要的服务线程,如balancer, catalog janior,executor services等;
3、等待RegionServers注册;
4、切割日志,并进行数据恢复(如果需要);
5、分配 meta/namespace regions;

阻塞等待ActiveMaster启动完成:

public void run() {try {    while (!master.isStopped() && master.isActiveMaster()) {        Thread.sleep(timeout);        if (master.isInitialized()) {            LOG.debug("Initialization completed within allotted tolerance. Monitor exiting.");        } else {            LOG.error("Master failed to complete initialization after " + timeout + "ms. Please"                    + " consider submitting a bug report including a thread dump of this process.");            if (haltOnTimeout) {                LOG.error("Zombie Master exiting. Thread dump to stdout");                Threads.printThreadInfo(System.out, "Zombie HMaster");                System.exit(-1);            }        }    }} catch (InterruptedException ie) {    LOG.trace("InitMonitor thread interrupted. Existing.");}}

HMaster初始化完成后,后面才是重头戏,简单列下都实例化了哪些东西。

// file system manager for the master FS operationsprivate MasterFileSystem fileSystemManager;// server manager to deal with region server infovolatile ServerManager serverManager;    /** * Cluster connection to be shared by services. * Initialized at server startup and closed when server shuts down. * Clients must never close it explicitly. */protected ClusterConnection clusterConnection;    /* * Long-living meta table locator, which is created when the server is started and stopped * when server shuts down. References to this locator shall be used to perform according * operations in EventHandlers. Primary reason for this decision is to make it mockable * for tests. */protected MetaTableLocator metaTableLocator;MasterCoprocessorHost cpHost;private TableNamespaceManager tableNamespaceManager;...

这些类都会被实例化。开学啦,等待HRegionServer报道:

// Wait for region servers to report inthis.serverManager.waitForRegionServers(status);

分配hbase:meta表前,先split meta表的日志:
先从ZNode上拿到meta region的状态信息。
切割WAL日志:

long splitTime = 0, splitLogSize = 0;List<Path> logDirs = getLogDirs(serverNames);splitLogManager.handleDeadWorkers(serverNames);splitTime = EnvironmentEdgeManager.currentTime();splitLogSize = splitLogManager.splitLogDistributed(serverNames, logDirs, filter);splitTime = EnvironmentEdgeManager.currentTime() - splitTime;if (this.metricsMasterFilesystem != null) {  if (filter == META_FILTER) {    this.metricsMasterFilesystem.addMetaWALSplit(splitTime, splitLogSize);  } else {    this.metricsMasterFilesystem.addSplit(splitTime, splitLogSize);  }}

分配meta表

发送RPC到指定的server

OpenRegionResponse response = admin.openRegion(null, request);

构建user regions

this.assignmentManager.joinCluster();

分配region的工作是由assignmentManager完成的,joinCluster()方法调用processDeadServersAndRegionsInTransition()方法的最后调用assignAllUserRegions()

if (!failover) {  // Fresh cluster startup.  LOG.info("Clean cluster startup. Assigning user regions");  assignAllUserRegions(allRegions);}

分配用户region的方法和分配meta表的方法基本一致。

至此,启动HMaster的过程基本完成了。

原创粉丝点击