HDFS HA中name nodes访问顺序的问题

配置HDFS HA时，需要指明两个name nodes，一个active，一个作为hot standby，当active NN出现问题时，可以及时手动或自动切换到standby NN（此时原来的standby NN转换为新的active NN），使HDFS cluster保持正常工作。


涉及的配置，以146.67这个name node为例。


hdfs-site.xml：


    <property>
      <name>dfs.nameservices</name>
      <value>hadoopcluster</value>
    </property>


    <property>
      <name>dfs.ha.namenodes.hadoopcluster</name>
      <value>nn1,nn2</value>
    </property>


    <property>
      <name>dfs.namenode.rpc-address.hadoopcluster.nn1</name>
      <value>storm14668:8020</value>
    </property>


    <property>
      <name>dfs.namenode.rpc-address.hadoopcluster.nn2</name>
      <value>storm14667:8020</value>
    </property>


    <property>
      <name>dfs.namenode.http-address.hadoopcluster.nn1</name>
      <value>storm14668:50070</value>
    </property>


    <property>
      <name>dfs.namenode.http-address.hadoopcluster.nn2</name>
      <value>storm14667:50070</value>
    </property>


    <property>
      <name>dfs.namenode.https-address.hadoopcluster.nn1</name>
      <value>storm14668:50470</value>
    </property>


    <property>
      <name>dfs.namenode.https-address.hadoopcluster.nn2</name>
      <value>storm14667:50470</value>
    </property>


    <property>
      <name>dfs.client.failover.proxy.provider.hadoopcluster</name>
      <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>




其中，最后一个配置项，即dfs.client.failover.proxy.provider.hadoopcluster，决定了这个问题的答案。


HDFS官方文档中对这个配置项的说明：
Configure the name of the Java class which will be used by the DFS Client to determine which NameNode is the current Active, and therefore which NameNode is currently serving client requests. The only implementation which currently ships with Hadoop is the ConfiguredFailoverProxyProvider, so use this unless you are using a custom one.


如上面列出的配置所示，我们就是用了这个Hadoop提供的Java class：ConfiguredFailoverProxyProvider。


这个类的实现：hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ConfiguredFailoverProxyProvider.java，其中关键的code snippet：


  private final List<AddressRpcProxyPair<T>> proxies =
      new ArrayList<AddressRpcProxyPair<T>>();


  private int currentProxyIndex = 0;


      try {
      ugi = UserGroupInformation.getCurrentUser();


      Map<String, Map<String, InetSocketAddress>> map = DFSUtil.getHaNnRpcAddresses(
          conf);
      Map<String, InetSocketAddress> addressesInNN = map.get(uri.getHost());


      if (addressesInNN == null || addressesInNN.size() == 0) {
        throw new RuntimeException("Could not find any configured addresses " +
            "for URI " + uri);
      }


      Collection<InetSocketAddress> addressesOfNns = addressesInNN.values();
      for (InetSocketAddress address : addressesOfNns) {
        proxies.add(new AddressRpcProxyPair<T>(address));
      }


      // The client may have a delegation token set for the logical
      // URI of the cluster. Clone this token to apply to each of the
      // underlying IPC addresses so that the IPC code can find it.
      HAUtil.cloneDelegationTokenForLogicalUri(ugi, uri, addressesOfNns);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }


  /**
   * Lazily initialize the RPC proxy object.
   */
  @Override
  public synchronized ProxyInfo<T> getProxy() {
    AddressRpcProxyPair<T> current = proxies.get(currentProxyIndex);
    if (current.namenode == null) {
      try {
        current.namenode = NameNodeProxies.createNonHAProxy(conf,
            current.address, xface, ugi, false, fallbackToSimpleAuth).getProxy();
      } catch (IOException e) {
        LOG.error("Failed to create RPC proxy to NameNode", e);
        throw new RuntimeException(e);
      }
    }
    return new ProxyInfo<T>(current.namenode, current.address.toString());
  }


  @Override
  public synchronized void performFailover(T currentProxy) {
    currentProxyIndex = (currentProxyIndex + 1) % proxies.size();
  }




根据上面的dfs.ha.namenodes.hadoopcluster这个配置项的值：nn1,nn2，HDFS client在访问name nodes时，首先尝试nn1，如果失败，再尝试nn2。
那么，有没有可能改变这个“写死”的访问name nodes的顺序呢？理论上是有的，就是按照上面配置项的说明：so use this unless you are using a custom one，自己实现一个FailoverProxyProvider。


需要提醒的是，不要被多次出现的“Failover”所迷惑。不仅仅是failover时，这个类才起作用。