hama程序出现Messages must never be behind the vertex in ID问题解决

来源:互联网 发布:同花顺mac 编辑:程序博客网 时间:2024/06/05 16:24

在运行hama程序时,在partition完成后,经常出现java.lang.IllegalArgumentException: Messages must never be behind the vertex in ID! Current Message ID: 100003 vs. 100004。异常

看了一下hama源码,如下:

/**

   * Iterating utility that ensures following things: <br/>

   * - if vertex is active, but the given message does not match the vertexID,

   * return null. <br/>

   * - if vertex is inactive, but received a message that matches the ID, build

   * an iterator that can be iterated until the next vertex has been reached

   * (not buffer in memory) and set the vertex active <br/>

   * - if vertex is active, and the given message does match the vertexID,

   * return an iterator that can be iterated until the next vertex has been

   * reached. <br/>

   * - if vertex is inactive, and received no message, return null.

   */

  @SuppressWarnings("unchecked")

  private VertexMessageIterable<V, M> iterate(GraphJobMessage currentMessage,

      V firstMessageId, Vertex<V, E, M> vertex,

      BSPPeer<Writable, Writable, Writable, Writable, GraphJobMessage> peer) {

    int comparision = firstMessageId.compareTo(vertex.getVertexID());

    if (conf.getBoolean("hama.check.missing.vertex", true)) {

      if (comparision < 0) {

        throw new IllegalArgumentException(

          "Messages must never be behind the vertex in ID! Current Message ID: "

              + firstMessageId + " vs. " + vertex.getVertexID());

      }

    } else {

      while (comparision < 0) {

        VertexMessageIterable<V, M> messageIterable = new VertexMessageIterable<V, M>(currentMessage,

          firstMessageId, peer);

        currentMessage = messageIterable.getOverflowMessage();

        firstMessageId = (V)currentMessage.getVertexId();

        comparision = firstMessageId.compareTo(vertex.getVertexID());

      }

    }

    if (comparision == 0) {

      // vertex id matches with the vertex, return an iterator with newest

      // message

      return new VertexMessageIterable<V, M>(currentMessage,

          vertex.getVertexID(), peer);

    } else {

      // return null

      return null;

    }

  }

问题出在传递的消息没有找到对应的节点,是自己输入的文件格式有问题。Hama中如果使用Vertex类,则输入的默认格式是(点、边<起点终点>)。与mapreduce类似,mapper类为文件的每一行调用一个map()函数,Vertex类为每一个点调用一个compute()函数。该点名作为一个peer的名字。所以在边的输入文件里的<起点终点>也要在点的位置出现,如果没有出现就会出现上面的错误。举个例子:如果输入文件如下就会出错(边只写终点):

1   2   3   17  

2   1

3   1

图形结构如下

17在边的位置出现了,但在点的位置没有出现。消息传递不到17的点,所以抛出异常。输入文件改为

1   2   3   17  

2   1

3   1

17 1

就可以正常运行了。

原创粉丝点击