spatialhadoop2.3源码阅读(八) RTree索引生成方法(二)

这一章主要介绍MapReduce的具体实现。

1. Map

 /**   * The map class maps each object to the cell with maximum overlap.   * @author Ahmed Eldawy   *   */  public static class RepartitionMapNoReplication<T extends Shape> extends MapReduceBase      implements Mapper<Rectangle, T, IntWritable, T> {    /**List of cells used by the mapper*/    private CellInfo[] cellInfos;        /**Used to output intermediate records*/    private IntWritable cellId = new IntWritable();        @Override    public void configure(JobConf job) {      try {        cellInfos = SpatialSite.getCells(job);        super.configure(job);      } catch (IOException e) {        throw new RuntimeException("Error loading cells", e);      }    }        /**     * Map function     * @param dummy     * @param shape     * @param output     * @param reporter     * @throws IOException     */    public void map(Rectangle cellMbr, T shape,        OutputCollector<IntWritable, T> output, Reporter reporter)        throws IOException {      Rectangle shape_mbr = shape.getMBR();      if (shape_mbr == null)        return;      double maxOverlap = -1.0;      int bestCell = -1;      // Only send shape to output if its lowest corner lies in the cellMBR      // This ensures that a replicated shape in an already partitioned file      // doesn't get send to output from all partitions      if (!cellMbr.isValid() || cellMbr.contains(shape_mbr.x1, shape_mbr.y1)) {        for (int cellIndex = 0; cellIndex < cellInfos.length; cellIndex++) {          Rectangle overlap = cellInfos[cellIndex].getIntersection(shape_mbr);          if (overlap != null) {            double overlapArea = overlap.getWidth() * overlap.getHeight();            if (bestCell == -1 || overlapArea > maxOverlap) {              maxOverlap = overlapArea;              bestCell = cellIndex;            }          }        }      }      if (bestCell != -1) {        cellId.set((int) cellInfos[bestCell].cellId);        output.collect(cellId, shape);      } else {        LOG.warn("Shape: "+shape+" doesn't overlap any partitions");      }    }  }

Map类大致可以分为两部分：configure方法和map方法。

configure方法的主要功能是获得上一章所讲的CellInfo数组。

接下来重点介绍map方法。

35行：获得输入数据的最小包围矩形。

43行：与spatialhadoop2.3源码阅读(六) grid 索引生成方法(二)中相比，当前cellMbr的值为Rectangle: (NaN,0.0)-(0.0,0.0)，所以if验证的前       半部分始终为true

44-51：遍历所有的网格，判断当前输入数据与哪一个网格的相交面积最大，则认为输入数据属于哪一个网格。

45：获得当前网格与输入数据的相交矩形，若不相交，则为null

47-50：若相交，与前一次的相交面积进行比较，若大于则更新。

55-57：将得出的当前输入数据所属的网格索引和输入数据一起输出。

2. Reduce

Reduce类与spatialhadoop2.3源码阅读(六) grid 索引生成方法(二)中的reduce完全相同，详情见该章。

3. outputCommiter

Commiter类与spatialhadoop2.3源码阅读(六) grid 索引生成方法(二)中的Commiter完全相同，详情见该章。

commiter的作用是将生成的所有包含_master的文件合并为同一个，生成_master.grid文件