spatialhadoop2.1源码阅读(三) 自定义InputFormat（SpatialInputFormat & ShapeInputFormat）

来源：互联网发布：xilica xd4080软件编辑：程序博客网时间：2024/05/22 06:29

SpatialInputFormat实现了InputFormat两大功能中的其中一个：getRecordReader。另外一个功能getSplits默认继承了FileInputFormat类的实现。

SpatialInputFormat的getRecordReader具体实现如下：

@SuppressWarnings("rawtypes")  protected Class<? extends RecordReader> rrClass;    @SuppressWarnings("unchecked")  @Override  public RecordReader<K, V> getRecordReader(InputSplit split, JobConf job,      Reporter reporter) throws IOException {    // Create compressionCodecs to be used by isSplitable method    if (compressionCodecs == null)      compressionCodecs = new CompressionCodecFactory(job);    if (split instanceof FileSplit) {      FileSplit fsplit = (FileSplit) split;      if (fsplit.getPath().getName().toLowerCase().endsWith(".hdf")) {        // HDF File. Create HDFRecordReader        return (RecordReader<K, V>) new HDFRecordReader(job, fsplit,            job.get(HDFRecordReader.DatasetName),            job.getBoolean(HDFRecordReader.SkipFillValue, true));      }      try {        @SuppressWarnings("rawtypes")        Constructor<? extends RecordReader> rrConstructor;        rrConstructor = rrClass.getDeclaredConstructor(constructorSignature);        rrConstructor.setAccessible(true);        return rrConstructor.newInstance(new Object [] {job, fsplit});      } catch (SecurityException e) {        e.printStackTrace();      } catch (NoSuchMethodException e) {        e.printStackTrace();      } catch (IllegalArgumentException e) {        e.printStackTrace();      } catch (InstantiationException e) {        e.printStackTrace();      } catch (IllegalAccessException e) {        e.printStackTrace();      } catch (InvocationTargetException e) {        e.printStackTrace();      }      throw new RuntimeException("Cannot generate a record reader");    } else {      throw new RuntimeException("Cannot handle splits of type "+split.getClass());    }  }

其中，rrclass为实际的RecordReader类，采用反射的方式生成，子类只需要继承SpatialInputFormat类，并指定rrclass。再调用父类的getRecordReader方法，即可产生RecordReader类。

ShapeInputFormat类继承了SpatialInputFormat类，其覆盖了父类的getRecordReader方法，具体实现如下：

public class ShapeInputFormat<S extends Shape> extends SpatialInputFormat<Rectangle, S> {    @Override  public RecordReader<Rectangle, S> getRecordReader(InputSplit split,      JobConf job, Reporter reporter) throws IOException {    reporter.setStatus(split.toString());    this.rrClass = ShapeRecordReader.class;    return super.getRecordReader(split, job, reporter);  }}

最终产生了ShapeRecordReader类。

0 0