Hadoop Sequence File 文件的读取和写入

写入代码

下面是写入了100条(key,value)的信息，其中以LongWriable为key，以Text作为value.

        Configuration config = new Configuration();        FileSystem fs  = FileSystem.get(conf);        int i = 0;        Path path = new Path("/home/lake/hello.xml");        SequenceFile.Writer writer = null;        SequenceFile.Writer.Option optPath = SequenceFile.Writer.file(path);        //定义key        SequenceFile.Writer.Option optKey = SequenceFile.Writer.keyClass(LongWritable.class);        //定义value        SequenceFile.Writer.Option optVal = SequenceFile.Writer.valueClass(Text.class);        writer = SequenceFile.createWriter(conf, optPath, optKey, optVal);        //写入的数据可以根据你的情况来定，我这只是测试        String value = "hello world";        while(i < 100){            writer.append(new LongWritable(i),new Text(value));            i ++;        }        writer.close();

上面程序运行完成之后，就可以在指定的路径上看到产生的文件。

读取的代码

                Configuration config = new Configuration();                FileSystem fs  = FileSystem.get(conf);                Path path = new Path("/home/lake/hello.xml");                SequenceFile.Reader reader = new SequenceFile.Reader(fs.getConf(), SequenceFile.Reader.file(path));                List<Object> sampleValues = new ArrayList<Object>();                Writable key = (Writable) ReflectionUtils.newInstance(reader.getKeyClass(), fs.getConf());                Writable value = (Writable) ReflectionUtils.newInstance(reader.getValueClass(), fs.getConf());                int count = 0;                String keyName = "Key";                String valueName = "Value";                //change data to json format                while (reader.next(key, value) && count < 12) {sampleValues.add("{\"" + keyName + "\": \"" + key + "\", \"" + valueName + "\": \"" + value + "\"}");                    count++;                }