Java IO读写大文件的几种方式及测试

读取文件大小：1.45G
第一种，OldIO:

Java代码

1. public staticvoid oldIOReadFile() throws IOException{
2. BufferedReader br = new BufferedReader(new FileReader("G://lily_947.txt"));
3. PrintWriter pw = new PrintWriter("G://oldIO.tmp");
4. char[] c = newchar[100*1024*1024];
5. for(;;){
6. if(br.read(c)!=-1){
7. pw.print(c);
8. }else{
9. break;
10. }
11. }
12. pw.close();
13. br.close();
14. }

public static void oldIOReadFile() throws IOException{BufferedReader br = new BufferedReader(new FileReader("G://lily_947.txt"));PrintWriter pw = new PrintWriter("G://oldIO.tmp");char[] c = new char[100*1024*1024];for(;;){if(br.read(c)!=-1){pw.print(c);}else{break;}}pw.close();br.close();}

耗时70.79s


第二种，newIO:

Java代码

1. public staticvoid newIOReadFile() throws IOException{
2. FileChannel read = new RandomAccessFile("G://lily_947.txt","r").getChannel();
3. FileChannel writer = new RandomAccessFile("G://newIO.tmp","rw").getChannel();
4. ByteBuffer bb = ByteBuffer.allocate(200*1024*1024);
5. while(read.read(bb)!=-1){
6. bb.flip();
7. writer.write(bb);
8. bb.clear();
9. }
10. read.close();
11. writer.close();
13. }

public static void newIOReadFile() throws IOException{FileChannel read = new RandomAccessFile("G://lily_947.txt","r").getChannel();FileChannel writer = new RandomAccessFile("G://newIO.tmp","rw").getChannel();ByteBuffer bb = ByteBuffer.allocate(200*1024*1024);while(read.read(bb)!=-1){bb.flip();writer.write(bb);bb.clear();}read.close();writer.close();}

耗时47.24s


第三种，RandomAccessFile:

Java代码

1. public staticvoid randomReadFile() throws IOException{
2. RandomAccessFile read = new RandomAccessFile("G://lily_947.txt","r");
3. RandomAccessFile writer = new RandomAccessFile("G://random.tmp","rw");
4. byte[] b = newbyte[200*1024*1024];
5. while(read.read(b)!=-1){
6. writer.write(b);
7. }
8. writer.close();
9. read.close();
10. }

public static void randomReadFile() throws IOException{RandomAccessFile read = new RandomAccessFile("G://lily_947.txt","r");RandomAccessFile writer = new RandomAccessFile("G://random.tmp","rw");byte[] b = new byte[200*1024*1024];while(read.read(b)!=-1){writer.write(b);}writer.close();read.close();}

耗时46.65

第四种，MappedByteBuffer：

Java代码

1. public staticvoid mappedBuffer() throws IOException{
2. FileChannel read = new FileInputStream("G://lily_947.txt").getChannel();
3. FileChannel writer = new RandomAccessFile("G://buffer.tmp","rw").getChannel();
4. long i = 0;
5. long size = read.size()/30;
6. ByteBuffer bb,cc = null;
7. while(i<read.size()&&(read.size()-i)>size){
8. bb = read.map(FileChannel.MapMode.READ_ONLY, i, size);
9. cc = writer.map(FileChannel.MapMode.READ_WRITE, i, size);
10. cc.put(bb);
11. i+=size;
12. bb.clear();
13. cc.clear();
14. }
15. bb = read.map(FileChannel.MapMode.READ_ONLY, i, read.size()-i);
16. cc.put(bb);
17. bb.clear();
18. cc.clear();
19. read.close();
20. writer.close();
22. }

public static void mappedBuffer() throws IOException{FileChannel read = new FileInputStream("G://lily_947.txt").getChannel();FileChannel writer = new RandomAccessFile("G://buffer.tmp","rw").getChannel();long i = 0;long size = read.size()/30;ByteBuffer bb,cc = null;while(i<read.size()&&(read.size()-i)>size){bb = read.map(FileChannel.MapMode.READ_ONLY, i, size);cc = writer.map(FileChannel.MapMode.READ_WRITE, i, size);cc.put(bb);i+=size;bb.clear();cc.clear();}bb = read.map(FileChannel.MapMode.READ_ONLY, i, read.size()-i);cc.put(bb);bb.clear();cc.clear();read.close();writer.close();}

耗时：36

前三种读法对应的资源占用图如下：
相对于最后一种内存直接映射方式前面的测试其实无意义，基本秒杀。。。。。
对于很大的文件直接分块映射时内存会不够，这是因为MappedByteBuffer未被释放造成的，sun未提供直接回收MappedByteBuffer区域的方法，这个时候有两种方法解决，第一种比较愚笨的：

Java代码

1. System.gc();
2. System.runFinalization();
3. try {
4. Thread.sleep(3000);
5. } catch (InterruptedException e) {
7. e.printStackTrace();
8. }

             System.gc();             System.runFinalization();             try {Thread.sleep(3000);} catch (InterruptedException e) {e.printStackTrace();}

第二种网上找来的，利用反射调用clean方法：

Java代码

1. public staticvoid unmap(final MappedByteBuffer buffer) {
2. if (buffer == null) {
3. return;
4. }
5. AccessController.doPrivileged(new PrivilegedAction<Object>() {
6. public Object run() {
7. try {
8. Method getCleanerMethod = buffer.getClass().getMethod("cleaner",new Class[0]);
9. if (getCleanerMethod != null) {
10. getCleanerMethod.setAccessible(true);
11. Object cleaner = getCleanerMethod.invoke(buffer, new Object[0]);
12. Method cleanMethod = cleaner.getClass().getMethod("clean",new Class[0]);
13. if (cleanMethod != null) {
14. cleanMethod.invoke(cleaner, new Object[0]);
15. }
16. }
17. } catch (Exception e) {
18. e.printStackTrace();
19. }
20. return null;
21. }
23. });
24. }

public static void unmap(final MappedByteBuffer buffer) {if (buffer == null) {return;}AccessController.doPrivileged(new PrivilegedAction<Object>() {public Object run() {try {Method getCleanerMethod = buffer.getClass().getMethod("cleaner", new Class[0]);if (getCleanerMethod != null) {getCleanerMethod.setAccessible(true);Object cleaner = getCleanerMethod.invoke(buffer, new Object[0]);Method cleanMethod = cleaner.getClass().getMethod("clean", new Class[0]);if (cleanMethod != null) {cleanMethod.invoke(cleaner, new Object[0]);}}} catch (Exception e) {e.printStackTrace();}return null;} });}

以上两种方法感觉都别扭，还有就是可以自己分割成物理文件再循环调用，这个也不太美观。
速度也会减慢好多。