Java NIO 2.0 : Memory-Mapped Files | MappedByteBuffer Tutorial

If you know how java IO works at lower level, then you will be aware of buffer handling, memory paging and other such concepts. For conventional file I/O, in which user processes issue read() and write() system calls to transfer data, there is almost always one or more copy operations to move the data between these filesystem pages in kernel space and a memory area in user space. This is because there is not usually a one-to-one alignment between filesystem pages and user buffers. There is, however, a special type of I/O operation supported by most operating systems that allows user processes to take maximum advantage of the page-oriented nature of system I/O and completely avoid buffer copies. This is called memory-mapped I/O and we are going to learn few things here around memory-mapped files.

Memory-Mapped Files

Memory-mapped I/O uses the filesystem to establish a virtual memory mapping from user space directly to the applicable filesystem pages. With a memory-mapped file, you can pretend that the entire file is in memory and that you can access it by simply treating it as a very large array. This approach greatly simplifies the code you write in order to modify the file.

To do both writing and reading in memory mapped files, we start with a RandomAccessFile, get a channel for that file. Memory mapped byte buffers are created via the FileChannel.map() method. This class extends the ByteBuffer class with operations that are specific to memory-mapped file regions. A mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is garbage-collected. Note that you must specify the starting point and the length of the region that you want to map in the file; this means that you have the option to map smaller regions of a large file.

import java.io.RandomAccessFile;import java.nio.MappedByteBuffer;import java.nio.channels.FileChannel;public class MemoryMappedFileExample{    static int length = 0x8FFFFFF; // 128 Mb    public static void main(String[] args) throws Exception    {        MappedByteBuffer out = new RandomAccessFile("howtodoinjava.dat", "rw")                                    .getChannel().map(FileChannel.MapMode.READ_WRITE, 0, length);        for (int i = 0; i < length; i++)        {            out.put((byte) 'x');        }        System.out.println("Finished writing");    }}

The file created with the above program is 128 MB long, which is probably larger than the space your OS will allow. The file appears to be accessible all at once because only portions of it are brought into memory, and other parts are swapped out. This way a very large file (up to 2 GB) can easily be modified.

Like conventional file handles, file mappings can be writable or read-only. The first two mapping modes, MapMode.READ_ONLY and MapMode.READ_WRITE, are fairly obvious. They indicate whether you want the mapping to be read-only or to allow modification of the mapped file. The third mode, MapMode.PRIVATE, indicates that you want a copy-on-write mapping. This means that any modifications you make via put( ) will result in a private copy of the data that only the MappedByteBuffer instance can see. No changes will be made to the underlying file, and any changes made will be lost when the buffer is garbage collected. Even though a copy-on-write mapping prevents any changes to the underlying file, you must have opened the file for read/write to set up a MapMode.PRIVATE mapping. This is necessary for the returned MappedByteBuffer object to allow put()s.

You’ll notice that there is no unmap() method. Once established, a mapping remains in effect until the MappedByteBuffer object is garbage collected. Also, mapped buffers are not tied to the channel that created them. Closing the associated FileChannel does not destroy the mapping; only disposal of the buffer object itself breaks the mapping.
A MemoryMappedBuffer has a fixed size, but the file it’s mapped to is elastic. Specifically, if a file’s size changes while the mapping is in effect, some or all of the buffer may become inaccessible, undefined data could be returned, or unchecked exceptions could be thrown. Be careful about how files are manipulated by other threads or external processes when they are memory-mapped.

Advantages of Memory-Mapped Files

Memory-Mapped IO have several advantages over normal I/O:

• The user process sees the file data as memory, so there is no need to issue read()or write() system calls.
• As the user process touches the mapped memory space, page faults will be generated automatically to bring in the file data from disk. If the user modifies the mapped memory space, the affected page is automatically marked as dirty and will be subsequently flushed to disk to update the file.
• The virtual memory subsystem of the operating system will perform intelligent caching of the pages, automatically managing memory according to system load.
• The data is always page-aligned, and no buffer copying is ever needed.
• Very large files can be mapped without consuming large amounts of memory to copy the data.

Reading a Memory-Mapped File

To read a file using memory mapped IO, use below code template:import java.io.File;import java.io.RandomAccessFile;import java.nio.MappedByteBuffer;import java.nio.channels.FileChannel;public class MemoryMappedFileReadExample{    private static String bigExcelFile = "bigFile.xls";    public static void main(String[] args) throws Exception    {        //Create file object        File file = new File(bigExcelFile);        //Get file channel in readonly mode        FileChannel fileChannel = new RandomAccessFile(file, "r").getChannel();        //Get direct byte buffer access using channel.map() operation        MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());        // the buffer now reads the file as if it were loaded in memory.        System.out.println(buffer.isLoaded());  //prints false        System.out.println(buffer.capacity());  //Get the size based on content size of file        //You can read the file from this buffer the way you like.        for (int i = 0; i < buffer.limit(); i++)        {            System.out.print((char) buffer.get()); //Print the content of file        }    }}

Writing into a Memory-Mapped File

To write data into a file using memory mapped IO, use below code template:import java.io.File;import java.io.RandomAccessFile;import java.nio.MappedByteBuffer;import java.nio.channels.FileChannel;public class MemoryMappedFileWriteExample {    private static String bigExcelFile = "test.txt";    public static void main(String[] args) throws Exception {        // Create file object        File file = new File(bigExcelFile);        //Delete the file; we will create a new file        file.delete();        // Get file channel in readonly mode        FileChannel fileChannel = new RandomAccessFile(file, "rw").getChannel();        // Get direct byte buffer access using channel.map() operation        MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0, 4096 * 8 * 8);        //Write the content using put methods        buffer.put("howtodoinjava.com".getBytes());    }}