Java NIO: Buffers

If you've worked with binary file formats, for example in the fields of data recovery or file compatibility, you'll be familiar with the idea of processing "blocks" of data from a file or stream, or processing data as groups of bytes that function together as other types of data such as 4-byte words, Unicode strings etc. For example, you might need to read data in from a file that consists of a "4 byte header", followed by a "4-byte offset" to something, followed by a "2-byte length" etc. Or in a database file, you might need to treat the file as fixed-size "pages" where you read in a page of data from a particular offset, modify it in memory, then write it back to the file.

In the standard java.io package, a file or stream is generally presented to the programmer as a series of bytes or byte array(s). The DataInput and DataOutput interfaces can help out, but they still only work on streams, and crucially don't allow byte order to be specified. Operations such as "read a little-endian 4-byte integer from offset 10" are clumsy in this stream-based model.

This is where the NIO Buffer class and its subclasses come in. A commonly used buffer class is ByteBuffer. This and its cousins:

Additionally, a type of buffer called a direct buffer provides access to a fixed area of memory allocated outside the Java heap.

At present, buffer sizes are limited to 2GB (the maximum positive number that can be represented in an int. An updated planned for Java 7 will allow large buffers (with the size and indexes held as a long).

On the next page, we'll start by having a look at the layout of a NIO buffer. This will help us understand the various methods that can be called on a buffer. We'll then look at the various types of buffer offered as part of the java.nio package, and crucially the ByteBuffer, probably the most common and generic NIO buffer class.