IO is a problem difficult to handle in various of systems because it always becomes a bottleneck in data transfer. in this section, I will introduce some Java classes for two categories of data transfer, Bytes and Characters, those classes were included in java.io package, then I will invoke some methods in java.nio.file.Files class, then the section looks at the mechanism of NIO.2 (non blocking IO), which with advantage feature in multiple concurrent.
let's list out the agenda for this section.
IO Stream (introduce interfaces/classes in interface java.io)
-Bytes IO
-Characters IO
-Buffered IO
NIO (introduce methods in class java.nio.file.Files)
-buffer
-channel
NIO.2
-Asynchronous io (non blocking IO)
IO Stream
Bytes Stream
Java uses 8-bit (Byte) as a basic unit in program, this is Bytes String. all of the class to handle Bytes String are descended fromInputString andOutputtring.
an important thing to use Byte String is always remember to close the Strings when no longer use them.
Bytes String is easy to be understood by machine and it is always used to handle those primitive/raw data (binary), but not appropriate to handle more complicated data - such as character, witch is more easier to be understood by human, thus we have another type of String - Characters String.
example to demostrate how to use Byte Stream
example
package ioStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
public class ByteStream {
public static void CopyBytes() throws IOException {
/*
* subclss of java.io.InputStream, use them to manipulate files by
* Bytes.
*/
FileInputStream in = null;
FileOutputStream out = null;
try {
out = new FileOutputStream("output.txt");
// byte range : -128 ~ +128
// java array is an object, use new to initialise (malloc)
byte[] buf = new byte [] { 50, 0, -1, 28, -24 };
for (byte b : buf) {
out.write(b);
}
in = new FileInputStream("output.txt");
out = new FileOutputStream("copyOutput.txt");
int c;
// in.read will return the next byte of data, or -1 if the end of the file is reached
while ((c = in.read()) != -1) {
out.write(c);
}
} finally {
// never forget to close a ByteString instance if no longer need them
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
}
}
Characters Stream
Each Characters String uses 16-bit as a basic unit in transfer, Characters String not only more appropriate in human understanding but also resolve the problem of internationalization. Java platform stores data with Unicode conventions, Characters String will automatically converts its internal characters set to and from locale characters set, so you will no longer need to care the internationalization if using Characters String.
All of Characters String handler are descended from Reader andWriter.
InputStreamReader and OutputStreamWriter can be used to convert Bytes String to Characters String, for example, in sockets communication, we offten create character streams from the byte streams provided by socket classes.
Buffered IO
In general, String will be communicated directly with OS underlying IO, reading or writing will directly handle by OS and it makes a program much less efficient, to reduce this kind of overhead, Java invoke Buffer mechanism to handle IO String. Java platform implements a buffered IO String, a buffer is a memory region, the native input API is called only when the buffer is empty, and the native output API is called only when the buffer is full.
Both Bytes String and Characters String can be buffed, the following four classes are used to be handle it: BufferedInputStream andBufferedOutputStream create buffered byte streams, whileBufferedReader andBufferedWriter create buffered character streams.
BufferReader/BufferWriter can convert an un-buffered string to a buffered String,
inputStream = new BufferedReader(new FileReader("xanadu.txt"));
outputStream = new BufferedWriter(new FileWriter("characteroutput.txt"));
a example to demostrate FileReader, BufferedReader, FileWriter, PrintWriter
package ioStream;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
public class CharStream {
public static void CopyChars() throws IOException {
FileReader inputStream = null;
FileWriter outputStream = null;
try {
char[] buf = new char[] {'a','b','c'};
outputStream = new FileWriter("output.txt");
outputStream.write(buf);
outputStream.close();
inputStream = new FileReader("output.txt");
int c;
while ((c = inputStream.read()) != -1) {
System.out.println(c);
}
} finally {
if (inputStream != null) {
inputStream.close();
}
if (outputStream != null) {
outputStream.close();
}
}
}
public static void CopyLines() throws IOException {
BufferedReader inputStream = null;
PrintWriter outputStream = null;
char[] buf = new char[] {'d','e','f'};
try {
outputStream = new PrintWriter(new FileWriter("output.txt"));
outputStream.println(buf);
outputStream.close();
inputStream = new BufferedReader(new FileReader("output.txt"));
String strLine;
while ((strLine = inputStream.readLine()) != null) {
System.out.println(strLine);
}
} finally {
if (outputStream != null) outputStream.close();
if (inputStream != null) inputStream.close();
}
}
}
NIO
above IO String we call them traditional IO, they all transfer with a byte unit in the underlying, this is much less efficient. since JDK 1.4, Java platform invoke the NIO as replace of traditional IO. NIO means new IO or non-blocking IO, the difference between traditional IO and NIO is the NIO using memory mapping to speed up reading and writing, at the same time, the non-blocking mechanism also resolve the IO string blocking in multiple IO.
There are several packages defined in NIO, in this section we only focus on two of them.
-java.nio.-defines the bufferd
-java.nio.channel.-defines channel and selector
File Class
buffer
buffer looks like an array which stores several same type data. Buffer is just an abstract base class, there are many subclass extend it, the most common subclasses are ByteBuffer and CharBuffer.
There are three essential properties for buffer, there are capacity, limit and position, each subclasses of buffer defines two categories of get and put operations.
-Relative operations;
-Absolute operations;
buffers are not safety for use by multiple concurrent threads.
buffers can be set to read-only by invoking isReadOnly method.
the flip() method sets the limit to the position and set the position to 0, thus the buffer pointer moves to the start place, that means after invocation of flip(), the buffer prepare for outputting.
after outputting, Buffer invoke the clear() method, it is not for clearing up buffer data, while it set limit to capacity and the position to 0 and prepare for inputting data into Buffer.
channel
channel looks like a string object of traditional IO, Channel directly maps the partial or entire of a file to a Buffer, program unable to access data directly via channel, instead, program need to use a buffer to retrieve data from a channel, then the program read data from the buffer.
Java provides different implemented class by their functionality, such as FileChannel, SockChannel, etc. all types of channel instance should not be built by invoking their constructor method, instead, they should be obtained from getChannel() method byinvoking its specific class. for example, a FileChannel instance should be obtained by FileInputString.getChannel() as blow
FileChannel inChannel = new FileInputString(new File("xxx.txt")).getChannel()
below example demonstrates how to use File, Buffer and Channel
NIO.2
jdk 1.7 made significant enhancements, the major changes are
- provides comprehensive supports in file IO and file system accessing.
- Asynchronous IO
in the section I will just introduce the fist Item which was introduced in java.nio.file package in JDK1.7, the Asynchronous will be introduced in network communication.
The Path interface was invoked to NIO.2 to resolve the problems that the defect in old File classes, including cannot working in other OS, low performance and ambiguous error message, etc. NIO.2 provides two tool classes, thePath and Files.
Paths
A Paths object contains the file name and directory list used to construct the path, and is used to examine, locate, and manipulate files.
A Path is not system independent, that means you can manipulate files with the same method in all system, that's cool1
Paths class offers various methods to manipulate a path including creating,removing, comparing, joining and retrieving information, etc. here list out the major methods.
- Paths.get("/tmp/foo"); // create a path. it is the shorthand for FileSystems.getDefault().getPath("/users/sally");
- path.getFileName(),path.subpath(0,2),path.getParent(),path.getRoot(). // retrieving information
- path.equals(otherPath),path.startsWith(beginning),path.endsWith(ending)... //comparing
Files
The Files class is another primary entrypoint of the java.nio.file package. the Files class works on instance of Paths object and offers a rich set of static methods for reading, writing and manipulating files and directories. here list out the out operations of Files class.
copy(InputStream in, Path target, CopyOption... options)
createDirectories(Path dir, FileAttribute<?>... attrs)
createFile(Path path, FileAttribute<?>... attrs)
delete(Path path)
find(Path start, int maxDepth, BiPredicate<Path,BasicFileAttributes> matcher, FileVisitOption... options)
lines(Path path) //Read all lines from a file as a Stream.
write(Path path, byte[] bytes, OpenOption... options) //Writes bytes to a file.
a example to demonstrate the Files class
package fileIO;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.util.Iterator;
import java.util.List;
public class FileOps {
public static void readAllBytes() throws IOException {
Path path = Paths.get("output.txt");
byte[] buf = new byte[] { 'a', 'b', 'c' };
Files.write(path, buf, StandardOpenOption.CREATE);
byte[] readByte;
readByte = Files.readAllBytes(path);
for (byte b : readByte) {
System.out.println(b);
}
}
public static void readAllLines() throws IOException {
// TODO Auto-generated method stub
Path path = Paths.get("output.txt");
byte[] buf = new byte[] { 'r', 't', 'y' };
Files.write(path, buf, StandardOpenOption.CREATE);
char[] readChar;
Charset cs = Charset.defaultCharset();
List<String> listStr = Files.readAllLines(path, cs);
Iterator it = listStr.iterator();
while (it.hasNext()) {
System.out.println(it.next());
}
}
public static void readBuffer() throws IOException {
Path path = Paths.get("output.txt");
Charset cs = Charset.defaultCharset();
// no need to close out if put it implicitly in to try () for java7
try (BufferedWriter out = Files.newBufferedWriter(path, cs,
StandardOpenOption.CREATE)) {
char[] buf = new char[] { 'a', 'p', 'l' };
out.write(buf);
out.close();
}
String strLine;
try (BufferedReader in = Files.newBufferedReader(path, cs)) {
while ((strLine = in.readLine()) != null) {
System.out.println(strLine);
}
}
}
/*
* use the newInputStream(Path, OpenOption...) method. This method returns
* an unbuffered input stream for reading bytes from the file.
*/
public static void readStream() throws IOException{
Path path = Paths.get("output.txt");
byte[] buf = new byte[] { 'b', 's', 'a' };
/*
*here will write file in unbuffered model
*
/*
try (OutputStream out = Files.newOutputStream(path,
StandardOpenOption.CREATE)) {
out.write(buf);
} catch (IOException e) {
throw e;
}
*/
/*
* Files.newOutputStream returns a OutputStream obj which will not
* BufferedOutputStream can handle bytes stream in buffer model
*/
/*
try (OutputStream out = new BufferedOutputStream(Files.newOutputStream(
path, StandardOpenOption.CREATE))) {
out.write(buf);
} catch (IOException e) {
throw e;
}
*/
/*
* BufferedWriter extends from java.io.Writer, returns and buffers chars stream
* OutputStreamWriter also extends from java.io.Writer, is a bridge from byte streams to char streams
* Files.newOutputStream(path) returns an unbuffered OutputStream (bytes stream)
*/
try (BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(Files.newOutputStream(path)))) {
char[] cbuf = new char[] {'h','p','f'};
writer.write(cbuf);
} catch (IOException e) {
throw e;
}
/*
* BufferedReader extends from java.io.Reader, returns and buffers chars stream
* InputStreamReader extends from java.io.Reader, is a bridge from byte streams to character streams
* Files.newInputStream(path) returns a unbuffered InputStream (bytes stream)
*/
try (BufferedReader reader = new BufferedReader(new InputStreamReader(Files.newInputStream(path)))) {
String strLine;
while ((strLine = reader.readLine()) != null) {
System.out.println(strLine);
}
} catch (IOException e) {
// TODO Auto-generated catch block
throw e;
}
}
public static void readChannel() throws IOException {
Path path = Paths.get("output.txt");
try (SeekableByteChannel sbc = Files.newByteChannel(path, StandardOpenOption.CREATE) ) {
//can not read file by Channel directly, instead, need to read into Buffer first
ByteBuffer buf = ByteBuffer.allocate(10);
String encoding = System.getProperty("file.encoding");
while (sbc.read(buf) > 0) {
buf.rewind();
System.out.print(Charset.forName(encoding).decode(buf));
buf.flip();
}
} catch (IOException e){
throw e;
}
}
}