java - Given InputStream replace character and produce OutputStream

Question

I have a lot of massive files I need convert to CSV by replacing certain characters.

I am looking for reliable approach given InputStream return OutputStream and replace all characters c1 to c2.

Trick here is to read and write in parallel, I can't fit whole file in memory.

Do I need to run it in separate thread if I want read and write at the same time?

Thanks a lot for your advices.

Answer 1

To copy data from an input stream to an output stream you write data while you're reading it either a byte (or character) or a line at a time.

Here is an example that reads in a file converting all 'x' characters to 'y'.

BufferedInputStream in = new BufferedInputStream(new FileInputStream("input.dat"));
BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("output.dat"));
int ch;
while((ch = in.read()) != -1) {
        if (ch == 'x') ch = 'y';
        out.write(ch);
}
out.close();
in.close();

Or if can use a Reader and process a line at a time then can use this aproach:

BufferedReader reader = new BufferedReader(new FileReader("input.dat"));
PrintWriter writer = new PrintWriter(
      new BufferedOutputStream(new FileOutputStream("output.dat")));
String str;
while ((str = reader.readLine()) != null) {
    str = str.replace('x', 'y');     // replace character at a time
    str = str.replace("abc", "ABC"); // replace string sequence
    writer.println(str);
}
writer.close();
reader.close();

BufferedInputStream and BufferedReader read ahead and keep 8K of characters in a buffer for performance. Very large files can be processed while only keeping 8K of characters in memory at a time.