stream - Buffer entire file in memory with Node.js

Question

I have a relatively small file (some hundreds of kilobytes) that I want to be in memory for direct access for the entire execution of the code.

I don't know exactly the internals of Node.js, so I'm asking if a fs open is enough or I have to read all file and copy to a Buffer?

question from:https://stackoverflow.com/questions/19705972/buffer-entire-file-in-memory-with-node-js

Answer 1

Basically, you need to use the readFile or readFileSync function from the fs module. They return the complete content of the given file, but differ in their behavior (asynchronous versus synchronous).

If blocking Node.js (e.g. on startup of your application) is not an issue, you can go with the synchronized version, which is as easy as:

var fs = require('fs');

var data = fs.readFileSync('/etc/passwd');

If you need to go asynchronous, the code is like that:

var fs = require('fs');

fs.readFile('/etc/passwd', function (err, data ) {
  // ...
});

Please note that in either case you can give an options object as the second parameter, e.g. to specify the encoding to use. If you omit the encoding, the raw buffer is returned:

var fs = require('fs');

fs.readFile('/etc/passwd', { encoding: 'utf8' }, function (err, data ) {
  // ...
});

Valid encodings are utf8, ascii, utf16le, ucs2, base64 and hex. There is also a binary encoding, but it is deprecated and should not be used any longer. You can find more details on how to deal with encodings and buffers in the appropriate documentation.