c++ - ifstream:: What is the maximum file size that a ifstream can read

Question

I tried to read a 3GB data file using ifstream and it gives me wrong file size, whereas when I read a 600MB file, it gives me the correct result. In addition to wrong file size, I am also unable to read the entire file using ifstream.

Here is the code that I used

        std::wstring name;
        name.assign(fileName.begin(), fileName.end());
        __stat64 buf;
        if (_wstat64(name.c_str(), &buf) != 0)
            std::cout << -1; // error, could use errno to find out more

        std::cout << " Windows file size : " << buf.st_size << std::endl;;


        std::ifstream fs(fileName.c_str(), std::ifstream::in | std::ifstream::binary);
        fs.seekg(0, std::ios_base::end);

        std::cout << " ifstream  file size: " << fs.tellg() << std::endl;

The output for 3GB file was

 Windows file size : 3147046042
 ifstream  file size: -1147921254

Whereas the output for 600 MB file was

 Windows file size : 678761111
 ifstream  file size: 678761111

Just in case, I also tested for 5GB file and 300 MB file,

The output for 5GB file was

Windows file size : 5430386900
 ifstream  file size: 1135419604

The output for 300MB file was

Windows file size : 318763632
 ifstream  file size: 318763632

It looks to me like it is reaching some limit.

I am testing the code using Visual Studio 2010 on a Windows Machine which has plenty of memory and disk space.

I am trying to read some large files. Which is a good stream reader to use if ifstream can't read large files?

Answer 1

I think you want to say:

std::cout << " ifstream  file size: " << fs.tellg().seekpos() << std::endl;

At least that works correctly for a 6GB file I have laying around. But I'm compiling with Visual Studio 2012. And even your original code works fine too on that environment.

So I suspect that this is an bug in the std library on VS 2010 that got fixed in VS 2012. Whether it's a bug in the operator overloading for the pos_type or if that class isn't 64-bit aware is unknown. I'd have to install VS 2010 to validate, but that is likely the problem.