memory management - how to declare 10000 x 100000 sized integer matrix in C++

Question

I need to create a matrix which size is 10000x100000. My RAM is 4GB. It works till the 25th iteration (debug), but after 25th iteration I get an error "bad allocation" however only 25% of RAM is used which means the problem is not related with memory. So what can I do?

EDIT:

int **arr;
arr=new int*[10000];
for(i=0;i<10000;i++)
    arr[i]=new int[100000];

My allocation is above.

Answer 1

If you're compiling for x64, you shouldn't have any problems.

If you're compiling for x86 (most likely), you can enable the /LARGEADDRESSAWARE linker flag if you're using Visual C++, or something similar for other compilers. For Visual C++, the option can also be found in the Linker -> System -> Enable Large Addresses property in the IDE.

This sets a flag in the resulting EXE file telling the OS that the code can handle addresses over 2 GB. When running such an executable on x64 Windows (your case), the OS gives it 4 GB of address space to play with, as opposed to just 2 GB normally.

I tested your code on my system, Windows 7 x64, 8 GB, compiled with Visual C++ Express 2013 (x86, of course) with the linker flag, and the code ran fine - allocated almost 4 GB with no error.

Anyway, the 25th iteration is far too quick for it to fail, regardless of where it runs and how it's compiled (it's roughly 10 MB), so there's something else going wrong in there.

By the way, the HEAP linker option doesn't help in this case, as it doesn't increase the maximum heap size, it just specifies how much address space to reserve initially and in what chunks to increase the amount of committed RAM. In short, it's mostly for optimization purposes.