R combn fails when asked for large number of combinations (even though memory should suffice)

Question

I wanted to get all the combinations for 73,000 choose 2 and I tried to use combn in order to calculate it.

combn(73000,2)

I received the following error:

Error in matrix(r, nrow = len.r, ncol = count) :
invalid 'ncol' value (too large or NA)

I figured that the number of combinations is 2,664,463,500 so multiplied by 8 should yield around 22GB which I had free on my machine. So even though it's a lot of combinations, it shouldn't fail. Any alternative way to calculate the number of combinations or explanations of why combn fails?

question from:https://stackoverflow.com/questions/66050972/r-combn-fails-when-asked-for-large-number-of-combinations-even-though-memory-sh

Answer 1

I dug into the code and apparently when constructing the output matrix the dimensions are converted to integer:

count <- as.integer(round(choose(n, m)))
out <- matrix(r, nrow = len.r, ncol = count) # matrix for now

Eliminating the as.integer from count increases its range and in my case it doesn't overflow anymore. It wasn't enough though, I continue to receive the same error again. I couldn't find a way to initialize the matrix as type integer, so I created two vectors instead (m=2)like that:

col1 <- vector(mode="integer", length=n)
col2 <- vector(mode="integer", length=n)

With few more adjustments it runs now.

I hope this might help to others as well.