matrix - System is computationally singular: reciprocal condition number in R

Question

x <- matrix(rnorm(80, mean = 0, sd = 0.1), 8, 8)
c <- cov(x)
solve(c)

I get the error message:

Error in solve.default(c) : system is computationally singular: reciprocal condition number = 6.57889e-18

I have been trying to figure out what is the reason behind the problem, and other threads at Stack Overflow have suggested the issue might be due to singular matrices, highly correlated variables, linear combination etc. However, I assumed that rnorm would avoid the mentioned problems.

For another matrix that I am working with det() gives 8.313969e-95, but it is still invertible with solve().

Answer 1

Two fundamental linear algebra properties:

1. A singular (square) matrix is a (square) matrix that is not invertible.
2. A matrix is not invertible if its determinant equals zero.

If you check

set.seed(2018);
x <- matrix(rnorm(80, mean = 0, sd = 0.1), 8, 8)
c <- cov(x)
det(c)
#[1] -3.109158e-38

So indeed, det(c) is zero (within machine precision); hence c is not invertible, which is exactly what solve(c) is trying to do.

PS 1: Take a look at ?solve to see that solve(a) will return the inverse of a.
PS 2: There exists a nice post on Cross Validated on the interpretation of the determinant of the covariance matrix. Take a look to understand why you're seeing what you're seeing.