Re: [R] Correlation matrix removing insignificant R values

2011-11-23 Thread Frank Harrell
I think it would be better to think of this as an estimation problem rather than a selection problem. If the correlation matrix is of interest, estimate the entire matrix. If you want to show that you can make decisions on the basis of the matrix, then use the bootstrap to get a confidence

[R] Correlation matrix removing insignificant R values

2011-11-23 Thread mgranlie
Hello. I have a large dataset with sales pr month for 56 products with 10 months and i have tried to see how the sales are correlated using cor() This has given me a 56X56 matrix with the R value for each product pair. Most of these correlations are insignificant, and i want only to retain the

Re: [R] Correlation matrix removing insignificant R values

2011-11-23 Thread R. Michael Weylandt
There have been two threads dealing with this in the last few weeks: please search the recent archives for those threads for a good discussion -- end result: Josh Wiley provided a useful little function to do so that I'll copy below. RSeek.org is a good place to do your searching. spec.cor -

Re: [R] Correlation matrix removing insignificant R values

2011-11-23 Thread R. Michael Weylandt
Looking over the code below, I think this patched version might return a better answer: spec.cor - function(dat, r, ...) { x - cor(dat, ...) x[upper.tri(x, TRUE)] - NA i - which(abs(x) = r, arr.ind = TRUE) data.frame(V1 = rownames(x)[i[,1]], V2 = colnames(x)[i[,2]], Value = x[i])