Re: [R] Possible bug of QR decomposition in package Matrix
Thank you, Martin.
Yes, on my computer (48Gb memory) it also gives a message "caught
segfault" as you got. So it is because the matrix size is too large.
I apologize for the offence I've caused.
2011/8/4 Martin Maechler :
>>>>>> C6H5NO2
>>>>>> on Thu, 4 Aug 2011 16:03:54 +0800 writes:
>
> > Thank you very much, Josh!
> > As you suggested, I will contact the developers of "Matrix".
>
> > PS, C6 are just initial characters of my email account :-)
>
> > Best wishes,
> > C6
>
> well, as the posting guide (http://www.R-project.org/posting-guide.html)
> says, this is regarded as impolite by many,
> and if I wasn't one of the Matrix package authors,
> I would not spend time helping 'C6' either.
>
> > 2011/8/4 Joshua Wiley :
> >> Hi C6 (were C1 - 5 already taken in your family?),
> >>
> >> I downloaded your data and can replicate your problem. R
> >> ceases responding and terminates. This does not occur with all
> >> uses of qr on a dgCMatrix object. I know nothing about sparse
> >> matrices, but if you believe this should not be occurring, you
> >> should contact the package maintainers. Here is my
> >> sessionInfo() (FYI, it would probably be helpful to report
> >> yours also in case the issue is version dependent):
> >>
> >> R Under development (unstable) (2011-07-30 r56564) Platform:
> >> x86_64-pc-mingw32/x64 (64-bit)
> >>
> >> locale: [1] LC_COLLATE=English_United States.1252 [2]
> >> LC_CTYPE=English_United States.1252 [3]
> >> LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5]
> >> LC_TIME=English_United States.1252
> >>
> >> attached base packages: [1] stats graphics grDevices utils
> >> datasets methods base
> >>
> >> other attached packages: [1] Matrix_0.999375-50 lattice_0.19-30
> >>
> >> loaded via a namespace (and not attached): [1] grid_2.14.0
> >> tools_2.14.0
> >>
> >> Cheers,
> >>
> >> Josh
> >>
> >> On Wed, Aug 3, 2011 at 4:26 PM, C6H5NO2
> >> wrote:
> >>> Hello R users,
> >>>
> >>> I am trying to give the QR decomposition for a large sparse
> >>> matrix in the format of dgCMatrix. When I run qr function for
> >>> this matrix, the R session simply stops and exits to the
> >>> shell. The matrix is of size 108595x108595, and it has
> >>> 4866885 non-zeros. I did the experiment on windows 7 and linux
> >>> mint 11 (both 64 bit), and the results are the same.
> >>>
> >>> I have uploaded my data file to
> >>> http://ifile.it/elf2p6z/A.RData . The file is 10.681 MB and I
> >>> hope someone could kindly download it. The code to see my
> >>> problem is:
>
> >>> library(Matrix)
> >>> load("A.RData")
> >>> B <- qr(A)
>
>
> >>> Best wishes, C6
>
> And what's the size of RAM your two computers have ??
> The answer is of quite some importance.
>
> Short answer: If you have a large very sparse matrix,
> you don't know if the QR decomposition of that matrix is also
> very sparse... and if it ain't it will blow up memory,
> and that's what I'm pretty sure happened with you.
>
> What I don't see is why R "simply stops" for you and does not
> through a an error message about insufficient memory.
> As I show below, I do get a seg.fault
> --- which may be considered a bug ---
> *BUT* I do get the message about memory problems.
> Did you really *not* get any such message?
> Is it because you've used a GUI that hides such valuable
> information from the user?
>
> Here's the more detailed reason / analysis about why the above
> "kills R". This is commented R code,
> you can cut paste after you've got 'A' :
>
> str(A)
> ## Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
> ## ..@ i : int [1:4866885] 0 1 2 16 32 33 2392 2417 0 1 ...
> ## ..@ p : int [1:108596] 0 8 21 35 44 51 59 63 69 78 ...
> ## ..@ Dim : int [1:2] 108595 108595
> ## ..@ Dimnames:List of 2
> ## .. ..$ : NULL
> ## .. ..$ : NULL
> ## ..@ x : num [1:4866885] 140.03 14.79 14.79 1.78 1.78 ...
> ## ..@ factors : list()
>
> system.time(# the following is still not as fast a
Re: [R] Possible bug of QR decomposition in package Matrix
Thank you very much, Josh!
As you suggested, I will contact the developers of "Matrix".
PS, C6 are just initial characters of my email account :-)
Best wishes,
C6
2011/8/4 Joshua Wiley :
> Hi C6 (were C1 - 5 already taken in your family?),
>
> I downloaded your data and can replicate your problem. R ceases
> responding and terminates. This does not occur with all uses of qr on
> a dgCMatrix object. I know nothing about sparse matrices, but if you
> believe this should not be occurring, you should contact the package
> maintainers. Here is my sessionInfo() (FYI, it would probably be
> helpful to report yours also in case the issue is version dependent):
>
> R Under development (unstable) (2011-07-30 r56564)
> Platform: x86_64-pc-mingw32/x64 (64-bit)
>
> locale:
> [1] LC_COLLATE=English_United States.1252
> [2] LC_CTYPE=English_United States.1252
> [3] LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> other attached packages:
> [1] Matrix_0.999375-50 lattice_0.19-30
>
> loaded via a namespace (and not attached):
> [1] grid_2.14.0 tools_2.14.0
>
> Cheers,
>
> Josh
>
> On Wed, Aug 3, 2011 at 4:26 PM, C6H5NO2 wrote:
>> Hello R users,
>>
>> I am trying to give the QR decomposition for a large sparse matrix in
>> the format of dgCMatrix. When I run qr function for this matrix, the R
>> session simply stops and exits to the shell.
>> The matrix is of size 108595x108595, and it has 4866885 non-zeros. I
>> did the experiment on windows 7 and linux mint 11 (both 64 bit), and
>> the results are the same.
>>
>> I have uploaded my data file to http://ifile.it/elf2p6z/A.RData . The
>> file is 10.681 MB and I hope someone could kindly download it.
>> The code to see my problem is:
>> library(Matrix)
>> load("A.RData")
>> B <- qr(A)
>>
>> Best wishes,
>> C6
>>
>> __
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
>
> --
> Joshua Wiley
> Ph.D. Student, Health Psychology
> Programmer Analyst II, ATS Statistical Consulting Group
> University of California, Los Angeles
> https://joshuawiley.com/
>
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Possible bug of QR decomposition in package Matrix
Hello R users,
I am trying to give the QR decomposition for a large sparse matrix in
the format of dgCMatrix. When I run qr function for this matrix, the R
session simply stops and exits to the shell.
The matrix is of size 108595x108595, and it has 4866885 non-zeros. I
did the experiment on windows 7 and linux mint 11 (both 64 bit), and
the results are the same.
I have uploaded my data file to http://ifile.it/elf2p6z/A.RData . The
file is 10.681 MB and I hope someone could kindly download it.
The code to see my problem is:
library(Matrix)
load("A.RData")
B <- qr(A)
Best wishes,
C6
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] columnwise nnzero for dgCMatrix
Hi R users,
I want to know whether there is a fast method to compute the nonzeroes for
each column of dgCMatrix.
For summation or average I can use colSums or colMeans. To count the
non-zeroes I write a function
colCounts <- function(Mat) {
M1 <- apply(Mat, MARGIN=2, FUN=nnzero)
}
But it seems this is quite slow because the apply function would transform a
dgCMatrix to a matrix.
So I come here to ask if there is anybody know some fast method.
--
View this message in context:
http://r.789695.n4.nabble.com/columnwise-nnzero-for-dgCMatrix-tp3625726p3625726.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
