[Rd] matrix bands
Dear R developers,
I was looking for a function analogous to base::diag() for getting and
setting bands of a matrix. The closest I could find was Matrix::band(),
but this was not exactly what I wanted for two reasons. Firstly,
Matrix::band() returns a matrix rather than just the specified band.
Secondly, Matrix::band() cannot be used for setting the values for a
matrix band.
Setting or getting a matrix band could be of interest for sufficiently
many users that you might consider including it in base R.
Alternatively, something like this could be included in the Matrix package.
I have included two versions of these functions, a simple and naive
version, and a more efficient version. The band index, n, is positive
for bands above the diagonal and negative for bands below the diagonal.
Regards,
Jeremy Silver
###
## less clear formulation - more efficient
band - function(x,n = 0){
dx - dim(x)
if(length(dx) != 2L)
stop(only matrix bands can be accessed)
max.dx - max(dx)
n - as.integer(n)
ij - cbind(i = seq(1,max.dx) - n,
j = seq(1,max.dx))
ij - ij[1 = ij[,1] ij[,1] = dx[1] 1 = ij[,2] ij[,2] =
dx[2],,drop=FALSE]
if(nrow(ij) == 0)
stop('cannot access this matrix band')
x[ij]
}
'band-' - function(x,n = 0, value){
dx - dim(x)
if(length(dx) != 2L)
stop(only matrix bands can be replaced)
max.dx - max(dx)
n - as.integer(n)
ij - cbind(i = seq(1,max.dx) - n,
j = seq(1,max.dx))
ij - ij[1 = ij[,1] ij[,1] = dx[1] 1 = ij[,2] ij[,2] =
dx[2],,drop=FALSE]
if(nrow(ij) == 0)
stop('cannot replace this matrix band')
x[ij] - value
x
}
## simple, clear formulation - not very efficient
band2 - function(x, n = 0) {
x[col(x) - row(x) == as.integer(n)]
}
'band2-' - function(x, n = 0, value) {
x[which(col(x) - row(x) == as.integer(n))] - value
x
}
## here are some examples to show that it works
## define a test matrix
A - matrix(rnorm(12),3,4)
A
[,1] [,2] [,3] [,4]
[1,] -1.5560200 0.6452762 1.072565 0.1923451
[2,] 0.7940685 1.2441817 1.699486 -0.2998814
[3,] -0.7762252 -0.4824173 -0.981055 -0.9265627
## access some of the bands
band(A,1)
[1] 0.6452762 1.6994858 -0.9265627
band(A,-2)
[1] -0.7762252
band(A,2)
[1] 1.0725649 -0.2998814
## set one of the bands
band(A,2) - 2:1
A
[,1] [,2] [,3] [,4]
[1,] -1.5560200 0.6452762 2.00 0.1923451
[2,] 0.7940685 1.2441817 1.699486 1.000
[3,] -0.7762252 -0.4824173 -0.981055 -0.9265627
## another example - a single column
A - matrix(1:10)
A
[,1]
[1,]1
[2,]2
[3,]3
[4,]4
[5,]5
[6,]6
[7,]7
[8,]8
[9,]9
[10,] 10
band(A,0)
[1] 1
band(A,1)
Error in band(A, 1) : cannot access this matrix band
band(A,-1)
[1] 2
band(A,-5)
[1] 6
## compare the results from the two versions of the function
for(i in -2:3){print(band(A,i));print(band2(A,i))}
[1] -0.7762252
[1] -0.7762252
[1] 0.7940685 -0.4824173
[1] 0.7940685 -0.4824173
[1] -1.556020 1.244182 -0.981055
[1] -1.556020 1.244182 -0.981055
[1] 0.6452762 1.6994858 -0.9265627
[1] 0.6452762 1.6994858 -0.9265627
[1] 2 1
[1] 2 1
[1] 0.1923451
[1] 0.1923451
## show that the naive version is very slow for large matrices
N - 1e4
M - matrix(0,N,N)
system.time(band(M,2))
user system elapsed
0.005 0.003 0.007
system.time(band2(M,2))
user system elapsed
18.509 2.121 20.754
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] methods() not listing some S3 plot methods...?
Dear List, This may be related to this email thread initiated by Ben Bolker last month: https://stat.ethz.ch/pipermail/r-devel/2011-July/061630.html In answering this Question on StackOverflow http://stackoverflow.com/q/7195628/429846 I noticed that `methods()` was not listing some S3 methods for `plot()` provided by the mgcv package. At the time I wanted to check the development version of R as I recalled Uwe mentioning that `plot.function` was listed by `methods()` there but not in R2.13.x. I have now compiled the development version on two Fedora installations and certain plot methods are still not being listed. Details of the exact revision of R Devel are shown at the end of this email. As an example, consider: require(mgcv) Loading required package: mgcv This is mgcv 1.7-6. For overview type 'help(mgcv-package)'. methods(plot) [1] plot.acf* plot.ACF* plot.augPred* [4] plot.compareFits* plot.data.frame* plot.decomposed.ts* [7] plot.default plot.dendrogram* plot.density [10] plot.ecdf plot.factor* plot.formula* [13] plot.function plot.gam plot.gls* [16] plot.hclust* plot.histogram*plot.HoltWinters* [19] plot.intervals.lmList* plot.isoreg* plot.lm [22] plot.lme* plot.lmList* plot.medpolish* [25] plot.mlm plot.nffGroupedData* plot.nfnGroupedData* [28] plot.nls* plot.nmGroupedData*plot.pdMat* [31] plot.ppr* plot.prcomp* plot.princomp* [34] plot.profile.nls* plot.ranef.lme*plot.ranef.lmList* [37] plot.shingle* plot.simulate.lme* plot.spec [40] plot.stepfun plot.stl* plot.table* [43] plot.trellis* plot.tsplot.tskernel* [46] plot.TukeyHSD plot.Variogram* Non-visible functions are asterisked pmeth - methods(plot) grep(plot.mgcv.smooth, pmeth) integer(0) getS3method(plot, mgcv.smooth) Error in getS3method(plot, mgcv.smooth) : S3 method 'plot.mgcv.smooth' not found pfun - getAnywhere(plot.mgcv.smooth) str(pfun) List of 5 $ name : chr plot.mgcv.smooth $ objs :List of 1 ..$ :function (x, P = NULL, data = NULL, label = , se1.mult = 1, se2.mult = 2, partial.resids = FALSE, rug = TRUE, se = TRUE, scale = -1, n = 100, n2 = 40, pers = FALSE, theta = 30, phi = 30, jit = FALSE, xlab = NULL, ylab = NULL, main = NULL, ylim = NULL, xlim = NULL, too.far = 0.1, shade = FALSE, shade.col = gray80, shift = 0, trans = I, by.resids = FALSE, scheme = NULL, ...) $ where : chr namespace:mgcv $ visible: logi FALSE $ dups : logi FALSE - attr(*, class)= chr getAnywhere Both `methods()` and `getS3method()` don't list/find this method, but the function exists in the mgcv name space and this method will be used via R's S3 dispatch system in `plot.gam()`. Shouldn't this method be returned by either `methods()` or `getS3method()`? TIA, Gavin sessionInfo() R Under development (unstable) (2011-08-26 r56801) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_GB.UTF-8LC_COLLATE=en_GB.UTF-8 [5] LC_MONETARY=en_GB.UTF-8LC_MESSAGES=en_GB.UTF-8 [7] LC_PAPER=C LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] mgcv_1.7-6 loaded via a namespace (and not attached): [1] grid_2.14.0lattice_0.19-33Matrix_0.9996875-3 nlme_3.1-102 [5] tools_2.14.0 -- %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% Dr. Gavin Simpson [t] +44 (0)20 7679 0522 ECRC, UCL Geography, [f] +44 (0)20 7679 0565 Pearson Building, [e] gavin.simpsonATNOSPAMucl.ac.uk Gower Street, London [w] http://www.ucl.ac.uk/~ucfagls/ UK. WC1E 6BT. [w] http://www.freshwaters.org.uk %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] matrix bands
Jeremy David Silver j...@dmu.dk
on Fri, 26 Aug 2011 13:23:43 +0200 writes:
Dear R developers, I was looking for a function analogous
to base::diag() for getting and setting bands of a
matrix. The closest I could find was Matrix::band(), but
this was not exactly what I wanted for two
reasons. Firstly, Matrix::band() returns a matrix rather
than just the specified band. Secondly, Matrix::band()
cannot be used for setting the values for a matrix band.
Yes, but did you not look at help(band)
or not look carefully enough ?
---
See Also:
‘bandSparse’ for the _construction_ of a banded sparse matrix
directly from its non-zero diagonals.
Setting or getting a matrix band could be of interest for
sufficiently many users that you might consider including
it in base R. Alternatively, something like this could be
included in the Matrix package.
well, see above and let us know if you see anything lacking in
bandSparse().
Till now we haven't got much feedback about it, and there may
well be room for improvement.
Martin Maechler, ETH Zurich and co- maintainer(Matrix)
I have included two versions of these functions, a simple
and naive version, and a more efficient version. The band
index, n, is positive for bands above the diagonal and
negative for bands below the diagonal.
Regards, Jeremy Silver
###
## less clear formulation - more efficient
band - function(x,n = 0){ dx - dim(x) if(length(dx) !=
2L) stop(only matrix bands can be accessed)
max.dx - max(dx) n - as.integer(n)
ij - cbind(i = seq(1,max.dx) - n, j = seq(1,max.dx))
ij - ij[1 = ij[,1] ij[,1] = dx[1] 1 = ij[,2]
ij[,2] = dx[2],,drop=FALSE]
if(nrow(ij) == 0) stop('cannot access this matrix
band')
x[ij] }
'band-' - function(x,n = 0, value){ dx - dim(x)
if(length(dx) != 2L) stop(only matrix bands can be
replaced)
max.dx - max(dx) n - as.integer(n)
ij - cbind(i = seq(1,max.dx) - n, j = seq(1,max.dx))
ij - ij[1 = ij[,1] ij[,1] = dx[1] 1 = ij[,2]
ij[,2] = dx[2],,drop=FALSE]
if(nrow(ij) == 0) stop('cannot replace this matrix
band')
x[ij] - value
x }
## simple, clear formulation - not very efficient band2 -
function(x, n = 0) { x[col(x) - row(x) == as.integer(n)] }
'band2-' - function(x, n = 0, value) { x[which(col(x) -
row(x) == as.integer(n))] - value x }
## here are some examples to show that it works
## define a test matrix
A - matrix(rnorm(12),3,4) A
[,1] [,2] [,3] [,4] [1,] -1.5560200 0.6452762
1.072565 0.1923451 [2,] 0.7940685 1.2441817 1.699486
-0.2998814 [3,] -0.7762252 -0.4824173 -0.981055 -0.9265627
## access some of the bands
band(A,1)
[1] 0.6452762 1.6994858 -0.9265627
band(A,-2)
[1] -0.7762252
band(A,2)
[1] 1.0725649 -0.2998814
## set one of the bands
band(A,2) - 2:1 A
[,1] [,2] [,3] [,4] [1,] -1.5560200 0.6452762
2.00 0.1923451 [2,] 0.7940685 1.2441817 1.699486
1.000 [3,] -0.7762252 -0.4824173 -0.981055 -0.9265627
## another example - a single column
A - matrix(1:10) A
[,1] [1,] 1 [2,] 2 [3,] 3 [4,] 4 [5,] 5 [6,] 6 [7,]
7 [8,] 8 [9,] 9 [10,] 10
band(A,0)
[1] 1
band(A,1)
Error in band(A, 1) : cannot access this matrix band
band(A,-1)
[1] 2
band(A,-5)
[1] 6
## compare the results from the two versions of the
function
for(i in -2:3){print(band(A,i));print(band2(A,i))}
[1] -0.7762252 [1] -0.7762252 [1] 0.7940685 -0.4824173 [1]
0.7940685 -0.4824173 [1] -1.556020 1.244182 -0.981055 [1]
-1.556020 1.244182 -0.981055 [1] 0.6452762 1.6994858
-0.9265627 [1] 0.6452762 1.6994858 -0.9265627 [1] 2 1 [1]
2 1 [1] 0.1923451 [1] 0.1923451
## show that the naive version is very slow for large
matrices
N - 1e4 M - matrix(0,N,N) system.time(band(M,2))
user system elapsed 0.005 0.003 0.007
system.time(band2(M,2))
user system elapsed 18.509 2.121 20.754
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] matrix bands
Thanks for the suggestion, Martin!
I looked at both Matrix::band and Matrix::bandSparse. Maybe I
misunderstand the help pages and the examples, but from what I can see
neither of them provides the functionality I was looking for.
For the getter version of the function I was looking for, I can't use
Matrix::band (by extracting the non-zero elements) because this would
require the assumption that the specified band is entirely non-zero, e.g.:
band3 - function(x,n){
x - Matrix::band(x,n,n)
x[x!=0] ## works for a dense matrix, but not if band n has zeroes
}
As for the setter version of the function I was looking for, it should
set the values of a specified band in an existing matrix to be the
vector of values provided. As far as I understand, Matrix::bandSparse
constructs a matrix rather than modifying the values of an existing matrix.
The functions included in my last post filled this gap. If Matrix::band
and Matrix::bandSparse can set/get bands in this way, I would like to
see how this can be done. If not, then perhaps they could be extended to
achieve this functionality.
On 2011-08-26 14:08, Martin Maechler wrote:
Jeremy David Silverj...@dmu.dk
on Fri, 26 Aug 2011 13:23:43 +0200 writes:
Dear R developers, I was looking for a function analogous
to base::diag() for getting and setting bands of a
matrix. The closest I could find was Matrix::band(), but
this was not exactly what I wanted for two
reasons. Firstly, Matrix::band() returns a matrix rather
than just the specified band. Secondly, Matrix::band()
cannot be used for setting the values for a matrix band.
Yes, but did you not look at help(band)
or not look carefully enough ?
---
See Also:
‘bandSparse’ for the _construction_ of a banded sparse matrix
directly from its non-zero diagonals.
Setting or getting a matrix band could be of interest for
sufficiently many users that you might consider including
it in base R. Alternatively, something like this could be
included in the Matrix package.
well, see above and let us know if you see anything lacking in
bandSparse().
Till now we haven't got much feedback about it, and there may
well be room for improvement.
Martin Maechler, ETH Zurich and co- maintainer(Matrix)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] methods() not listing some S3 plot methods...?
On 08/26/2011 04:44 AM, Gavin Simpson wrote: Dear List, This may be related to this email thread initiated by Ben Bolker last month: https://stat.ethz.ch/pipermail/r-devel/2011-July/061630.html In answering this Question on StackOverflow http://stackoverflow.com/q/7195628/429846 I noticed that `methods()` was not listing some S3 methods for `plot()` provided by the mgcv package. Hi Gavin -- In the mgcv NAMESPACE, the methods is not registered with S3method (which would have made it appear with a *) and is not export'ed; the author of the package apparently intends that it be strictly internal to the package. Dispatch works within the package name space, but not outside, e.g., a=list; class(a) = mgcv.smooth; plot(a) ends up at plot.default. At the time I wanted to check the development version of R as I recalled Uwe mentioning that `plot.function` was listed by `methods()` there but not in R2.13.x. I have now compiled the development version on two It looks like the cog that has changed between release and devel is the addition of export(plot.function) and S3method(plot, function) to the NAMESPACE of graphics. Martin Fedora installations and certain plot methods are still not being listed. Details of the exact revision of R Devel are shown at the end of this email. As an example, consider: require(mgcv) Loading required package: mgcv This is mgcv 1.7-6. For overview type 'help(mgcv-package)'. methods(plot) [1] plot.acf* plot.ACF* plot.augPred* [4] plot.compareFits* plot.data.frame* plot.decomposed.ts* [7] plot.default plot.dendrogram* plot.density [10] plot.ecdf plot.factor* plot.formula* [13] plot.function plot.gam plot.gls* [16] plot.hclust* plot.histogram*plot.HoltWinters* [19] plot.intervals.lmList* plot.isoreg* plot.lm [22] plot.lme* plot.lmList* plot.medpolish* [25] plot.mlm plot.nffGroupedData* plot.nfnGroupedData* [28] plot.nls* plot.nmGroupedData*plot.pdMat* [31] plot.ppr* plot.prcomp* plot.princomp* [34] plot.profile.nls* plot.ranef.lme*plot.ranef.lmList* [37] plot.shingle* plot.simulate.lme* plot.spec [40] plot.stepfun plot.stl* plot.table* [43] plot.trellis* plot.tsplot.tskernel* [46] plot.TukeyHSD plot.Variogram* Non-visible functions are asterisked pmeth- methods(plot) grep(plot.mgcv.smooth, pmeth) integer(0) getS3method(plot, mgcv.smooth) Error in getS3method(plot, mgcv.smooth) : S3 method 'plot.mgcv.smooth' not found pfun- getAnywhere(plot.mgcv.smooth) str(pfun) List of 5 $ name : chr plot.mgcv.smooth $ objs :List of 1 ..$ :function (x, P = NULL, data = NULL, label = , se1.mult = 1, se2.mult = 2, partial.resids = FALSE, rug = TRUE, se = TRUE, scale = -1, n = 100, n2 = 40, pers = FALSE, theta = 30, phi = 30, jit = FALSE, xlab = NULL, ylab = NULL, main = NULL, ylim = NULL, xlim = NULL, too.far = 0.1, shade = FALSE, shade.col = gray80, shift = 0, trans = I, by.resids = FALSE, scheme = NULL, ...) $ where : chr namespace:mgcv $ visible: logi FALSE $ dups : logi FALSE - attr(*, class)= chr getAnywhere Both `methods()` and `getS3method()` don't list/find this method, but the function exists in the mgcv name space and this method will be used via R's S3 dispatch system in `plot.gam()`. Shouldn't this method be returned by either `methods()` or `getS3method()`? TIA, Gavin sessionInfo() R Under development (unstable) (2011-08-26 r56801) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_GB.UTF-8LC_COLLATE=en_GB.UTF-8 [5] LC_MONETARY=en_GB.UTF-8LC_MESSAGES=en_GB.UTF-8 [7] LC_PAPER=C LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] mgcv_1.7-6 loaded via a namespace (and not attached): [1] grid_2.14.0lattice_0.19-33Matrix_0.9996875-3 nlme_3.1-102 [5] tools_2.14.0 -- Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] matrix bands
Jeremy David Silver j...@dmu.dk
on Fri, 26 Aug 2011 15:08:03 +0200 writes:
Thanks for the suggestion, Martin! I looked at both
Matrix::band and Matrix::bandSparse. Maybe I misunderstand the
help pages and the examples, but from what I can see neither of
them provides the functionality I was looking for.
For the getter version of the function I was looking for, I
can't use Matrix::band (by extracting the non-zero elements)
because this would require the assumption that the specified
band is entirely non-zero, e.g.:
band3 - function(x,n){
x - Matrix::band(x,n,n)
x[x!=0] ## works for a dense matrix, but not if band n has zeroes
}
As for the setter version of the function I was looking for, it should
set the values of a specified band in an existing matrix to be the
vector of values provided. As far as I understand, Matrix::bandSparse
constructs a matrix rather than modifying the values of an existing matrix.
Yes, you are right. The current bandSparse() doesn't do that...
and while you could use constructions like
diag(A[,-1]) - dd
there will probably better (more efficient) alternatives.
The functions included in my last post filled this gap. If Matrix::band
and Matrix::bandSparse can set/get bands in this way, I would like to
see how this can be done. If not, then perhaps they could be extended to
achieve this functionality.
That's a good suggestion, and I'll look at it,
further I'll look at your examples of getting ...
and at the moment I'm still a bit curious why, i.e., to what end
/ in what application you need them [ rather than band(*,.,.) ].
We may e-talk about this off-public if you want.
Martin
On 2011-08-26 14:08, Martin Maechler wrote:
Jeremy David Silverj...@dmu.dk
on Fri, 26 Aug 2011 13:23:43 +0200 writes:
Dear R developers, I was looking for a function analogous
to base::diag() for getting and setting bands of a
matrix. The closest I could find was Matrix::band(), but
this was not exactly what I wanted for two
reasons. Firstly, Matrix::band() returns a matrix rather
than just the specified band. Secondly, Matrix::band()
cannot be used for setting the values for a matrix band.
Yes, but did you not look at help(band)
or not look carefully enough ?
---
See Also:
‘bandSparse’ for the _construction_ of a banded sparse matrix
directly from its non-zero diagonals.
Setting or getting a matrix band could be of interest for
sufficiently many users that you might consider including
it in base R. Alternatively, something like this could be
included in the Matrix package.
well, see above and let us know if you see anything lacking in
bandSparse().
Till now we haven't got much feedback about it, and there may
well be room for improvement.
Martin Maechler, ETH Zurich and co- maintainer(Matrix)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] read.table segfaults
fil2s - read.table(../Data/fil2_s.txt, header = FALSE, sep = \t) Program received signal SIGSEGV, Segmentation fault. 0x0041c2e1 in RunGenCollect (size_needed=8192000) at memory.c:1514 1514PROCESS_NODES(); (gdb) sessionInfo() R version 2.13.1 Patched (2011-08-25 r56798) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base The text file 'fil2_s.txt' is Huge, around 11 million records and 17 variables, but ...? -- Göran Broström __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table segfaults
Another one: The 'death.RData' was created about a year ago, but ...? Same info as below. Göran load(../Data/death.RData) summary(death) *** caught segfault *** address 0x4e04959, cause 'memory not mapped' Traceback: 1: match(x, levels) 2: factor(a, levels = ll[!(ll %in% exclude)], exclude = if (useNA == no) NA) 3: table(object) 4: summary.factor(X[[6L]], ...) 5: FUN(X[[6L]], ...) 6: lapply(as.list(object), summary, maxsum = maxsum, digits = 12, ...) 7: summary.data.frame(death) 8: summary(death) Possible actions: 1: abort (with core dump, if enabled) 2: normal R exit 3: exit R without saving workspace 4: exit R saving workspace Selection: 2011/8/26 Göran Broström goran.brost...@gmail.com: fil2s - read.table(../Data/fil2_s.txt, header = FALSE, sep = \t) Program received signal SIGSEGV, Segmentation fault. 0x0041c2e1 in RunGenCollect (size_needed=8192000) at memory.c:1514 1514 PROCESS_NODES(); (gdb) sessionInfo() R version 2.13.1 Patched (2011-08-25 r56798) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base The text file 'fil2_s.txt' is Huge, around 11 million records and 17 variables, but ...? -- Göran Broström -- Göran Broström __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table segfaults
One further note: No problem with R version 2.13.0 (2011-04-13) Göran 2011/8/26 Göran Broström goran.brost...@gmail.com: Another one: The 'death.RData' was created about a year ago, but ...? Same info as below. Göran load(../Data/death.RData) summary(death) *** caught segfault *** address 0x4e04959, cause 'memory not mapped' Traceback: 1: match(x, levels) 2: factor(a, levels = ll[!(ll %in% exclude)], exclude = if (useNA == no) NA) 3: table(object) 4: summary.factor(X[[6L]], ...) 5: FUN(X[[6L]], ...) 6: lapply(as.list(object), summary, maxsum = maxsum, digits = 12, ...) 7: summary.data.frame(death) 8: summary(death) Possible actions: 1: abort (with core dump, if enabled) 2: normal R exit 3: exit R without saving workspace 4: exit R saving workspace Selection: 2011/8/26 Göran Broström goran.brost...@gmail.com: fil2s - read.table(../Data/fil2_s.txt, header = FALSE, sep = \t) Program received signal SIGSEGV, Segmentation fault. 0x0041c2e1 in RunGenCollect (size_needed=8192000) at memory.c:1514 1514 PROCESS_NODES(); (gdb) sessionInfo() R version 2.13.1 Patched (2011-08-25 r56798) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base The text file 'fil2_s.txt' is Huge, around 11 million records and 17 variables, but ...? -- Göran Broström -- Göran Broström -- Göran Broström __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table segfaults
It does look like you've got a memory issue. perhaps using as.is=TRUE, and/or stringsAsFactors=FALSE will help as optional arguments to read.table if you don't specify these sorts of things, R can have to look through the file and figure out which columns are characters/factors etc and so the larger files cause more of a headache for R I'm guess. Hopefully someone else can comment further on this? I'd true toggling TRUE/FALSE for as.is and stringsAsFactors. do you have other objects loaded in memory as well? this file by itself might not be the problem - but it's a cumulative issue. have you checked the file structure in any other manner? how large (Mb/kb) is the file that you're trying to read? if you just read in parts of the file, is it okay? read.table(filename,header=FALSE,sep=\t,nrows=100) read.table(filename,header=FALSE,sep=\t,skip=2,nrows=100) -- View this message in context: http://r.789695.n4.nabble.com/read-table-segfaults-tp3771793p3771817.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Problem in calling R functions from Matlab
Hi,
I wish to use R (version 2.13.1) from within Matlab(ver R2009a) on windows XP
plaform (on both 64 bit and 32 bit OS) . For this I have installed
StatConnector
(http://rcom.univie.ac.at/download/current/statconnDCOM.latest.exe) for calling
R from within Matlab (R2009a) on Windows XP platform. I have added all the
files of MATLAB_RLINK folder (downloaded from
http://www.mathworks.com/matlabcentral/fileexchange/5051) into the
default Matlab working path (D:\User\MATLAB) and also added the folder
MATLAB_RLINK to the matlab path C:\Program Files\MATLAB\R2009a\toolbox . It
seems that Matlab is able to connect to R however I am facing a problem while
trying to call R functions from within Matlab. I have loaded all required
libraries. Shown below is the sequence of commands tried in blue, outcomes in
black and errors/unexpected/ undesired outcomes in red colour, and my comments
in black,italic after the command:
[a b c] = openR
a = 1
b = ''
c = COM.StatConnectorSrv_StatConnector
Rdemo
b = 1 4 9 16 25 36 49 64 81 100
c = 2 5 10 17 26 37 50 65 82 101
I then tried running the Rdemo available on
http://www.mathworks.com/matlabcentral/fx_files/5051/1/content/Rdemo.html
evalR('demo(persp)') works well. Also the arithmetic functions all work well:
a = 1:10;
putRdata('a',a)
b = evalR('a^2')
evalR('b - a^2');
evalR('c - b + 1');
c = getRdata('c')
However Now copy the volcano data into MATLAB section and all related
sessions therefore dont work.
volcano = getRdata('volcano')
volcano = []
size(volcano)
ans = 0 0
surf(volcano);
(see the attached figure:surf_volcano_output.jpg)
It seems Matlab is able to read/get data from R. Other subsequent commands
related to this section dont work.
I tried looking at the loaded libraries in R and in Matlab. In R everything
works well, but doesnt seem to be so in Matlab:
[a b c] = evalR('.libPaths()')
a = C:/Program Files/R/R-2.13.1/library
b = 1
c = ''
[a b c] = evalR('library()')
a = 'ActiveX VT_ERROR: '
'base'
'ActiveX VT_ERROR: '
b = 1
c = ''
[a b c] = evalR('library(rgdal)')
a = 'rscproxy'
'lattice'
'fields'
'spam'
'rgdal'
'sp'
'stats'
'graphics'
'grDevices'
'utils'
'datasets'
'methods'
'base'
b = 1
c = ''
As it shows that rgdal library is loaded, also the rscproxy library is loaded,
in addition to all others. I tried following:
imshow(imread('testimg_p.tif')) displays the image properly, however calling
this from R using rgdal from within Matlab gives error;
[a b c] = evalR('img - readGDAL(testimg_p.tif)')
a = []
b = 0
c = Invoke Error, Dispatch Exception: Object is static; operation not allowed
The command works from R prompt:
img - readGDAL(testimg_p.tif)
testimg_p.tif has GDAL driver GTiff
and has 280 rows and 272 columns
It is giving same error while loading gstat library:
[a b c] = evalR('library(gstat) ')
a = []
b = 0
c = Invoke Error, Dispatch Exception: Object is static; operation not allowed
I guess the error is not related to readGDAL or gstat library but something
else which I am unable to trace. I have also tried to explore if this is
problem related to version of Matlab/windows 64/32 bit or R version but even
that does not seem to be an issue. (I have tried this on two differemt machines
one Workstation with WinXP 64 bit OS with R version 2.13.1and Matlab Version
7.8.0 R2009a and also on another portable workstation with WinXP 32 bit OS with
R version 2.12.1 and Matlab Version 7.8.0 R2009a. This indicates that version
of R or Matlab or win version 64 bit/32 bit does not seem to be an issue.
I also checked for some solution on page
http://www.mathworks.com/matlabcentral/fileexchange/5051 where others also
faced somewhat similar problem and it was suggested to check for setting R_HOME
and PATH variables in the environment variables for your system. I understand I
had followed all instructions carefully and hopefully all paths/R_HOME were set
correctly, as the command evalR('.libPaths()') recognized R home path
correctly.
I request R team to help.
Vandita Srivastava
Scientist/Engineer SE,
Indian Institute of Remote Sensing(NRSC),
ISRO/Department of Space, Govt. of India
Please don't print this Email unless you really need to - this will preserve
trees
on planet earth.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] S3 methods in default namespace
Hi guys, Are there any plans for figuring out potential S3 methods and declaring them with S3method() in the automatic default NAMESPACE? Thanks, Michael [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table segfaults
Scott ncbi2r at googlemail.com writes: It does look like you've got a memory issue. perhaps using as.is=TRUE, and/or stringsAsFactors=FALSE will help as optional arguments to read.table if you don't specify these sorts of things, R can have to look through the file and figure out which columns are characters/factors etc and so the larger files cause more of a headache for R I'm guess. Hopefully someone else can comment further on this? I'd true toggling TRUE/FALSE for as.is and stringsAsFactors. do you have other objects loaded in memory as well? this file by itself might not be the problem - but it's a cumulative issue. have you checked the file structure in any other manner? how large (Mb/kb) is the file that you're trying to read? if you just read in parts of the file, is it okay? read.table(filename,header=FALSE,sep=\t,nrows=100) read.table(filename,header=FALSE,sep=\t,skip=2,nrows=100) There seem to be two issues here: 1. what can the original poster (OP) do to work around this problem? (e.g. get the data into a relational data base and import it from there; use something from the High Performance task view such as ff or data.table ...) 2. reporting a bug -- according to the R FAQ, any low-level (segmentation-fault-type) crash of R when one is not messing around with dynamically loaded code constitutes a bug. Unfortunately, debugging problems like this is a huge pain in the butt. Goran, can you randomly or systematically generate an object of this size, write it to disk, read it back in, and generate the same error? In other words, does something like set.seed(1001) d - data.frame(label=rep(LETTERS[1:11],1e6), values=matrix(rep(1.0,11*17*1e6),ncol=17) write.table(d,file=big.txt) read.table(big.txt) do the same thing? Reducing it to this kind of reproducible example will make it possible for others to debug it without needing to gain access to your huge file ... __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] methods() not listing some S3 plot methods...?
On Fri, 2011-08-26 at 07:06 -0700, Martin Morgan wrote: On 08/26/2011 04:44 AM, Gavin Simpson wrote: Dear List, This may be related to this email thread initiated by Ben Bolker last month: https://stat.ethz.ch/pipermail/r-devel/2011-July/061630.html In answering this Question on StackOverflow http://stackoverflow.com/q/7195628/429846 I noticed that `methods()` was not listing some S3 methods for `plot()` provided by the mgcv package. Hi Gavin -- In the mgcv NAMESPACE, the methods is not registered with S3method (which would have made it appear with a *) and is not export'ed; the author of the package apparently intends that it be strictly internal to the package. Dispatch works within the package name space, but not outside, e.g., a=list; class(a) = mgcv.smooth; plot(a) ends up at plot.default. Thanks for the explanation Martin. This strikes me as being somewhat suboptimal and not conducive to studying and understanding code. Now that I am aware of the distinction I won't be surprised when methods don't show up. Cheers G At the time I wanted to check the development version of R as I recalled Uwe mentioning that `plot.function` was listed by `methods()` there but not in R2.13.x. I have now compiled the development version on two It looks like the cog that has changed between release and devel is the addition of export(plot.function) and S3method(plot, function) to the NAMESPACE of graphics. Martin Fedora installations and certain plot methods are still not being listed. Details of the exact revision of R Devel are shown at the end of this email. As an example, consider: require(mgcv) Loading required package: mgcv This is mgcv 1.7-6. For overview type 'help(mgcv-package)'. methods(plot) [1] plot.acf* plot.ACF* plot.augPred* [4] plot.compareFits* plot.data.frame* plot.decomposed.ts* [7] plot.default plot.dendrogram* plot.density [10] plot.ecdf plot.factor* plot.formula* [13] plot.function plot.gam plot.gls* [16] plot.hclust* plot.histogram*plot.HoltWinters* [19] plot.intervals.lmList* plot.isoreg* plot.lm [22] plot.lme* plot.lmList* plot.medpolish* [25] plot.mlm plot.nffGroupedData* plot.nfnGroupedData* [28] plot.nls* plot.nmGroupedData*plot.pdMat* [31] plot.ppr* plot.prcomp* plot.princomp* [34] plot.profile.nls* plot.ranef.lme*plot.ranef.lmList* [37] plot.shingle* plot.simulate.lme* plot.spec [40] plot.stepfun plot.stl* plot.table* [43] plot.trellis* plot.tsplot.tskernel* [46] plot.TukeyHSD plot.Variogram* Non-visible functions are asterisked pmeth- methods(plot) grep(plot.mgcv.smooth, pmeth) integer(0) getS3method(plot, mgcv.smooth) Error in getS3method(plot, mgcv.smooth) : S3 method 'plot.mgcv.smooth' not found pfun- getAnywhere(plot.mgcv.smooth) str(pfun) List of 5 $ name : chr plot.mgcv.smooth $ objs :List of 1 ..$ :function (x, P = NULL, data = NULL, label = , se1.mult = 1, se2.mult = 2, partial.resids = FALSE, rug = TRUE, se = TRUE, scale = -1, n = 100, n2 = 40, pers = FALSE, theta = 30, phi = 30, jit = FALSE, xlab = NULL, ylab = NULL, main = NULL, ylim = NULL, xlim = NULL, too.far = 0.1, shade = FALSE, shade.col = gray80, shift = 0, trans = I, by.resids = FALSE, scheme = NULL, ...) $ where : chr namespace:mgcv $ visible: logi FALSE $ dups : logi FALSE - attr(*, class)= chr getAnywhere Both `methods()` and `getS3method()` don't list/find this method, but the function exists in the mgcv name space and this method will be used via R's S3 dispatch system in `plot.gam()`. Shouldn't this method be returned by either `methods()` or `getS3method()`? TIA, Gavin sessionInfo() R Under development (unstable) (2011-08-26 r56801) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_GB.UTF-8LC_COLLATE=en_GB.UTF-8 [5] LC_MONETARY=en_GB.UTF-8LC_MESSAGES=en_GB.UTF-8 [7] LC_PAPER=C LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] mgcv_1.7-6 loaded via a namespace (and not attached): [1] grid_2.14.0lattice_0.19-33Matrix_0.9996875-3 nlme_3.1-102 [5] tools_2.14.0 -- %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% Dr. Gavin Simpson [t] +44 (0)20 7679 0522 ECRC, UCL Geography, [f] +44 (0)20 7679 0565 Pearson Building,
