On 31/10/2009 10:18 PM, mark.braving...@csiro.au wrote:
Does this happen in R-patched? I've seen similar errors in 2.10.0, but
not in a current build.
Yes, still there in R-patched.
(Still haven't got to your code, this was in
mine. I'm reluctant to spend time on code that is messing with
internals, because you might be using things in a way not intended by
the author. Now, if you can show me some code that demonstrates the
problem without using internals directly, I'll follow up.)
I did try, but it's not completely possible, because 'makeLazyLoadDB' is internal and
there is no public alternative (a pity-- it's useful). However, the problem(s) can be
demonstrated without directly calling 'parse_Rd', and with 'lazyLoad' (public) instead of
'fetchRdDB' (private), as per "pointer 1" below. If you have a look at
'tools:::.install_package_Rd_objects', you'll see that my use of 'makeLazyLoadDB' is
quite standard.
The problem is not easy to reproduce. It took 4-5 hours work to get the 3-line
reproducible example that I posted, plus another couple since, so I'm also
reluctant to spend more time...
Okay, then we both agree we should drop it.
Duncan Murdoch
The examples in my previous post still apply-- the first one involves just 3
statements-- but here are some more pointers I've unearthed since:
1. Sometimes 'fetchRdDB' or 'lazyLoad' called directly from the prompt doesn't work, but
public 'Rd_db' (which directly calls 'fetchRdDB') does. I've experimented with copying
the installed 'tools' package into a new library "d:/temp/fakelib", then stuff
like this:
test> e <- new.env()
test> lazyLoad( 'd:/temp/fakelib/tools/help/tools', e) # original files
tools.rdx, tools.rdb
test> e <- as.list( e) # force evaluation
test> tools:::makeLazyLoadDB( e, 'd:/temp/fakelib/tools/help/tools') # modify
tools.rd*
test> e1 <- new.env()
test> lazyLoad( 'd:/temp/fakelib/tools/help/tools', e1)
test> as.list( e1) # try to force evaluation...
Error in as.list.environment(e1) :
cannot allocate memory block of size 2.7 Gb
test>
test> Rd_db( 'tools', 'd:/temp/fakelib') # no probs !?
2. Sometimes 'fetchRdDB' or 'lazyLoad' will fail in one R session, but will
work in a fresh session on exactly the same files. For example, after
restarting R, the previous commands involving 'e1' work fine.
Mark
Duncan Murdoch
I'm encountering problems when making lazy-loadable databases of the output
from 'parse_Rd'. The lazy-
load database is of seemingly limitless size when I try to reload it...
Admittedly I am using functions
that I'm not really supposed to use, which is why this isn't a bug report, but
there does seem to be
something strange going on; my code is very similar to code that lives inside
'tools:::.install_package_Rd_objects'. The problems occur with just-released
R2.10.0 on Windows.
object.size() has problems when working on Rd objects, because it counts
every environment separately, even though they may all be references to
the same one. I haven't looked at your code, but that could be a problem.
Mark
________________________________________
From: Duncan Murdoch [murd...@stats.uwo.ca]
Sent: 31 October 2009 08:59
To: Bravington, Mark (CMIS, Hobart)
Cc: r-devel@r-project.org
Subject: Re: [Rd] parse_Rd and/or lazyload problem
On 30/10/2009 12:50 AM, mark.braving...@csiro.au wrote:
I'm encountering problems when making lazy-loadable databases of the output
from 'parse_Rd'. The lazy-load database is of seemingly limitless size when I
try to reload it... Admittedly I am using functions that I'm not really
supposed to use, which is why this isn't a bug report, but there does seem to
be something strange going on; my code is very similar to code that lives
inside 'tools:::.install_package_Rd_objects'. The problems occur with
just-released R2.10.0 on Windows.
object.size() has problems when working on Rd objects, because it counts
every environment separately, even though they may all be references to
the same one. I haven't looked at your code, but that could be a problem.
Duncan Murdoch
The examples below use files which can be found at ftp://ftp.csiro.au/MarkBravington, but you'll obviously need to
modify the paths. The file "scrunge.Rd" is just "Rdiff.Rd" from the 'tools' package. The file
"fakepack.7z" should unzip to create a fake package with a DESCRIPTION file and a "man" directory
that contains two Rd files.
Transcript of first example:
eglist <- list( scrunge=tools:::prepare_Rd( 'd:/temp/scrunge.Rd',
defines=.Platform$OS.type, stages='install', warningCalls=FALSE))
tools:::makeLazyLoadDB( eglist, 'd:/temp/ll', compress=FALSE)
tools:::fetchRdDB( 'd:/temp/ll')
# Error: cannot allocate vector of size 1.4 Gb
The second example triggers an error with 'tools:::.install_package_Rd_objects'
itself. It doesn't happen the first time, but frequently does after the Rd
files have been changed. Transcript:
# Make sure d:/temp/help/ is empty, then
test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
test> tools:::fetchRdDB( 'd:/temp/help/temp')
# All good. Next, I *removed* one of the two Rd files in
"d:/temp/fakepack/man", ...
# ...deleted "d:/temp/help/temp*", and tried again
test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
test> tools:::fetchRdDB( 'd:/temp/help/temp')
Warning: Reached total allocation of 1535Mb: see help(memory.size)
Warning: Reached total allocation of 1535Mb: see help(memory.size)
Warning: Reached total allocation of 1535Mb: see help(memory.size)
Warning: Reached total allocation of 1535Mb: see help(memory.size)
Error: cannot allocate vector of size 1.9 Gb
# Or on other occasions I get
Error: internal error -3 in R_decompress1
Mark Bravington
CSIRO CMIS
Hobart
Australia
--please do not edit the information below--
Version:
platform = i386-pc-mingw32
arch = i386
os = mingw32
system = i386, mingw32
status =
major = 2
minor = 10.0
year = 2009
month = 10
day = 26
svn rev = 50208
language = R
version.string = R version 2.10.0 (2009-10-26)
Windows XP (build 2600) Service Pack 2
Locale:
LC_COLLATE=English_Australia.1252;LC_CTYPE=English_Australia.1252;LC_MONETARY=English_Australia.1252;LC_NUMERIC=C;LC_TIME=English_Australia.1252
Search Path:
.GlobalEnv, ROOT, package:grDevices, package:ad, package:chstuff,
package:handy2, package:tweedie, package:statmod, package:handy, package:debug,
package:mvbutils, mvb.session.info, package:tools, package:tcltk,
package:stats, package:graphics, package:utils, package:methods, Autoloads,
package:base
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
