Re: [R-pkg-devel] Examples taking too long depend on object that takes a while to generate
> John Harrold
> on Thu, 15 Sep 2022 05:11:15 -0700 writes:
> Not to be pedantic but it's not a dataset per-se. It's an R object that
the
> examples need.
Yes.
(and see below)
> On Thu, Sep 15, 2022 at 2:49 AM Duncan Murdoch
> wrote:
>> On 15/09/2022 5:29 a.m., Martin Maechler wrote:
>> >> Duncan Murdoch
>> >> on Thu, 15 Sep 2022 04:42:04 -0400 writes:
>> >
>> > > On 15/09/2022 3:45 a.m., Martin Maechler wrote:
>> > >>> Duncan Murdoch
>> > >>> on Wed, 14 Sep 2022 13:02:28 -0400 writes:
>> > >>
>> > >> > On 14/09/2022 12:43 p.m., Ivan Krylov wrote:
>> > >> >> On Wed, 14 Sep 2022 12:31:49 -0400
>> > >> >> Duncan Murdoch wrote:
>> > >> >>
>> > >> >>> It's also possible to put .R files in the data directory,
and they
>> > >> >>> are executed to create the data object. I think that
happens at the
>> > >> >>> time when you call data() rather than at install time, so
it might
>> > >> >>> not be helpful.
>> > >> >>
>> > >> >> Some time ago I was hoping to compress a package of mine by
generating a
>> > >> >> dataset during a data() call instead loading it from an
.rda file, but
>> > >> >> it turned out that the .R file is executed during R CMD
build:
>> > >> >>
>>
https://github.com/r-devel/r-svn/blob/03df313ad37456c6a62158328d4e373408ce4d59/src/library/tools/R/build.R#L794
>> > >>
>> > >> > Thanks for that info. That's not good for John, because the
>> > >> > architecture isn't known at build time.
>> > >>
>> > >> > Duncan Murdoch
>> > >>
>> > >> Sorry to muddy the water, but what *is* "build time"?
>> > >> There's the big difference between building
>> > >> 1) a Source tarballand
>> > >> 2) a MacOS or Windows binary package
>> > >>
>> > >> Unfortunately, the two situations are very different notably in
>> > >> this case, where '(2)' is really much closer to the
>> > >> "install time" you mention.
>> > >>
>> >
>> > > I meant building the tarball, and assumed that was what Ivan
was talking
>> > > about as well.
>> >
>> > > Duncan Murdoch
>> >
>> > Ok, thank you, for the clarification.
>> >
>> > Note that `R CMD build --help` mentions (among more)
>> >
>> >--resave-data=re-save data files as compactly as possible:
>> > "no", "best", "gzip" (default)
>> >--no-resave-data same as --resave-data=no
>> >
>> > so when building the package,
>> > Ivan should get what he wanted with
>> >
>> > R CMD build --no-resave-data
>> >
>> > no ?
>>
>> It's actually John Harrold who has the problem: a dataset that he wants
>> to use in examples that takes a long time to build, causing his examples
>> to exceed the CRAN 5 second limit.
>>
>> So what I was suggesting is that he should arrange for it to be created
>> before running the example; the problem is that the dataset depends on
>> the architecture of the machine that's running the example. To follow
>> my suggestion he would need to have the dataset created when the package
>> was installed (or the binary was built).
and now John can use the data() "trick" Ivan recommended *if*
John builds his source tar ball using
R CMD build --no-resave-data
*and* puts the code that creates the R object into a /data/myRobj.R
file
*and* adds a
data(myRobj)
into his .onLoad() hook.
This would *not* create the object at package installation time
but each time the package is loaded .. which is at least only
once for all the examples in R CMD check
OTOH, I do agree with you Duncan, that in this case your
suggestion seems preferable and we add the R code which creates
the object somewhere "nakedly" in /R/zzz.R {zzz: to be alphabetically
last}
and hence the object would live (hidden) in his_pkg namespace.
He'd ideally also provide an *exported* function, say
myData <- function(name = "") get(name,
asNamespace(""))
and then would use myData() {or myData(".") } in his
package examples.
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Examples taking too long depend on object that takes a while to generate
Not to be pedantic but it's not a dataset per-se. It's an R object that the examples need. On Thu, Sep 15, 2022 at 2:49 AM Duncan Murdoch wrote: > On 15/09/2022 5:29 a.m., Martin Maechler wrote: > >> Duncan Murdoch > >> on Thu, 15 Sep 2022 04:42:04 -0400 writes: > > > > > On 15/09/2022 3:45 a.m., Martin Maechler wrote: > > >>> Duncan Murdoch > > >>> on Wed, 14 Sep 2022 13:02:28 -0400 writes: > > >> > > >> > On 14/09/2022 12:43 p.m., Ivan Krylov wrote: > > >> >> On Wed, 14 Sep 2022 12:31:49 -0400 > > >> >> Duncan Murdoch wrote: > > >> >> > > >> >>> It's also possible to put .R files in the data directory, > and they > > >> >>> are executed to create the data object. I think that > happens at the > > >> >>> time when you call data() rather than at install time, so it > might > > >> >>> not be helpful. > > >> >> > > >> >> Some time ago I was hoping to compress a package of mine by > generating a > > >> >> dataset during a data() call instead loading it from an .rda > file, but > > >> >> it turned out that the .R file is executed during R CMD build: > > >> >> > https://github.com/r-devel/r-svn/blob/03df313ad37456c6a62158328d4e373408ce4d59/src/library/tools/R/build.R#L794 > > >> > > >> > Thanks for that info. That's not good for John, because the > > >> > architecture isn't known at build time. > > >> > > >> > Duncan Murdoch > > >> > > >> Sorry to muddy the water, but what *is* "build time"? > > >> There's the big difference between building > > >> 1) a Source tarballand > > >> 2) a MacOS or Windows binary package > > >> > > >> Unfortunately, the two situations are very different notably in > > >> this case, where '(2)' is really much closer to the > > >> "install time" you mention. > > >> > > > > > I meant building the tarball, and assumed that was what Ivan was > talking > > > about as well. > > > > > Duncan Murdoch > > > > Ok, thank you, for the clarification. > > > > Note that `R CMD build --help` mentions (among more) > > > >--resave-data=re-save data files as compactly as possible: > > "no", "best", "gzip" (default) > >--no-resave-data same as --resave-data=no > > > > so when building the package, > > Ivan should get what he wanted with > > > > R CMD build --no-resave-data > > > > no ? > > It's actually John Harrold who has the problem: a dataset that he wants > to use in examples that takes a long time to build, causing his examples > to exceed the CRAN 5 second limit. > > So what I was suggesting is that he should arrange for it to be created > before running the example; the problem is that the dataset depends on > the architecture of the machine that's running the example. To follow > my suggestion he would need to have the dataset created when the package > was installed (or the binary was built). > > Duncan Murdoch > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > -- John :wq [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Examples taking too long depend on object that takes a while to generate
On Wed, 14 Sep 2022 12:31:49 -0400 Duncan Murdoch wrote: > It's also possible to put .R files in the data directory, and they > are executed to create the data object. I think that happens at the > time when you call data() rather than at install time, so it might > not be helpful. Some time ago I was hoping to compress a package of mine by generating a dataset during a data() call instead loading it from an .rda file, but it turned out that the .R file is executed during R CMD build: https://github.com/r-devel/r-svn/blob/03df313ad37456c6a62158328d4e373408ce4d59/src/library/tools/R/build.R#L794 -- Best regards, Ivan __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Examples taking too long depend on object that takes a while to generate
Hello Greg, I'm already using a reduced dataset. I can reduce it a little bit further but it doesn't really change things much. John On Tue, Sep 13, 2022 at 8:35 PM Greg Hunt wrote: > John, > Can you cut the test data size back? Do you have some processing costs > that are non-linear with data size? 27 seconds of user CPU is an awful lot > of processing these days. > > Greg > > On Wed, 14 Sept 2022 at 13:21, John Harrold > wrote: > >> Hello, >> >> I'm working on submitting a new package. I've created examples for each of >> the functions. Most of these functions depend on an R object that uses >> architecture-specific compiled code. To create that object can take >> between >> 10-20 seconds. The function that creates it will cache it in the >> tempdir(). >> So the first example that uses that object will take more than the 5 >> seconds allowed by CRAN. If I wrap that example in donttest, then the next >> function that uses it will exceed the 5 second limit, and on. I submitted >> the package with all of these examples wrapped in donttest, but that got >> me >> dinged :). So I'm trying to come up with a solution and I'd appreciate any >> help here. >> >> Is there some way to build an object before tests are run? >> >> Thank you, >> John >> >> If it would help this is an example where I'm testing it in win-builder: >> >> https://win-builder.r-project.org/6641irOr4mI6/ >> >> [[alternative HTML version deleted]] >> >> __ >> R-package-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-package-devel >> > -- John :wq [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Examples taking too long depend on object that takes a while to generate
Hello, I'm working on submitting a new package. I've created examples for each of the functions. Most of these functions depend on an R object that uses architecture-specific compiled code. To create that object can take between 10-20 seconds. The function that creates it will cache it in the tempdir(). So the first example that uses that object will take more than the 5 seconds allowed by CRAN. If I wrap that example in donttest, then the next function that uses it will exceed the 5 second limit, and on. I submitted the package with all of these examples wrapped in donttest, but that got me dinged :). So I'm trying to come up with a solution and I'd appreciate any help here. Is there some way to build an object before tests are run? Thank you, John If it would help this is an example where I'm testing it in win-builder: https://win-builder.r-project.org/6641irOr4mI6/ [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
