Re: [Rd] length(unclass(x)) without unclass(x)?

[Radford Neal]

> Henrik Bengtsson:
>
> I'm looking for a way to get the length of an object 'x' as given by
> base data type without dispatching on class.


The performance improvement you're looking for is implemented in the
latest version of pqR (pqR-2016-10-24, see pqR-project.org), along
with corresponding improvements in several other circumstances where
unclass(x) does not create a copy of x.

Here are some examples (starting with yours), using pqR's Rprofmemt
function to get convenient traces of memory allocations:

  > Rprofmemt(nelem=1000)  # trace allocations of vectors with >= 1000 elements
  > 
  > x <- structure(double(1e6), class = c("foo", "numeric"))
  RPROFMEM: 840 (double 100):"double" "structure" 
  RPROFMEM: 840 (double 100):"structure" 
  > length.foo <- function(x) 1L
  > length(x)
  [1] 1
  > length(unclass(x))
  [1] 100
  > 
  > `+.foo` <- function (e1, e2) (unclass(e1) + unclass(e2)) %% 100
  > z <- x + x
  RPROFMEM: 840 (double 100):"+.foo" 
  > 
  > `<.foo` <- function (e1, e2) any(unclass(e1) x 
  > y <- unclass(x)
  RPROFMEM: 840 (double 100):

There is no large allocation with length(unclass(x)), and only the
obviously necessarily single allocation in +.foo (not two additional
allocations for unclass(e1) and unclass(e1).  For <.foo, there is no
large allocation at all, because not only are allocations avoided for
unclass(e1) and unclass(e2), but 'any' also avoids an allocation for
the result of the comparison.  Unfortunately, assigning unclass(x) to
a variable does result in a copy being made (this might often be
avoided in future).

These performance improvements are implemented using pqR's "variant
result" mechanism, which also allows many other optimizations.  See

https://radfordneal.wordpress.com/2013/06/30/how-pqr-makes-programs-faster-by-not-doing-things/

for some explanation.  There is no particular reason this mechanism
couldn't be incorporated into R Core's implementation of R.

   Radford Neal

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Potential clue for Bug 16975 - lme fixed sigma - inconsistent REML estimation

[Francisco Mauro]

Dear list,

I was trying to create a VarClass for nlme to work with Fay-Herriot
(FH) models. The idea was to create a modification of VarComb that
instead of multiplying the variance functions made their sum (I called
it varSum). After some fails etc... I found that the I was not getting
the expected results because I needed to make sigma fixed. Trying to
find how to make sigma fixed I run into this bug (with uconfirmed
status but listed) report

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16975

The ideas they propose make unnecessary the use of the variance
function I was thinking of, and I left my idea aside for a couple
days. I recently tried the variance function I mentioned before and I
got estimates that were consistent with those provided by the packages
sae and metafor and with the s-plus version of nlme.

The way of fitting a FH model proposed by Maciej in the bug report is
different to the way I fitted the model using the additive variance
function varSum. Both formulations lead to a similar distribution of
the response. However, Maciej's formulation adds the unknown variance
component as a random effect and the formulation with the additive
varClass treat this variance as a variance of an error component.

Results using varSum are the same as those provided by packages sae
and metafor, while the alternative proposed by Maciej does not fit
with those two packages, which is the subject of the bug mentioned
above. Considering this and the fact that the bug is active I thought
that this example (below) could be helpful and provide some clues to
figure out what is the problem with the bug.

I'm not sure about what way would be better to fit models like the FH
model. I find Maciej's solution more flexible for several things than
the route I was taking, but the reported bug made me loose some
confidence. I believe that information about the bug 16975 can be very
interesting. I hope the example below can help or provide some clues.

Thanks

Paco Mauro

# TODO: Add comment
#
# Author: Paco
###
sessionInfo()
#R version 3.3.2 (2016-10-31)
#Platform: x86_64-w64-mingw32/x64 (64-bit)
#Running under: Windows 10 x64 (build 14393)
#
#locale:
#  [1] LC_COLLATE=English_United States.1252
#[2] LC_CTYPE=English_United States.1252
#[3] LC_MONETARY=English_United States.1252
#[4] LC_NUMERIC=C
#[5] LC_TIME=English_United States.1252
#
#attached base packages:
#  [1] stats graphics  grDevices utils datasets  methods   base
#
#other attached packages:
#  [1] sae_1.1  MASS_7.3-45  nlme_3.1-131 rj_2.0.5-2
#
#loaded via a namespace (and not attached):
#  [1] tools_3.3.2 grid_3.3.2  lattice_0.20-34

#Package: nlme
#Version: 3.1-131
library(nlme)
library(sae)
* varSum, a modification of varComb to make the combination
additive instead of multiplicative
varSum <-
  ## constructor for the varSum class
  function(...)
{
 val <- list(...)
 if (!all(unlist(lapply(val, inherits, "varFunc" {
  stop("all arguments to 'varSum' must be of class \"varFunc\".")
 }
 if (is.null(names(val))) {
  names(val) <- LETTERS[seq_along(val)]
 }
 class(val) <- c("varSum","varComb", "varFunc")
 val
}
varWeights.varSum <-
  function(object)
{
 apply(as.data.frame(lapply(object, varWeights)), 1, function(x){

1/sqrt(sum((1/x)^2))

   })
}
Initialize.varSum <-
  function(object, data, ...)
{
 val <- lapply(object, Initialize, data)
 attr(val, "plen") <- unlist(lapply(val, function(el) length(coef(el
 class(val) <- c("varSum","varComb", "varFunc")
 val
}
logLik.varSum <-
  function(object, ...)
{
 lls <- lapply(object, logLik)

 lls2 <- apply(as.data.frame(lapply(object, varWeights)), 1, function(x){

1/sqrt(sum((1/x)^2))

   })

 val <- sum(log(lls2))
 attr(val, "df") <- sum(unlist(lapply(lls, attr, "df")))
 class(val) <- "logLik"
 val
}

* The methods from here to the example are just copies of the
varComb methods with different names
coef.varSum <-
  function(object, unconstrained = TRUE, allCoef = FALSE, ...)
{
 unlist(lapply(object, coef, unconstrained, allCoef))
}

"coef<-.varSum" <-
  function(object, ..., value)
{
 plen <- attr(object, "plen")
 if ((len <- sum(plen)) > 0) {  # varying parameters
  if (length(value) != len) {
   stop("cannot change parameter length of initialized \"varSum\" object")
  }
  start <- 0
  for (i in seq_along(object)) {
   if (plen[i] > 0) {
coef(object[[i]]) <- value[start + (1:plen[i])]
start <- start + plen[i]
   }
  }
 }
 object
}
formula.varSum <-
  function(x, ...) lapply(x, formula)
needUpdate.varSum <-
  function(object) any(unlist(lapply(object, needUpdate)))

print.varSum <-
  function(x, ...)
{
 cat("Sum of:\n")
 lapply(x, print)
 invisible(x)
}

print.summary.varSum <-
  function(x, ...)
{
 cat(attr(x, "structName"),"\n")
 lapply(x, print, FALSE)
 invisible(x)
}

summary.varSum <-
  function(object, structName = "Sum of variance functions:", ...)
{
 object[] <- 

Re: [Rd] can we override "if" in R?

[Gabriel Becker]

Mark,

I do understand this desire, though notably, if selecting a single element
from one of these custom vectors gave you a normal value, if would be safe,
right? e.g. if the [ method, etc checked if the index length is one, and
returned the non-compressed value if it was, as an off-the-cuff possibility.

Also, I do feel compelled to say that I'm working with R-core (Luke Tierney
primarily) on low-level support for this kind of data-storage abstraction
that would let you create custom vector/matrix storage mechanisms that
still behave themselves as atomic vectors. See
https://www.r-project.org/dsc/2016/slides/customvectors.html for my initial
pitch and https://svn.r-project.org/R/branches/ALTREP/
 for the work-in-progress
branch. We hope to merge that in after the release of 3.4, but there's no
firm date as some work on our part and discussion amongst r-core remains
before it can happen.

So in the future, I hope you'll be able to do this in ways that R sees as
actual atomic vectors. They do have to be define in C-code currently,
though.

~G

On Sun, Mar 5, 2017 at 5:09 PM,  wrote:

> I can't comment for Da, but one example where the ability to make 'if'
> generic would have been desirable:
>
> A couple of years ago I wrote S3 classes and methods for 1-byte integers
> and logicals stored as raw vectors, in order to handle massive amounts of
> genetic data (by the standards of the day). Everything worked pretty
> nicely, ie I could "methodize" just about everything I needed--- except
> if-statements, which would fail to respect eg my definitions of NA. [ The
> precise details elude me, but if() was untrustworthy. ]  To use 'if()',  I
> had to remember to "typecast", which was prone to "user error".
>
> Whether this kind of thing is worth the "risk", is another matter.
>
> cheers
> Mark
>
> Mark Bravington
> CSIRO Marine Lab
> Hobart
> Australia
>
> 
> From: R-devel [r-devel-boun...@r-project.org] on behalf of Gabriel Becker
> [gmbec...@ucdavis.edu]
> Sent: 06 March 2017 11:43
> To: Da Zheng
> Cc: r-devel@r-project.org
> Subject: Re: [Rd] can we override "if" in R?
>
> Da,
>
> I've been following this thread and I'm still confused as to exactly what
> you want/why you want it.
>
> I'm probably just missing some context here, but, If() doesn't operate on
> matrices, generally. Can you give an example of the type of code you want
> to have continue to run that requires if operation *directly* on one of
> your matrix objects, as opposed, say, to a value pulled out from it, or the
> dot-product of two vectors in your system, both of which would be values
> (scalars) not matrices.
>
> Now ifelse(), is of course, a different beast altogether, and would need to
> be overloaded within your system, I imagine.
>
> Best,
> ~G
>
> On Sun, Mar 5, 2017 at 12:52 PM, Da Zheng  wrote:
>
> > On Sun, Mar 5, 2017 at 2:50 PM, Michael Lawrence
> >  wrote:
> > >
> > >
> > > On Sat, Mar 4, 2017 at 12:36 PM, Da Zheng 
> wrote:
> > >>
> > >> In my case, I create a new type of matrices and override matrix
> > >> operations in R for these matrices.
> > >> My goal is to make the system as transparent as possible, which means
> > >> my system should execute the existing R code without modification.
> > >> The problem is that when data is in my own vectors or matrices, "if"
> > >> or "while" can't access their values unless we explicitly convert them
> > >> into R objects. But this means users need to modify the existing code.
> > >> So I hope I can override "if", "while", etc to access data in my own
> > >> vectors and matrices directly.
> > >> Does this sound reasonable?
> > >>
> > >
> > > Would you really need the alternate representation for scalar logicals?
> > >
> > > I can see a case in the deferred evaluation context, although it would
> be
> > > problematic wrt side effects unless the deferral is complete.
> > This is exactly why I want to use my own matrix objects and redefine
> > "if" for the matrices. In my framework, all matrices are read-only, so
> > there isn't side effect.
> >
> > Best,
> > Da
> > >
> > >
> > >
> > >>
> > >> Best,
> > >> Da
> > >>
> > >> On Sat, Mar 4, 2017 at 3:22 PM, Michael Lawrence
> > >>  wrote:
> > >> > I'm curious as to precisely why someone would want to do this.
> > >> >
> > >> > On Sat, Mar 4, 2017 at 11:49 AM, Da Zheng 
> > wrote:
> > >> >>
> > >> >> I'm just curious. Why making "if" generic is even more dangerous?
> > >> >>
> > >> >> Best,
> > >> >> Da
> > >> >>
> > >> >> On Sat, Mar 4, 2017 at 1:22 PM, Gábor Csárdi <
> csardi.ga...@gmail.com
> > >
> > >> >> wrote:
> > >> >> > `!` is a generic, `if` is not. You can define an `if` that is
> > >> >> > generic,
> > >> >> > but this might be even more dangerous
> > >> >> >
> > >> >> > ❯ `if` <- 

Re: [Rd] Seeking advice regarding compilation of large libraries using RTools (Windows)

[Richard Beare]

Yep - simpleITK is available at github.com/SimpleITK/SimpleITK. There's
also github.com/SimpleITK/SimpleITKRInstaller - a devtools based installer
for mac and linux.

CMake has a range of build environments. I experimented with MSYS2 and
mingw makefiles, but had trouble with incompatibilities in the path
required by CMake and those options - from memory the sh in RTools/bin
caused problems. Although it sounds like you are saying it is necessary to
install the MSYS2 system as well.

The unix makefile option for CMake appears to work well until the linking
stage. Ninja has problems at a similar stage.

I'll steer clear of dll's, as you suggest. I'm checking those links for
compiler/links flags to see if we're missing anything

Thanks



On Mon, Mar 6, 2017 at 9:51 PM, Jeroen Ooms  wrote:

> On Mon, Mar 6, 2017 at 6:33 AM, Richard Beare 
> wrote:
> > I am working on the SimpleITK package for R. This is an enormous package
> > that is largely automatically generated via a set of swig/json/lua magic,
> > and is working well under linux and osx.
>
> Is it available somewhere so we can try it?
>
>
> > However we're having a lot of trouble with the Windows side. In fact, we
> are struggling to get the base libraries to build using the RTools 3.4
> toolchain, even before the worrying about the R-specific parts.
>
> What build environment do you use? The version of gcc with Rtools
> should be ok, but the Rtools build utilities in the "bin" folder (in
> particular 'make') are old and a frequent source of problems. However
> for building external libs you can use other tools, for example those
> from msys2. Just make sure you use gcc/g++ from Rtools.
>
>
> > The current issue is very long time (possibly infinite) linking of dlls,
> or
> > test executables. I've tried using a FAT32 file system for the build, as
> > suggested by some old bug reports, but still have the issue.
>
> On Windows you can avoid the run-time dll mess by building static libs
> of external libraries. See rwinlib for examples:
> https://github.com/rwinlib
>
>
> > Any suggestions on where to turn next? Are cross compilers the next step?
>
> Try building with msys2, but make sure to use gcc/g++ from Rtools by
> setting the `CC` and `CXX` variables in the configure script. Cross
> compiling will make things even more complicated because binaries
> might not be compatible if your cross compiler has a different version
> of gcc or has been configured for another exception model (seh/drawf).
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Please add me to bugzilla

[Martin Maechler]

> Bradley Broom 
> on Mon, 6 Mar 2017 06:55:35 -0600 writes:

> Apologies, I thought I was following exactly that sentence
> and trying to make a minimal post that would waste as
> little developer bandwidth as possible given the lack of a
> better system.

I understand.   My apologies now, as I was mistrusting, clearly
wrongly in this case.

> Anyway, I have been using R for like forever (20 years).

> In my current project, I have run into problems with stack
> overflows in R's dendrogram code when trying to use either
> str() or as.hclust() on very deep dendrograms.

I understand.  Indeed, bug PR#16424 
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16424
encountered the same problem in other dendrogram functions and
solved it by re-programming the relevant parts non-recursively,
too.

   [.]  

> What should happen: Function completes without a stack
> overflow.

> 2nd bug: hh <- as.hclust(de)

> What happens: Error: C stack usage 7971248 is too close to the limit

> What should happen: Function completes without a stack
> overflow.

> A knowledgeable user might be able to increase R's limits
> to avoid these errors on this particular dendrogram, but
> a) my users aren't that knowledgeable about R and this is
> expected to be a common problem, and b) there will be
> bigger dendrograms (up to at least 25000 leaves).

Agreed.  The current help pages warns about the problem and
gives advice (related to increasing the stack), but what you propose
is better, i.e., re-implementing relevant parts non-recursively.

> Please see attached patch for non-recursive
> implementations.

Very well done, thank you a lot!
[and I will add you to bugzilla .. so you can use it for the
 next bug .. ;-)]

Best,
Martin

> Regards, Bradley



> On Mon, Mar 6, 2017 at 3:50 AM, Martin Maechler
>  wrote:

>> > Bradley Broom  > on Sun, 5
>> Mar 2017 16:03:30 -0600 writes:
>> 
>> > Please add me to R bugzilla.  Thanks, Bradley
>> 
>> Well, I will not do it just like that (mean "after such a
>> minimal message").
>> 
>> I don't see any evidence as to your credentials,
>> knowledge of R, etc, as part of this request.  We are all
>> professionals, devoting part of our (work and free) time
>> to the R project (rather than employees of the company
>> you paid to serve you ...)
>> 
>> It may be that you have read
>> https://www.r-project.org/bugs.html
>> 
>> Notably this part
>> 
--> NOTE: due to abuse by spammers, since 2016-07-09 only
--> users who have
>> previously submitted bugs can submit new ones on R’s
>> Bugzilla. We’re working on a better system… In the mean
>> time, post (e-mail) to R-devel or ask an R Core member to
>> add you manually to R’s Bugzilla members.
>> 
>> The last sentence was *meant* to say you should post
>> (possibly parts, ideally a minimal reproducible example
>> of) your bug report to R-devel so others could comment on
>> it, agree or disagree with your assessment etc, __or__
>> ask an R-core member to add you to bugzilla (if you
>> really read the other parts of the 'R bugs' web page
>> above).
>> 
>> Posting to all 1000 R-devel readers with no content about
>> what you consider a bug is a waste of bandwidth for at
>> least 99% of these readers.
>> 
>> [Yes, I'm also using their time ... in the hope to
>> *improve* the quality of future such postings].
>> 
>> Martin Maechler ETH Zurich
>> 
> x external: dendro-non-recursive.patch text/x-patch, u
> [Click mouse-2 to display text]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Please add me to bugzilla

[Bradley Broom]

Apologies, I thought I was following exactly that sentence and trying to
make a minimal post that would waste as little developer bandwidth as
possible given the lack of a better system.

Anyway, I have been using R for like forever (20 years).

In my current project, I have run into problems with stack overflows in R's
dendrogram code when trying to use either str() or as.hclust() on very deep
dendrograms.

To duplicate, use this function from tests/reg-tests-1c.R in the R source
code:
mkDend <- function(n, lab, method = "complete",
   ## gives *ties* often:
   rGen = function(n) 1+round(16*abs(rnorm(n {
stopifnot(is.numeric(n), length(n) == 1, n >= 1, is.character(lab))
a <- matrix(rGen(n*n), n, n)
colnames(a) <- rownames(a) <- paste0(lab, 1:n)
.HC. <<- hclust(as.dist(a + t(a)), method=method)
as.dendrogram(.HC.)
}

Get a nasty dendrogram:

de <- mkDend(2000, 'x', 'single')

1st bug:
sink('somefile.txt'); str(de); sink();

What happens:
Error in getOption("OutDec") : node stack overflow

Also, the last call to sink() isn't executed because of the error, so
you'll need to call sink() after the error to clear the diversion.

What should happen:
Function completes without a stack overflow.

2nd bug:
hh <- as.hclust(de)

What happens:
Error: C stack usage  7971248 is too close to the limit

What should happen:
Function completes without a stack overflow.

A knowledgeable user might be able to increase R's limits to avoid these
errors on this particular dendrogram, but a) my users aren't that
knowledgeable about R and this is expected to be a common problem, and b)
there will be bigger dendrograms (up to at least 25000 leaves).

Please see attached patch for non-recursive implementations.

Regards,
Bradley



On Mon, Mar 6, 2017 at 3:50 AM, Martin Maechler 
wrote:

> > Bradley Broom 
> > on Sun, 5 Mar 2017 16:03:30 -0600 writes:
>
> > Please add me to R bugzilla.  Thanks, Bradley
>
> Well, I will not do it just like that (mean "after such a
> minimal message").
>
> I don't see any evidence as to your credentials, knowledge of R,
> etc, as part of this request.  We are all professionals,
> devoting part of our (work and free) time to the R project
> (rather than employees of the company you paid to serve you ...)
>
> It may be that you have read   https://www.r-project.org/bugs.html
>
> Notably this part
>
> --> NOTE: due to abuse by spammers, since 2016-07-09 only users who have
> previously submitted bugs can submit new ones on R’s Bugzilla. We’re
> working on a better system… In the mean time, post (e-mail) to R-devel or
> ask an R Core member to add you manually to R’s Bugzilla members.
>
> The last sentence was *meant* to say you should post (possibly
> parts, ideally a minimal reproducible example of) your bug
> report to R-devel so others could comment on it, agree or
> disagree with your assessment etc,
> __or__ ask an R-core member to add you to bugzilla (if you really read the
> other parts of the 'R bugs' web page above).
>
> Posting to all 1000 R-devel readers with no content about what
> you consider a bug  is a waste of bandwidth for at least 99% of
> these readers.
>
> [Yes, I'm also using their time ... in the hope to *improve* the
>  quality of future such postings].
>
> Martin Maechler
> ETH Zurich
>
Index: src/library/stats/R/dendrogram.R
===
--- src/library/stats/R/dendrogram.R	(revision 72314)
+++ src/library/stats/R/dendrogram.R	(working copy)
@@ -81,60 +81,130 @@
 structure(z[[as.character(k)]], class = "dendrogram")
 }
 
+# Count the number of leaves in a dendrogram.
+nleaves <- function (node) {
+if (is.leaf(node)) { return (1L) }
+todo <- NULL # Non-leaf nodes to traverse after this one.
+count <- 0L
+repeat {
+# For each child: count iff a leaf, add to todo list otherwise.
+	while (length(node)) {
+	child <- node[[1L]];
+node <- node[-1L];
+	if (is.leaf(child)) {
+count <- count + 1L
+} else {
+		todo <- list(node=child, todo=todo)
+	}
+	}
+# Advance to next node, terminating when no nodes left to count.
+	if (is.null(todo)) {
+break
+	} else {
+	node <- todo$node
+	todo <- todo$todo
+}
+}
+return (count)
+}
+
 ## Reversing the above (as much as possible)
 ## is only possible for dendrograms with *binary* splits
 as.hclust.dendrogram <- function(x, ...)
 {
-stopifnot(is.list(x), length(x) == 2)
-n <- length(ord <- as.integer(unlist(x)))
+stopifnot(is.list(x), length(x) == 2L)
+n <- nleaves(x)
+stopifnot(n == attr(x, "members"))
+
+# Ord and labels for each leaf node (in preorder).
+ord <- integer(n)
+labsu <- character(n)
+
+# Height and (parent,index) for each internal node (in preorder).
+n.h <- n - 1L
+height <- numeric(n.h)
+myIdx 

Re: [Rd] Control statements with condition with greater than one should give error (not just warning) [PATCH]

[Martin Maechler]

> Michael Lawrence 
> on Sat, 4 Mar 2017 12:20:45 -0800 writes:

> Is there really a need for these complications? Packages
> emitting this warning are broken by definition and should be fixed. 

I agree and probably Henrik, too.

(Others may disagree to some extent .. and find it convenient
 that R does translate 'if(x)'  to  'if(x[1])'  for them albeit
 with a warning .. )

> Perhaps we could "flip the switch" in a test
> environment and see how much havoc is wreaked and whether
> authors are sufficiently responsive?

> Michael

As we have > 10'000 packages on CRAN alonce,  and people have
started (mis)using suppressWarnings(.) in many places,  there
may be considerably more packages affected than we optimistically assume...

As R core member who would  "flip the switch"  I'd typically then
have to be the one sending an e-mail to all package maintainers
affected and in this case I'm very reluctant to volunteer
for that and so, I'd prefer the environment variable where R
core and others can decide how to use it .. for a while .. until
the flip is switched for all.

or have I overlooked an issue?

Martin

> On Sat, Mar 4, 2017 at 12:04 PM, Martin Maechler
> > wrote:

>> > Henrik Bengtsson  >
>> on Fri, 3 Mar 2017 10:10:53 -0800 writes:
>> 
>> > On Fri, Mar 3, 2017 at 9:55 AM, Hadley Wickham >
>>  wrote: >>> But, how you propose a
>> warning-to-error transition >>> should be made without
>> wreaking havoc?  Just flip the >>> switch in R-devel and
>> see CRAN and Bioconductor packages >>> break overnight?
>> Particularly Bioconductor devel might >>> become
>> non-functional (since at times it requires >>> R-devel).
>> For my own code / packages, I would be able >>> to handle
>> such a change, but I'm completely out of >>> control if
>> one of the package I'm depending on does not >>> provide
>> a quick fix (with the only option to remove >>> package
>> tests for those dependencies).
>> >>
>> >> Generally, a package can not be on CRAN if it has any
>> >> warnings, so I don't think this change would have any
>> >> impact on CRAN packages.  Isn't this also true for >>
>> bioconductor?
>> 
>> > Having a tests/warn.R file with:
>> 
>> > warning("boom")
>> 
>> > passes through R CMD check --as-cran unnoticed.
>> 
>> Yes, indeed.. you are right Henrik that many/most R
>> warning()s would not produce R CMD check 'WARNING's ..
>> 
>> I think Hadley and I fell into the same mental pit of
>> concluding that such warning()s from
>> if() ...  would not currently happen
>> in CRAN / Bioc packages and hence turning them to errors
>> would not have a direct effect.
>> 
>> With your 2nd e-mail of saying that you'd propose such an
>> option only for a few releases of R you've indeed
>> clarified your intent to me.  OTOH, I would prefer using
>> an environment variable (as you've proposed as an
>> alternative) which is turned "active" at the beginning
>> only manually or for the "CRAN incoming" checks of the
>> CRAN team (and bioconductor submission checks?)  and
>> later for '--as-cran' etc until it eventually becomes the
>> unconditional behavior of R (and the env.variable is no
>> longer used).
>> 
>> Martin
>> 
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 

>   [[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [R-pkg-devel] best way the handle errors conditionally

[Duncan Murdoch]

On 06/03/2017 5:17 AM, Thierry Onkelinx wrote:

Dear all,

I'd like your advice on handling errors conditionally within a function.
Imagine a function that does several database operations. The user has the
option to run them inside a transaction. So the code has somewhere near the
top:

if (transaction) {
DBI::dbBegin(conn)
}

At the end of the function there is a command which commits the transaction.

if (transaction) {
DBI::dbCommit(conn)
}

If something goes wrong, one reverses the database operations by issuing
the DBI::dbRollback(conn) command. The first option for issuing that is to
use tryCatch(). The drawback is that the set of command can be a few
hundred lines, which harms readability when wrapping them into a tryCatch().


You could put the long set of commands into their own function, perhaps 
locally defined within the main function.  For example,


longset <- function() { ... }
if (transaction) {
DBI::dbBegin(conn)
}
tryCatch(
  longset()
  error = function(e){
 if (transaction) {
   dbRollback(conn)
 }
 stop(e)
   }
)

Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [Rd] Seeking advice regarding compilation of large libraries using RTools (Windows)

[Jeroen Ooms]

On Mon, Mar 6, 2017 at 6:33 AM, Richard Beare  wrote:
> I am working on the SimpleITK package for R. This is an enormous package
> that is largely automatically generated via a set of swig/json/lua magic,
> and is working well under linux and osx.

Is it available somewhere so we can try it?


> However we're having a lot of trouble with the Windows side. In fact, we are 
> struggling to get the base libraries to build using the RTools 3.4 toolchain, 
> even before the worrying about the R-specific parts.

What build environment do you use? The version of gcc with Rtools
should be ok, but the Rtools build utilities in the "bin" folder (in
particular 'make') are old and a frequent source of problems. However
for building external libs you can use other tools, for example those
from msys2. Just make sure you use gcc/g++ from Rtools.


> The current issue is very long time (possibly infinite) linking of dlls, or
> test executables. I've tried using a FAT32 file system for the build, as
> suggested by some old bug reports, but still have the issue.

On Windows you can avoid the run-time dll mess by building static libs
of external libraries. See rwinlib for examples:
https://github.com/rwinlib


> Any suggestions on where to turn next? Are cross compilers the next step?

Try building with msys2, but make sure to use gcc/g++ from Rtools by
setting the `CC` and `CXX` variables in the configure script. Cross
compiling will make things even more complicated because binaries
might not be compatible if your cross compiler has a different version
of gcc or has been configured for another exception model (seh/drawf).

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Please add me to bugzilla

[Martin Maechler]

> Bradley Broom 
> on Sun, 5 Mar 2017 16:03:30 -0600 writes:

> Please add me to R bugzilla.  Thanks, Bradley

Well, I will not do it just like that (mean "after such a
minimal message").

I don't see any evidence as to your credentials, knowledge of R,
etc, as part of this request.  We are all professionals,
devoting part of our (work and free) time to the R project
(rather than employees of the company you paid to serve you ...)

It may be that you have read   https://www.r-project.org/bugs.html

Notably this part

--> NOTE: due to abuse by spammers, since 2016-07-09 only users who have 
previously submitted bugs can submit new ones on R’s Bugzilla. We’re working on 
a better system… In the mean time, post (e-mail) to R-devel or ask an R Core 
member to add you manually to R’s Bugzilla members.

The last sentence was *meant* to say you should post (possibly
parts, ideally a minimal reproducible example of) your bug
report to R-devel so others could comment on it, agree or
disagree with your assessment etc,
__or__ ask an R-core member to add you to bugzilla (if you really read the
other parts of the 'R bugs' web page above).

Posting to all 1000 R-devel readers with no content about what
you consider a bug  is a waste of bandwidth for at least 99% of
these readers.

[Yes, I'm also using their time ... in the hope to *improve* the
 quality of future such postings].

Martin Maechler
ETH Zurich

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Seeking advice regarding compilation of large libraries using RTools (Windows)

[Richard Beare]

Hello,

I am working on the SimpleITK package for R. This is an enormous package
that is largely automatically generated via a set of swig/json/lua magic,
and is working well under linux and osx. However we're having a lot of
trouble with the Windows side. In fact, we are struggling to get the base
libraries to build using the RTools 3.4 toolchain, even before the worrying
about the R-specific parts.

I'm hoping someone has observed and solved this problem.

The current issue is very long time (possibly infinite) linking of dlls, or
test executables. I've tried using a FAT32 file system for the build, as
suggested by some old bug reports, but still have the issue.

The build system is CMake-based, and we have the same issues with either
unix makefiles or ninja. The current build host is windows 2012 server, but
we've had the same problems on windows 7 and 10.

We're using the -Wa -mbig-obj flags.

One of the problem commands looks like this:

cd D:/B/SimpleITK-build/Code/Explicit/src && C:/Rtools/mingw_64/bin/g++.exe
-Wno-long-long -Wno-unused-local-typedefs -Wno-strict-overflow -Wextra
-Wformat=2 -Wno-format-nonliteral -Wunused -Wpointer-arith -Winvalid-pch
-Wcast-align -Wdisabled-optimization -Woverloaded-virtual -Wshadow
-Wwrite-strings -Wstrict-null-sentinel -Wno-invalid-offsetof  -Wa,-mbig-obj
-Wa,-mbig-obj  -mthreads   -mthreads -O3 -DNDEBUG   -mthreads   -mthreads
-shared -o ../../../bin/libSimpleITKExplicit-1.0.dll
-Wl,--out-implib,../../../lib/libSimpleITKExplicit-1.0.dll.a
-Wl,--major-image-version,1,--minor-image-version,0 -Wl,--whole-archive
CMakeFiles/SimpleITKExplicit.dir/objects.a -Wl,--no-whole-archive
@CMakeFiles/SimpleITKExplicit.dir/linklibs.rsp


As a guide, the final shared library is 259M on linux. The windows build
doesn't appear to be memory limited - the ld process is using about 1.5G
and consuming 1 core, but doesn't complete given an entire day.

Any suggestions on where to turn next? Are cross compilers the next step?

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel