[Rd] S4 methods with default argument of NULL

[Benjamin Tyner]

Greetings,

I noticed that starting with R version 3.5.0, it is now possible to do, 
for example:


   setGeneric(name = "foo",
   def = function(x, y, ...) standardGeneric("foo"),
   valueClass = "fooResult"
   )

   setMethod(f = "foo",
  signature = c("character", "character"),
  definition = function(x, y = NULL, z = NULL, ...) {

  force(y)
  message(sprintf("x is %s", x))

  structure(list(), class = "fooResult")
  })

   foo("a", "b") # this works in R >= 3.5.0, but fails in R < 3.5.0

(Prior to R version 3.5.0, it gives "Error in force(y) : object 'y' not 
found")


I am curious to learn whether the change permitting this was:

   1. an intentional enhancement
   2. unintentional, but a "happy accident" (i.e., a desirable side
   effect of some other enhancement)
   3. unintentional, and perhaps undesirable (i.e., a loosening of
   standards)

For reference, this came up in the debugging of the following issue:

   https://github.com/DyfanJones/noctua/issues/170

Any comments to shed some light would be greatly appreciated.

Regards,

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] failed check in reg-tests-1b.R

[Benjamin Tyner]

Not certain this is actually a bug, so posting here.

I'm on Ubuntu 18.04.4 LTS, building R version 4.0.0. The "configure" and 
"make" steps are successful, but the "make check" step fails when it 
gets to this part of ./tests/reg-tests-1b.R:


   > ## methods() gave two wrong warnings in some cases:
   > op <- options(warn = 2)# no warning, please!
   > m1 <- methods(na.omit) ## should give (no warning):
   > ##
   > setClass("bla")
   > setMethod("na.omit", "bla", function(object, ...) "na.omit()")
   Error: package 'codetools' was installed before R 4.0.0: please 
re-install it

   Execution halted

It appears to be picking up the older version of codetools from $R_LIBS; 
if I unset R_LIBS, then it works just fine.


So I'm wondering, is it a bug, or is the user's own fault for having 
R_LIBS set whilst trying to build R?


Regards,

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[R-pkg-devel] use of getNativeSymbolInfo

[Benjamin Tyner]

Hello,

Using package nlme as an example, where there is a registered routine 
fit_gnls:


> library(nlme)
> getDLLRegisteredRoutines("nlme")$.C$fit_gnls
$name
[1] "fit_gnls"

$address

attr(,"class")
[1] "RegisteredNativeSymbol"

$dll
DLL name: nlme
Filename: /home/btyner/R-77784/lib/R/library/nlme/libs/nlme.so
Dynamic lookup: FALSE

$numParameters
[1] 10

attr(,"class")
[1] "CRoutine" "NativeSymbolInfo"

However when I tried this,

> getNativeSymbolInfo(name = "fit_gnls", PACKAGE = "nlme")

it says:

Error in FUN(X[[i]], ...) : no such symbol fit_gnls in package nlme

so I am wondering what I'm doing wrong...

Regards,
Ben

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [Rd] [External] Re: rpois(9, 1e10)

[Benjamin Tyner]

On 1/20/20 12:33 PM, Martin Maechler wrote:


It's really something that should be discussed (possibly not
here, .. but then I've started it here ...).

The  NEWS  for R 3.0.0 contain (in NEW FEATURES) :

 * Functions rbinom(), rgeom(), rhyper(), rpois(), rnbinom(),
   rsignrank() and rwilcox() now return integer (not double)
   vectors.  This halves the storage requirements for large
   simulations.

and what I've been suggesting is to revert this change
(svn rev r60225-6) which was purposefully and diligently done by
a fellow R core member, so indeed must be debatable.

Martin


For the record, I don't personally objects to the change here (as my 
philosophy tends toward treating most warnings as errors anyway) but for 
the sake of other useRs who may get bitten, perhaps we should be more 
explicit that backwards-compatibility won't be preserved under certain 
use patterns, for example:


   # works (with warning) in R 3.6.2 but fails (with error) in R-devel:
   vapply(list(1e9, 1e10),
   function(lambda) {
  rpois(1L, lambda)
   },
   FUN.VALUE = integer(1L)
   )

   # in R-devel, a little extra work to achieve a warning as before:
   vapply(list(1e9, 1e10),
   function(lambda) {
  tmp <- rpois(1L, lambda)
  if (!is.integer(tmp)) {
 warning("NAs produced")
 tmp <- NA_integer_
  }
  tmp
   },
   FUN.VALUE = integer(1L)
   )

(and yes I realize that rpois() vectorizes on lambda, so vapply is 
re-inventing the wheel in this toy example, but there could be (?) a 
justified use for it in more complicated simulations).


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [External] Re: rpois(9, 1e10)

[Benjamin Tyner]

On 1/20/20 4:26 AM, Martin Maechler wrote:

Coming late here -- after enjoying a proper weekend ;-) --
I have been agreeing (with Spencer, IIUC) on this for a long
time (~ 3 yrs, or more?), namely that I've come to see it as a
"design bug" that  rpois() {and similar} must return return typeof() "integer".

More strongly, I'm actually pretty convinced they should return
(integer-valued) double instead of NA_integer_   and for that
reason should always return double:
Even if we have (hopefully) a native 64bit integer in R,
2^64 is still teeny tiny compared .Machine$double.max

(and then maybe we'd have .Machine$longdouble.max  which would
  be considerably larger than double.max unless on Windows, where
  the wise men at Microsoft decided to keep their workload simple
  by defining "long double := double" - as 'long double'
  unfortunately is not well defined by C standards)

Martin


Martin if you are in favor, then certainly no objection from me! ;-)

So now what about other discrete distributions e.g. could a similar 
enhancement apply here?


> rgeom(10L, 1e-10)
 [1] NA 1503061294 NA NA 1122447583 NA
 [7] NA NA NA NA
Warning message:
In rgeom(10L, 1e-10) : NAs produced

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Benjamin Tyner]

So imagine rpois is changed, such that the storage mode of its return 
value is sometimes integer and sometimes numeric. Then imagine the case 
where lambda is itself a realization of a random variable. Do we really 
want the storage mode to inherit that randomness?



On 1/19/20 10:47 AM, Avraham Adler wrote:

Maybe there should be code for 64 bit R to use long long or the like?

On Sun, Jan 19, 2020 at 10:45 AM Spencer Graves 
mailto:spencer.gra...@prodsyse.com>> wrote:




On 2020-01-19 09:34, Benjamin Tyner wrote:
>>

>> Hello, All:
>>
>>
>>     Consider:
>>
>>
>> Browse[2]> set.seed(1)
>> Browse[2]> rpois(9, 1e10)
>> NAs produced[1] NA NA NA NA NA NA NA NA NA
>>
>>
>>     Should this happen?
>>
>>
>>     I think that for, say, lambda>1e6, rpois should return
rnorm(.,
>> lambda, sqrt(lambda)).
> But need to implement carefully; rpois should always return a
> non-negative integer, whereas rnorm always returns numeric...
>

   Thanks for the reply.


   However, I think it's not acceptable to get an NA from a
number
that cannot be expressed as an integer.  Whenever a randomly
generated
number would exceed .Machine$integer.max, the choice is between
returning NA or a non-integer numeric.  Consider:


 > 2*.Machine$integer.max
[1] 4294967294
 > as.integer(2*.Machine$integer.max)
[1] NA
Warning message:
NAs introduced by coercion to integer range


   I'd rather have the non-integer numeric.


   Spencer

__
R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

--
Sent from Gmail Mobile


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] rpois(9, 1e10)

[Benjamin Tyner]

Hello, All:


    Consider:


Browse[2]> set.seed(1)
Browse[2]> rpois(9, 1e10)
NAs produced[1] NA NA NA NA NA NA NA NA NA


    Should this happen?


    I think that for, say, lambda>1e6, rpois should return rnorm(.,
lambda, sqrt(lambda)).
But need to implement carefully; rpois should always return a 
non-negative integer, whereas rnorm always returns numeric...


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Error in close.connection(p) : ignoring SIGPIPE signal

[Benjamin Tyner]

Andreas,

How right you are! Still, I find it curious that in the context of the 
while(TRUE) loop, I am allowed to do this 653 times, with failure on the 
654th attempt. Perhaps there is something asynchronous going on? If I 
eliminate the looping, it does indeed fail (as expected) on the first 
attempt to close the pipe.


Regards

Ben

On 12/6/19 2:04 AM, Andreas Kersting wrote:

Hi Benjamin,

you cannot pipe to echo, since it does not read from stdin.

echo just echos is first arg, i.e. echo /dev/stdin > /dev/null will echo the string 
"/dev/stdin"to /dev/stdout, which is redirected to /dev/null.

Try

p <- pipe("cat > /dev/null", open = "w")

instead.

Regards,
Andreas

2019-12-06 02:46 GMT+01:00 Benjamin Tyner:

Not sure if this is a bug, so posting here first. If I run:
    cnt <- 0L
    while (TRUE) {
        cnt <- cnt + 1L
        p <- pipe("echo /dev/stdin > /dev/null", open = "w")
        writeLines("foobar", p)
        tryCatch(close(p), error = function(e) { print(cnt); stop(e)})
    }

then once cnt gets to around 650, it fails with:

    [1] 654
    Error in close.connection(p) : ignoring SIGPIPE signal

Should I not be using pipe() in this way? Here is my sessionInfo()

    R version 3.6.0 (2019-04-26)
    Platform: x86_64-pc-linux-gnu (64-bit)
    Running under: Ubuntu 18.04.3 LTS

    Matrix products: default
    BLAS:   /home/btyner/R360/lib64/R/lib/libRblas.so
    LAPACK: /home/btyner/R360/lib64/R/lib/libRlapack.so

    locale:
     [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
     [3] LC_TIME=en_US.UTF-8    LC_COLLATE=en_US.UTF-8
     [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
     [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
     [9] LC_ADDRESS=C   LC_TELEPHONE=C
    [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

    attached base packages:
    [1] stats graphics  grDevices utils datasets  methods base

    loaded via a namespace (and not attached):
    [1] compiler_3.6.0

Regards,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Error in close.connection(p) : ignoring SIGPIPE signal

[Benjamin Tyner]

Not sure if this is a bug, so posting here first. If I run:

   cnt <- 0L
   while (TRUE) {
       cnt <- cnt + 1L
       p <- pipe("echo /dev/stdin > /dev/null", open = "w")
       writeLines("foobar", p)
       tryCatch(close(p), error = function(e) { print(cnt); stop(e)})
   }

then once cnt gets to around 650, it fails with:

   [1] 654
   Error in close.connection(p) : ignoring SIGPIPE signal

Should I not be using pipe() in this way? Here is my sessionInfo()

   R version 3.6.0 (2019-04-26)
   Platform: x86_64-pc-linux-gnu (64-bit)
   Running under: Ubuntu 18.04.3 LTS

   Matrix products: default
   BLAS:   /home/btyner/R360/lib64/R/lib/libRblas.so
   LAPACK: /home/btyner/R360/lib64/R/lib/libRlapack.so

   locale:
    [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
    [3] LC_TIME=en_US.UTF-8    LC_COLLATE=en_US.UTF-8
    [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
    [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
    [9] LC_ADDRESS=C   LC_TELEPHONE=C
   [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

   attached base packages:
   [1] stats graphics  grDevices utils datasets  methods base

   loaded via a namespace (and not attached):
   [1] compiler_3.6.0

Regards,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] BUG?: A copy of base::`+` (primitive) is not a clone but a "pointer"

[Benjamin Tyner]

For what it's worth, the current behavior seems to have begun starting 
with version 3.6.0. If I run in version 3.5.3:


> p1 <- .Primitive('+') ; p2 <- p1 ; attr(p1, "myattr") <- 1 ; p2
function (e1, e2)  .Primitive("+")



On 11/18/19 10:45 AM, Martin Maechler wrote:
>>/Tomas Kalibera />>/on Mon, 18 Nov 2019 09:36:14 +0100 writes: />/> On 11/18/19 9:18 AM, Martin Maechler wrote: />/>>> Henrik Bengtsson />/>>> on Sun, 17 Nov 2019 14:31:07 -0800 writes: />/>> > $ R --vanilla R version 3.6.1 (2019-07-05) -- "Action of />/>> > the Toes" Copyright (C) 2019 The R Foundation for />/>> > Statistical Computing Platform: x86_64-pc-linux-gnu />/>> > (64-bit) ... />/>> />/>> >> str(base::`+`) />/>> > function (e1, e2) />/>> />/>> >> plus <- structure(base::`+`, class = "plus") str(plus) />/>> > function (e1, e2) - attr(*, "class")= chr "plus" />/>> />/>> > ## Hmm ... />/>> >> str(base::`+`) />/>> > function (e1, e2) - attr(*, "class")= chr "plus" />/>> />/>> >> class(base::`+`) <- NULL str(base::`+`) />/>> > function (e1, e2) />/>> />/>> > ## Hmm ... />/>> >> str(plus) />/>> > function (e1, e2) />/>> 
/>/>> > Even without assigning to `plus`, you get this behavior: />/>> />/>> > $ R --vanilla />/>> >> structure(base::`+`, class = "plus") />/>> > function (e1, e2) .Primitive("+") attr(,"class") [1] />/>> > "plus" />/>> />/>> > # Hmm... />/>> >> str(base::`+`) />/>> > function (e1, e2) - attr(*, "class")= chr "plus" />/>> />/>> > Looks to be the case for common (all?) .Primitive />/>> > functions. />/>> />/>> No need for 'base::' (who would be crazy enough to redefine `+`?) />/>> nor str() actually: />/>> />/>> attr(`+`, "class") <- NULL # (reset) />/>> `+` />/>> structure(`+`, class = "plus") />/>> `+` />/>> />/>> is clearly convincing and minimal />/>> />/>>> attr(`+`, "class") <- NULL />/>>> `+` />/>> function (e1, e2) .Primitive("+") />/>>> structure(`+`, class = "plus") />/>> function (e1, e2) .Primitive("+") 
/>/>> attr(,"class") />/>> [1] "plus" />/>>> `+` />/>> function (e1, e2) .Primitive("+") />/>> attr(,"class") />/>> [1] "plus" />/>> - />/>> />/>> > Is this expected? />/>> />/>> no. (at least not by 99.999% of R users) />/>> />/>> />/>> > Should I report this one to Bugzilla? />/>> yes, please. />/>> />/>> > /Henrik />//>/> A shorter example is />//>/>> p1 <- .Primitive('+') ; p2 <- p1 ; attr(p1, "myattr") <- 1 ; p2 />//>/> function (e1, e2)  .Primitive("+") />/> attr(,"myattr") />/> [1] 1 />//>/beautiful ; thank you, Tomas ! />//>/> Builtins have referential semantics in R (like e.g. environments, but />/> also some other types). />//>/> Tomas />//>/[aarh.. I knew it ... but am showing my age: I had forgotten about it.] />//>/I forget (and don't have time just now to find out) where we />/have documented it; it may be good to document also in official 
/>/user exposed places such say the ?.Primitive help page. /
I think internal objects such as builtins should never be modified by
user code in the first place. We had to add detection for modifications
of NIL and symbols in the past, maybe we will have to do it for builtins
as well.

Environments can be used in user code as objects with referential
semantics, e.g. to keep mutable state.

Tomas
>//>/Martin /


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Should slot<-() alter its first argument?

[Benjamin Tyner]

It appears this started with R version 3.5.0. Under R 3.4.4 we have:

> setClass("Z", rep=representation(x="character"))
> z <- new("Z", x="orig")
> `@<-`(z, "x", value="newer")
An object of class "Z"
Slot "x":
[1] "newer"

> z
An object of class "Z"
Slot "x":
[1] "newer"

> `slot<-`(z, "x", value="newest")
An object of class "Z"
Slot "x":
[1] "newest"

> z
An object of class "Z"
Slot "x":
[1] "newest"



We noticed that the slot<- function alters its first argument, which goes
against the grain of a functional language.  The similar @<- does not
change its first argument.  Is this intended?  The timeSeries and distr
package depend on this altering.

>/setClass("Z", rep=representation(x="character")) />/z <- new("Z", x="orig") />/`@<-`(z, "x", 
value="newer") /An object of class "Z"
Slot "x":
[1] "newer"

>/z /An object of class "Z"
Slot "x":
[1] "orig"

>//>/`slot<-`(z, "x", value="newest") /An object of class "Z"
Slot "x":
[1] "newest"

>/z /An object of class "Z"
Slot "x":
[1] "newest"

Bill Dunlap
TIBCO Software
wdunlap tibco.com

[[alternative HTML version deleted]]


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-devel (rev 76409) fails 'make check': non-generic function 'isSymmetric' given to findMethods()

[Benjamin Tyner]

Looks fixed as of revision 76417; thanks Brian!

On 4/21/19 9:02 PM, Benjamin Tyner wrote:

Duncan that does indeed look to be the case. Many thanks!

In particular, tests/reg-tests-1d.R optionally loads the Matrix 
namespace which allows the test to succeed. Compare:


   ~/R-rc_2019-04-21_r76409/bin/Rscript -e "options(warn=2);
   library(Matrix); res <- findMethods('isSymmetric'); print('success')"
   [1] "success"

versus

   ~/R-rc_2019-04-21_r76409/bin/Rscript -e "options(warn=2); res <-
   findMethods('isSymmetric'); print('success')"
   Error in findMethods("isSymmetric") :
      (converted from warning) non-generic function 'isSymmetric' given
   to findMethods()
   Execution halted


On 4/21/19 7:47 PM, Duncan Murdoch wrote:


Likely the problem is that you don't have the recommended packages 
loaded.  When I was running tests regularly, they were required. 
Later, I think they became optional.  Perhaps a new test has been 
added that once again assumes the required packages are installed.


Duncan Murdoch



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R-devel (rev 76409) fails 'make check': non-generic function 'isSymmetric' given to findMethods()

[Benjamin Tyner]

Duncan that does indeed look to be the case. Many thanks!

In particular, tests/reg-tests-1d.R optionally loads the Matrix 
namespace which allows the test to succeed. Compare:


   ~/R-rc_2019-04-21_r76409/bin/Rscript -e "options(warn=2);
   library(Matrix); res <- findMethods('isSymmetric'); print('success')"
   [1] "success"

versus

   ~/R-rc_2019-04-21_r76409/bin/Rscript -e "options(warn=2); res <-
   findMethods('isSymmetric'); print('success')"
   Error in findMethods("isSymmetric") :
  (converted from warning) non-generic function 'isSymmetric' given
   to findMethods()
   Execution halted


On 4/21/19 7:47 PM, Duncan Murdoch wrote:


Likely the problem is that you don't have the recommended packages 
loaded.  When I was running tests regularly, they were required. 
Later, I think they became optional.  Perhaps a new test has been 
added that once again assumes the required packages are installed.


Duncan Murdoch



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R-devel (rev 76409) fails 'make check': non-generic function 'isSymmetric' given to findMethods()

[Benjamin Tyner]

Hello,

Most likely I'm doing something wrong, but am at a loss as to what the 
issue is. I have a clean checkout of trunk here:


   ~/svn/r-devel/R$ svn info
   Path: .
   Working Copy Root Path: /home/btyner/svn/r-devel/R
   URL: https://svn.r-project.org/R/trunk
   Relative URL: ^/trunk
   Repository Root: https://svn.r-project.org/R
   Repository UUID: 00db46b3-68df-0310-9c12-caf00c1e9a41
   Revision: 76409
   Node Kind: directory
   Schedule: normal
   Last Changed Author: ripley
   Last Changed Rev: 76409
   Last Changed Date: 2019-04-21 06:32:24 -0400 (Sun, 21 Apr 2019)

I then ran:

   mkdir ~/svn/R-devel-build
   cd ~/svn/R-devel-build
   ../r-devel/R/configure --prefix=/home/btyner/R-76409 
--enable-R-shlib --with-blas --with-lapack --with-readline 
--without-recommended-packages

   make
   make check

which fails at reg-tests-1d.R; the failure being:

   > ## str() now even works with invalid objects:
   > moS <- mo <- findMethods("isSymmetric")
   Error in findMethods("isSymmetric") :
     (converted from warning) non-generic function 'isSymmetric' given 
to findMethods()

   Execution halted

I am able to build revision 76407 from source just fine (using 
https://cran.r-project.org/src/base-prerelease/R-devel_2019-04-20_r76407.tar.gz) 
and all make check tests are successful.


Any ideas? Here is my sessionInfo() from the revision 76407 build:

   R Under development (unstable) (2019-04-20 r76407)
   Platform: x86_64-pc-linux-gnu (64-bit)
   Running under: Ubuntu 18.04.2 LTS

   Matrix products: default
   BLAS: /home/btyner/R-devel_2019-04-20_r76407/lib64/R/lib/libRblas.so
   LAPACK: /home/btyner/R-devel_2019-04-20_r76407/lib64/R/lib/libRlapack.so

   locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8    LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
   [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

   attached base packages:
   [1] stats graphics  grDevices utils datasets  methods base

   loaded via a namespace (and not attached):
   [1] compiler_3.7.0

Regards,

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] longint

[Benjamin Tyner]

Thanks for the replies and for confirming my suspicion.

Interestingly, src/include/S.h uses a trick:

   #define longint int

and so does the nlme package (within src/init.c).

On 08/15/2018 02:47 PM, Hervé Pagès wrote:
No segfault but a BIG warning from the compiler. That's because 
dereferencing the pointer inside your myfunc() function will

produce an int that is not predictable i.e. it is system-dependent.
Its value will depend on sizeof(long int) (which is not
guaranteed to be 8) and on the endianness of the system.

Also if the pointer you pass in the call to the function is
an array of long ints, then pointer arithmetic inside your myfunc()
won't necessarily take you to the array element that you'd expect.

Note that there are very specific situations where you can actually
do this kind of things e.g. in the context of writing a callback
function to pass to qsort(). See 'man 3 qsort' if you are on a Unix
system. In that case pointers to void and explicit casts should
be used. If done properly, this is portable code and the compiler won't
issue warnings.

H.


On 08/15/2018 07:05 AM, Brian Ripley wrote:



On 15 Aug 2018, at 12:48, Duncan Murdoch  
wrote:



On 15/08/2018 7:08 AM, Benjamin Tyner wrote:
Hi
In my R package, imagine I have a C function defined:
 void myfunc(int *x) {
    // some code
 }
but when I call it, I pass it a pointer to a longint instead of a
pointer to an int. Could this practice potentially result in a 
segfault?


I don't think the passing would cause a segfault, but "some code" 
might be expecting a positive number, and due to the type error you 
could pass in a positive longint and have it interpreted as a 
negative int.


Are you thinking only of a little-endian system?  A 32-bit lookup of 
a pointer to a 64-bit area could read the wrong half and get a 
completely different value.




Duncan Murdoch

__
R-devel@r-project.org mailing list
https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel=DwIFAg=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=ERck0y30d00Np6hqTNYfjusx1beZim0OrKe9O4vkUxU=x1gI9ACZol7WbaWQ7Ocv60csJFJClZotWkJIMwUdjIc= 



__
R-devel@r-project.org mailing list
https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel=DwIFAg=eRAMFD45gAfqt84VtBcfhQ=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA=ERck0y30d00Np6hqTNYfjusx1beZim0OrKe9O4vkUxU=x1gI9ACZol7WbaWQ7Ocv60csJFJClZotWkJIMwUdjIc= 







__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] longint

[Benjamin Tyner]

Hi

In my R package, imagine I have a C function defined:

   void myfunc(int *x) {
  // some code
   }

but when I call it, I pass it a pointer to a longint instead of a 
pointer to an int. Could this practice potentially result in a segfault?


Regards
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Library lib.loc Option Ignored for Dependencies

[Benjamin Tyner]

On 07/25/2018 04:28 AM, Uwe Ligges wrote:


Not sure if this is a bug: where is it documented that this works not 
as above (which is what I would expect)?
The documentation doesn't say one way or the other. But the fact remains 
that base:::.getRequiredPackages2, when provided a non-null lib.loc, 
uses that lib.loc when checking versions of dependencies. So (to me) it 
seems a little incongruous that base:::.getRequiredPackages2 offers this 
functionality but library() doesn't take advantage of it. Moreover, it 
is already passing the 'quietly' argument...


For development puposes, you may want to try some devekopment version 
of one package agaiunst release versions of the others.
Agreed, in which case I would explicitly load the dependency first 
specifying a custom lib.loc.
This is the purpose of the above: take spam from ~lib and the other 
from .libPath().


Best,
Uwe Ligges



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Library lib.loc Option Ignored for Dependencies

[Benjamin Tyner]

On 07/24/2018 07:50 AM, Martin Maechler wrote:

Benjamin Tyner
 on Sat, 21 Jul 2018 13:42:43 -0400 writes:

 > Not sure whether it is the same issue as was raised here:
 > https://stat.ethz.ch/pipermail/r-devel/2010-October/058729.html

 > but in any case perhaps the problem could partially be remedied on line
 > 245 of src/library/base/R/library.R by passing the lib.loc to
 > .getRequiredPackages2() ...here is a patch (untested)

 > Index: src/library/base/R/library.R
 > ===
 > --- src/library/base/R/library.R    (revision 74997)
 > +++ src/library/base/R/library.R    (working copy)
 > @@ -242,7 +242,7 @@
 >  pos <- 2
 >  } else pos <- npos
 >  }
 > -    .getRequiredPackages2(pkgInfo, quietly = quietly)
 > +    .getRequiredPackages2(pkgInfo, lib.loc = lib.loc, quietly = 
quietly)
 >  deps <- unique(names(pkgInfo$Depends))

 >  ## If the namespace mechanism is available and the package

This - directly - fails even more miserably e.g. in my own setup
when I have dependency to my package.

But it seems a good idea to use 'lib.loc', and safer and
probably better than the current code maybe to use

   .getRequiredPackages2(pkgInfo,
lib.loc = c(lib.loc, .libPaths()),
quietly = quietly)

instead of the current code which uses lib.loc = NULL
equivalently to   lib.loc = .libPaths()

Seems reasonable to me.


Others / ideas?
Reproducible examples with small fake packages?

Or how about an example with a "real" CRAN package with just one dependency:

    > dir.create("~/lib")
    > list.files("~/lib")
    character(0)
    > install.packages("spam", lib = "~/lib", dependencies = TRUE)
    > list.files("~/lib")
    [1] "dotCall64" "spam"
    > library(spam, lib.loc = "~/lib")
    Error: package ‘dotCall64’ required by ‘spam’ could not be found



Martin

 > On 07/21/2018 12:34 PM, Martin Maechler wrote:
 >>>>>>> Benjamin Tyner
 >>>>>>> on Fri, 20 Jul 2018 19:42:09 -0400 writes:
 >> > Here's a trick/workaround; if lib.loc is the path to your
 >> > library, then prior to calling library(),
 >>
 >> >> environment(.libPaths)$.lib.loc <- lib.loc
 >>
 >> Well, that is quite a "trick"  -- and potentially a pretty
 >> dangerous one, not intended when making .libPaths a closure 
 >>
 >>
 >> I do think that there is a problem with R's dealing of R_LIBS
 >> and other libPaths settings, notably when checking packages and
 >> within that recompiling vignettes etc, where the R process
 >> starts new versions of R via system() / system2() and then gets
 >> to wrong .libPaths() settings,
 >> and I personally would be very happy if we got reprex'es with
 >> small fake packages -- possibly only easily reproducible on
 >> unix-alikes ... so we could address this as a bug (or more than
 >> one) to be fixed.
 >>
 >> Notably with the 3.4.x --> 3.5.0 transition and my/our tendency
 >> of having quite a few paths in R_LIBS / lib.loc / ... I've been
 >> bitten by problems when the wrong version of package was taken
 >> from the wrong library path 
 >>
 >> Martin
 >>
 >>
 >> >> 

 >> >> Good day,
 >> >>
 >> >> If there's a library folder of the latest R packages and
 >> >> a particular package from it is loaded using the lib.loc
 >> >> option, the dependencies of that package are still
 >> >> attempted to be loaded from another folder of older
 >> >> packages specified by R_LIBS, which may cause errors
 >> >> about version requirements not being met. The
 >> >> documentation of the library function doesn't explain
 >> >> what the intended result is in such a case, but it could
 >> >> reasonably be expected that R would also load the
 >> >> dependencies from the user-specified lib.loc folder.
 >> >>
 >> >> --
 >> >> Dario Strbenac University of Sydney Camperdown NSW 2050
 >> >> Australia


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Should there be a confint.mlm ?

[Benjamin Tyner]

BTW --- and this is a diversion ---  This is nice mathematically
 (and used in other places, also in "base R" I think)
but in principle is a waste:  Computing a full
k x k matrix and then throwing away all but the length-k
diagonal ...
In the past I had contemplated but never RFC'ed or really
implemented a stderr() generic with default method

stderr.default <- function(object) sqrt(diag(vcov(object)))

but allow non-default methods to be smarter and hence more efficient.

A fine idea for sure, but perhaps use a different name in order to avoid 
conflicts with:


> base::stderr
function ()
.Internal(stderr())



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Library lib.loc Option Ignored for Dependencies

[Benjamin Tyner]

Not sure whether it is the same issue as was raised here:

https://stat.ethz.ch/pipermail/r-devel/2010-October/058729.html

but in any case perhaps the problem could partially be remedied on line 
245 of src/library/base/R/library.R by passing the lib.loc to 
.getRequiredPackages2() ...here is a patch (untested)


Index: src/library/base/R/library.R
===
--- src/library/base/R/library.R    (revision 74997)
+++ src/library/base/R/library.R    (working copy)
@@ -242,7 +242,7 @@
 pos <- 2
 } else pos <- npos
 }
-    .getRequiredPackages2(pkgInfo, quietly = quietly)
+    .getRequiredPackages2(pkgInfo, lib.loc = lib.loc, quietly = 
quietly)

 deps <- unique(names(pkgInfo$Depends))

 ## If the namespace mechanism is available and the package


On 07/21/2018 12:34 PM, Martin Maechler wrote:

Benjamin Tyner
 on Fri, 20 Jul 2018 19:42:09 -0400 writes:

 > Here's a trick/workaround; if lib.loc is the path to your
 > library, then prior to calling library(),

 >> environment(.libPaths)$.lib.loc <- lib.loc

Well, that is quite a "trick"  -- and potentially a pretty
dangerous one, not intended when making .libPaths a closure 


I do think that there is a problem with R's dealing of R_LIBS
and other libPaths settings, notably when checking packages and
within that recompiling vignettes etc, where the R process
starts new versions of R via system() / system2() and then gets
to wrong .libPaths() settings,
and I personally would be very happy if we got reprex'es with
small fake packages -- possibly only easily reproducible on
unix-alikes ... so we could address this as a bug (or more than
one) to be fixed.

Notably with the 3.4.x --> 3.5.0 transition and my/our tendency
of having quite a few paths in R_LIBS / lib.loc / ... I've been
bitten by problems when the wrong version of package was taken
from the wrong library path 

Martin


 >> 
 >> Good day,
 >>
 >> If there's a library folder of the latest R packages and
 >> a particular package from it is loaded using the lib.loc
 >> option, the dependencies of that package are still
 >> attempted to be loaded from another folder of older
 >> packages specified by R_LIBS, which may cause errors
 >> about version requirements not being met. The
 >> documentation of the library function doesn't explain
 >> what the intended result is in such a case, but it could
 >> reasonably be expected that R would also load the
 >> dependencies from the user-specified lib.loc folder.
 >>
 >> --
 >> Dario Strbenac University of Sydney Camperdown NSW 2050
 >> Australia


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Library lib.loc Option Ignored for Dependencies

[Benjamin Tyner]

Here's a trick/workaround; if lib.loc is the path to your library, then 
prior to calling library(),


> environment(.libPaths)$.lib.loc <- lib.loc



Good day,

If there's a library folder of the latest R packages and a particular package 
from it is loaded using the lib.loc option, the dependencies of that package 
are still attempted to be loaded from another folder of older packages 
specified by R_LIBS, which may cause errors about version requirements not 
being met. The documentation of the library function doesn't explain what the 
intended result is in such a case, but it could reasonably be expected that R 
would also load the dependencies from the user-specified lib.loc folder.

--
Dario Strbenac
University of Sydney
Camperdown NSW 2050
Australia



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] missing news entry?

[Benjamin Tyner]

Hi,

Unless I am mistaken, this enhancement to gc():


r73749 | luke | 2017-11-18 13:26:25 -0500 (Sat, 18 Nov 2017) | 2 lines

Added 'full' argument to gc() with default 'TRUE' for now.



appears to be lacking an entry in doc/NEWS.Rd. Just FYI, in case there 
is capacity to add one.


Regards

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

[Benjamin Tyner]

True or False: when USE_OPENMP=1 is not used, then race conditions are 
not unexpected.


If True, and we wish to avoid race conditions, then sources such as the 
conda channel and ubuntu would need to add this enhancement.


If False, then what is the next step (i.e. forum) for debugging the race 
condition?



On 01/11/2018 07:56 AM, Ista Zahn wrote:



On Jan 10, 2018 8:24 PM, "Benjamin Tyner" <bty...@gmail.com 
<mailto:bty...@gmail.com>> wrote:


Thanks Keith. We checked, and indeed libopenblas is not linked
against libomp nor libgomp. We suspect this is because we used
conda to install R and OpenBLAS. So I guess we should be barking
up the conda tree instead?


What are you barking about? I don't understand what you are trying to 
accomplish.


By the way, I also noticed on my home machine (Ubuntu),
/usr/lib/libopenblas.so.0 is also not linked against those, for
what that's worth.

Regards,
Ben


On 01/10/2018 12:04 AM, Keith O'Hara wrote:

Check if libopenblas is linked against libomp or libgomp.

I’d be curious to see any errors that arise when an OpenMP
version of OpenBLAS is linked with R.

Keith


On Jan 9, 2018, at 11:01 PM, Benjamin Tyner
<bty...@gmail.com <mailto:bty...@gmail.com>> wrote:

I didn't do the compile; is there a way to check whether
that was used? If not, I'll inquire with our sysadmin and
report back.

In any case, my suggestion was motivated by the fact that
some parts of R use OpenMP while others do not, in the
hope that the former could have their OpenBLAS omelet
without breaking the OpenMP eggs, so to speak.


On 01/09/2018 06:41 PM, Keith O'Hara wrote:

Do those issues still arise when OpenBLAS is compiled
with USE_OPENMP=1 ?

Keith

    On Jan 9, 2018, at 6:03 PM, Benjamin Tyner
<bty...@gmail.com <mailto:bty...@gmail.com>> wrote:

Please pardon my ignorance, but doesn't OpenBLAS
still not always play nicely with multi-threaded
OpenMP? (for example, don't race conditions
sometimes crop up)? If so, it might be nice to
have the ability to temporarily disable
multi-threaded OpenMP (effectively:
omp_set_num_threads(1)) for the duration of
operations using OpenBLAS.

Regards
Ben

Julia using OpenBLAS is *very* reassuring.

I agree that having it included as an
options(...) feature should be OK.

On Sun, Dec 17, 2017, 3:22 PM Juan Telleria
http://gmail.com>
<https://stat.ethz.ch/mailman/listinfo/r-devel
<https://stat.ethz.ch/mailman/listinfo/r-devel>>>
wrote:

/Julia Programming Language uses also
OpenBlas, and it is actively />/maintained
with bugs being fixed as I have checked it
out:
/>//>/http://www.openblas.net/Changelog.txt
<http://www.openblas.net/Changelog.txt>
/>//>/So I still see it ok to be included
as an options(...) feature (by default
/>/off, just for safety), over other Blas
libraries. />//>/R could not use Intel MKL
for legal reasons (I think), because as
long />/that R ships with GPL libraries,
shipping R by default with Non-GPL is
/>/illegal. />//>/Cheers, />/Juan />//>/El
17/12/2017 2:50 a. m., "Avraham Adler"
http://gmail.com>
<https://stat.ethz.ch/mailman/listinfo/r-devel
<https://stat.ethz.ch/mailman/listinfo/r-devel>>>
/>/escribió: />//>>/On Sat, Dec 16, 2017
at 7:41 PM, Kenny Bell http://gmail.com>
<https://stat.ethz.ch/mailman/listinfo/r-devel
<https://stat.ethz.ch/mailman/listinfo/r-devel>>>
wrote: />>/> It seems like many of the
multi-threaded BLASes have some sort of
/>>/> fundamental problem preventing us

Re: [Rd] OpenBLAS in everyday R?

[Benjamin Tyner]

Thanks Keith. We checked, and indeed libopenblas is not linked against 
libomp nor libgomp. We suspect this is because we used conda to install 
R and OpenBLAS. So I guess we should be barking up the conda tree instead?


By the way, I also noticed on my home machine (Ubuntu), 
/usr/lib/libopenblas.so.0 is also not linked against those, for what 
that's worth.


Regards,
Ben

On 01/10/2018 12:04 AM, Keith O'Hara wrote:

Check if libopenblas is linked against libomp or libgomp.

I’d be curious to see any errors that arise when an OpenMP version of OpenBLAS 
is linked with R.

Keith



On Jan 9, 2018, at 11:01 PM, Benjamin Tyner <bty...@gmail.com> wrote:

I didn't do the compile; is there a way to check whether that was used? If not, 
I'll inquire with our sysadmin and report back.

In any case, my suggestion was motivated by the fact that some parts of R use 
OpenMP while others do not, in the hope that the former could have their 
OpenBLAS omelet without breaking the OpenMP eggs, so to speak.


On 01/09/2018 06:41 PM, Keith O'Hara wrote:

Do those issues still arise when OpenBLAS is compiled with USE_OPENMP=1 ?

Keith


On Jan 9, 2018, at 6:03 PM, Benjamin Tyner <bty...@gmail.com> wrote:

Please pardon my ignorance, but doesn't OpenBLAS still not always play nicely 
with multi-threaded OpenMP? (for example, don't race conditions sometimes crop 
up)? If so, it might be nice to have the ability to temporarily disable 
multi-threaded OpenMP (effectively: omp_set_num_threads(1)) for the duration of 
operations using OpenBLAS.

Regards
Ben


Julia using OpenBLAS is *very* reassuring.

I agree that having it included as an options(...) feature should be OK.

On Sun, Dec 17, 2017, 3:22 PM Juan Telleria https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote:


/Julia Programming Language uses also OpenBlas, and it is actively />/maintained with bugs being fixed as I have checked it out: />//>/http://www.openblas.net/Changelog.txt />//>/So I still see it ok to be included as an options(...) feature (by default />/off, just for safety), over other Blas libraries. />//>/R could not use Intel MKL for legal reasons (I think), because as long />/that R ships with GPL libraries, shipping R by default with Non-GPL is />/illegal. />//>/Cheers, />/Juan />//>/El 17/12/2017 2:50 a. m., "Avraham Adler" https://stat.ethz.ch/mailman/listinfo/r-devel>> />/escribió: />//>>/On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> It seems like many of the multi-threaded BLASes have some sort of />>/> fundamental problem preventing use in the way Juan suggests: />>/> />>/> - Dirk's vignette states that ATLAS "fixes the number of cores used at />>/> compile-time and cannot vary this setting at run-time", so any />>/> user-friendly 
implementation for R would have to compile ATLAS for 1-16 />>/> threads to allow the user to switch at run-time. This might dramatically />>/> affect install times. />>/> />>/> - MKL seems like it's been outright rejected in the past based on not />>/> being "free-enough". />>/> />>/> - OpenBLAS causes crashes. />>/> />>/> Has anyone tried ExBLAS for use with R? />>/> />>/> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < />>/> peter.langfelder at gmail.com 
<https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> />>/>> I would be very cautious about OpenBLAS in particular... from time to />>/>> time I get complains from users that compiled code calculations in my />>/>> WGCNA package crash or produce wrong answers with large data, and they />>/>> all come from OpenBLAS users. I am yet to reproduce any of their />>/>> crashes when using MKL and ATLAS BLAS implementations. />>/>> />>/>> Just my 2 cents... />>/>> />>/>> Peter 
/>>//>>/I've been building R on Windows 64 bit with OpenBLAS for years and my />>/builds pass check-devel. For a while in the past it failed one check />>/as the tolerance was 5e-5 and with my build of OpenBLAS the error was />>/5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall />>/correctly. I provide descriptions here [1], but I haven't gone so far />>/as to post compiled Rblas.dlls just yet. My personal build sets 4 />>/threads when compiling OpenBLAS itself as I'm currently on a quad-core />>/SandyBridge. In tests I ran a few years ago, both 
single and multi />>/threaded BLAS compile and then can be compiled into R with no issues />>/(on my platforms, at least). Most matrix operations performed better />>/with multi-threaded except fo

Re: [Rd] OpenBLAS in everyday R?

[Benjamin Tyner]

I didn't do the compile; is there a way to check whether that was used? 
If not, I'll inquire with our sysadmin and report back.


In any case, my suggestion was motivated by the fact that some parts of 
R use OpenMP while others do not, in the hope that the former could have 
their OpenBLAS omelet without breaking the OpenMP eggs, so to speak.



On 01/09/2018 06:41 PM, Keith O'Hara wrote:

Do those issues still arise when OpenBLAS is compiled with USE_OPENMP=1 ?

Keith


On Jan 9, 2018, at 6:03 PM, Benjamin Tyner <bty...@gmail.com> wrote:

Please pardon my ignorance, but doesn't OpenBLAS still not always play nicely 
with multi-threaded OpenMP? (for example, don't race conditions sometimes crop 
up)? If so, it might be nice to have the ability to temporarily disable 
multi-threaded OpenMP (effectively: omp_set_num_threads(1)) for the duration of 
operations using OpenBLAS.

Regards
Ben


Julia using OpenBLAS is *very* reassuring.

I agree that having it included as an options(...) feature should be OK.

On Sun, Dec 17, 2017, 3:22 PM Juan Telleria https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote:


/Julia Programming Language uses also OpenBlas, and it is actively />/maintained with bugs being fixed as I have checked it out: />//>/http://www.openblas.net/Changelog.txt />//>/So I still see it ok to be included as an options(...) feature (by default />/off, just for safety), over other Blas libraries. />//>/R could not use Intel MKL for legal reasons (I think), because as long />/that R ships with GPL libraries, shipping R by default with Non-GPL is />/illegal. />//>/Cheers, />/Juan />//>/El 17/12/2017 2:50 a. m., "Avraham Adler" https://stat.ethz.ch/mailman/listinfo/r-devel>> />/escribió: />//>>/On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> It seems like many of the multi-threaded BLASes have some sort of />>/> fundamental problem preventing use in the way Juan suggests: />>/> />>/> - Dirk's vignette states that ATLAS "fixes the number of cores used at />>/> compile-time and cannot vary this setting at run-time", so any />>/> user-friendly 
implementation for R would have to compile ATLAS for 1-16 />>/> threads to allow the user to switch at run-time. This might dramatically />>/> affect install times. />>/> />>/> - MKL seems like it's been outright rejected in the past based on not />>/> being "free-enough". />>/> />>/> - OpenBLAS causes crashes. />>/> />>/> Has anyone tried ExBLAS for use with R? />>/> />>/> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < />>/> peter.langfelder at gmail.com 
<https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote: />>/> />>/>> I would be very cautious about OpenBLAS in particular... from time to />>/>> time I get complains from users that compiled code calculations in my />>/>> WGCNA package crash or produce wrong answers with large data, and they />>/>> all come from OpenBLAS users. I am yet to reproduce any of their />>/>> crashes when using MKL and ATLAS BLAS implementations. />>/>> />>/>> Just my 2 cents... />>/>> />>/>> Peter 
/>>//>>/I've been building R on Windows 64 bit with OpenBLAS for years and my />>/builds pass check-devel. For a while in the past it failed one check />>/as the tolerance was 5e-5 and with my build of OpenBLAS the error was />>/5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall />>/correctly. I provide descriptions here [1], but I haven't gone so far />>/as to post compiled Rblas.dlls just yet. My personal build sets 4 />>/threads when compiling OpenBLAS itself as I'm currently on a quad-core />>/SandyBridge. In tests I ran a few years ago, both 
single and multi />>/threaded BLAS compile and then can be compiled into R with no issues />>/(on my platforms, at least). Most matrix operations performed better />>/with multi-threaded except for R's eigenvalue decomposition, to the />>/nest of my recollection. />>//>>/Avi />>//>>/[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ />>//>>/__ />>/R-devel at r-project.org <https://stat.ethz.ch/mailman/listinfo/r-devel> mailing list 
/>>/https://stat.ethz.ch/mailman/listinfo/r-devel />>//>//

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

[Benjamin Tyner]

Please pardon my ignorance, but doesn't OpenBLAS still not always play 
nicely with multi-threaded OpenMP? (for example, don't race conditions 
sometimes crop up)? If so, it might be nice to have the ability to 
temporarily disable multi-threaded OpenMP (effectively: 
omp_set_num_threads(1)) for the duration of operations using OpenBLAS.


Regards
Ben


Julia using OpenBLAS is *very* reassuring.

I agree that having it included as an options(...) feature should be OK.

On Sun, Dec 17, 2017, 3:22 PM Juan Telleria https://stat.ethz.ch/mailman/listinfo/r-devel>> wrote:

>/Julia Programming Language uses also OpenBlas, and it is actively />/maintained with bugs being fixed as I have checked it out: />//>/http://www.openblas.net/Changelog.txt />//>/So I still see it ok to be included as an options(...) feature (by 
default />/off, just for safety), over other Blas libraries. />//>/R could not use Intel MKL for legal reasons (I think), because as long />/that R ships with GPL libraries, shipping R by default with Non-GPL is />/illegal. />//>/Cheers, />/Juan />//>/El 17/12/2017 2:50 a. m., "Avraham Adler" > />/escribió: />//>>/On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell > wrote: />>/> It seems like many of the multi-threaded BLASes have some sort of />>/> fundamental problem preventing use in the way Juan suggests: />>/> />>/> - Dirk's vignette states that ATLAS "fixes the number of cores used at />>/> compile-time and cannot vary this setting at run-time", so any />>/> user-friendly implementation for R would have to compile ATLAS for 
1-16 />>/> threads to allow the user to switch at run-time. This might 
dramatically />>/> affect install times. />>/> />>/> - MKL seems like it's been outright rejected in the past based on not />>/> being "free-enough". />>/> />>/> - OpenBLAS causes crashes. />>/> />>/> Has anyone tried ExBLAS for use with R? />>/> />>/> On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < />>/> peter.langfelder at gmail.com 
> wrote: />>/> />>/>> I would be very cautious about OpenBLAS in particular... from time to />>/>> time I get complains from users that compiled code calculations in my />>/>> WGCNA package crash or produce wrong answers with large data, and 
they />>/>> all come from OpenBLAS users. I am yet to reproduce any of their />>/>> crashes when using MKL and ATLAS BLAS implementations. />>/>> />>/>> Just my 2 cents... />>/>> />>/>> Peter />>//>>/I've been building R on Windows 64 bit with OpenBLAS for years and my />>/builds pass check-devel. For a while in the past it failed one check />>/as the tolerance was 5e-5 and with my build of OpenBLAS the error was />>/5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall />>/correctly. I provide descriptions here [1], but I haven't gone so far />>/as to post compiled Rblas.dlls just yet. My personal build sets 4 />>/threads when compiling OpenBLAS itself as I'm currently on a quad-core />>/SandyBridge. In tests I ran a few years ago, both single and multi />>/threaded BLAS compile and then can be compiled into R with no issues />>/(on my platforms, at least). Most matrix operations performed better />>/with multi-threaded except for R's eigenvalue decomposition, to the />>/nest of my recollection. />>//>>/Avi />>//>>/[1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ />>//>>/__ />>/R-devel at r-project.org 
 mailing list />>/https://stat.ethz.ch/mailman/listinfo/r-devel />>//>//

[[alternative HTML version deleted]]


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Coping with non-standard evaluation in R program analysis

[Benjamin Tyner]

Evan,

The pryr package provides some utilities which may be handy here. In 
particular see the function: is_promise.


Also, next time please post in plaintext.

Regards
Ben

Hello R experts,


I plan to develop a tool for dynamic analysis of R programs. I would like to 
trace function calls at runtime, capturing argument and return values. 
Following a suggestion made some time ago on this list, my high-level 
implementation strategy is to rewrite the AST, augmenting call expressions with 
pre-call and post-call shims to capture the arguments and return value, 
respectively.


I can think of only one fundamental conceptual obstacle to this approach: R functions are not 
necessarily referentially transparent. The arguments received by a function are not values but 
promises. They can be evaluated directly ("standard evaluation"), after applying 
arbitrary syntactic transformations ("non-standard evaluation", aka NSE), or not at all. 
Therefore, if you peek at the values of function arguments before evaluating the function, you risk 
altering the semantics of the program, possibly fatally.


I'm looking for general advice about how to cope with NSE in this context. I 
also have some specific questions:


1) Is it possible to determine whether a given function (primitive, in R, or 
external) uses NSE on some or all of its arguments?


2) Is it possible to inspect the promise objects received by functions, say to 
determine whether they have been evaluated, without actually evaluating them? 
The R manual is not encouraging in this area:


https://cran.r-project.org/doc/manuals/r-release/R-lang.html#Promise-objects


Thank you,


Evan


[[alternative HTML version deleted]]



--

Subject: Digest Footer

___
R-devel@r-project.org mailing list  DIGESTED
https://stat.ethz.ch/mailman/listinfo/r-devel

--

End of R-devel Digest, Vol 179, Issue 1
***


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] force promises inside lapply

[Benjamin Tyner]

Thanks again Bill; I agree that substitute is overkill here.

As an aside, for cases where someone may be tempted to use substitute(), 
it seems quote() might be a safer alternative; compare


   > lapply(list(1), function(y) c(quote(y), substitute(y)))
   [[1]]
   [[1]][[1]]
   y

   [[1]][[2]]
   X[[i]]

versus in R < 3.2,

   > lapply(list(1), function(y) c(quote(y), substitute(y)))
   [[1]]
   [[1]][[1]]
   y

   [[1]][[2]]
   X[[1L]]

in any case, the lesson seems to be that quote and substitute are not 
interchangeable, even though for example


   > (function() identical(quote({a}), substitute({a})))()
   [1] TRUE


On 07/29/2017 09:39 AM, William Dunlap wrote:
Functions, like your loader(), that use substitute to let users 
confound things and their names, should give the user a way to avoid 
the use of substitute.  E.g., library() has the 'character.only' 
argument; if TRUE then the package argument is treated as an ordinary 
argument and not passed through substitute().


myLoader <- function(package, quietly = TRUE) {
   wrapper <- if (quietly) suppressPackageStartupMessages else `{`
   wrapper(library(package = package, character.only=TRUE))
   }

> lapply(c("MASS","boot"), myLoader, quietly=FALSE)
[[1]]
 [1] "MASS"  "splines"   "pryr"  "stats" "graphics"  "grDevices"
 [7] "utils" "datasets"  "methods"   "base"

[[2]]
 [1] "boot"  "MASS"  "splines"   "pryr"  "stats" 
"graphics"

 [7] "grDevices" "utils" "datasets"  "methods"   "base"

"Non-standard" evaluation (using substitute(), formulas, promises, the 
rlang or lazyeval packages, etc.) has it uses but I wouldn't use it 
for such a function as your loader().



Bill Dunlap
TIBCO Software
wdunlap tibco.com <http://tibco.com>

On Fri, Jul 28, 2017 at 8:20 PM, Benjamin Tyner <bty...@gmail.com 
<mailto:bty...@gmail.com>> wrote:


Thanks Bill. I think my confusion may have been in part due to my
conflating two distinct meanings of the term "evaluate"; the help
for force says it "forces the evaluation of a function argument"
whereas the help for eval says it "evaluates the ... argument ...
and returns the computed value". I found it helpful to compare:

   > lapply(list(a=1,b=2,c=3), function(x){ force(substitute(x)) })
   $a
   X[[i]]

   $b
   X[[i]]

   $c
   X[[i]]

versus

   > lapply(list(a=1,b=2,c=3), function(x){ eval(substitute(x)) })
   Error in eval(substitute(x)) : object 'X' not found

Now for the context my question arose in: given a function

   loader <- function(package, quietly = TRUE) {

   wrapper <- if (quietly) suppressPackageStartupMessages else `{`

   expr <- substitute(wrapper(library(package = package)))

   eval(expr)
   }

prior to R version 3.2, one could do things like

lapply(c("MASS", "boot"), loader)

but not anymore (which is fine; I agree that one should not depend
on lapply's implementation details).

Regards,
Ben


On 07/28/2017 06:53 PM, William Dunlap wrote:

1: substitute(), when given an argument to a function (which
will be a promise) gives you the unevaluated expression given
as the argument:

>  L <- list(a=1, b=2, c=3)
> str(lapply(L, function(x) substitute(x)))
List of 3
 $ a: language X[[i]]
 $ b: language X[[i]]
 $ c: language X[[i]]

The 'X' and 'i' are in a frame constructed by lapply and you
are not really supposed to depend on the precise form of those
expressions.

2: An evaluated promise is still a promise: it has the
'evaled' field set to TRUE and the 'value' field set to the
result of evaluating 'code' in 'env'.

> f <- function(x, force) {
 if (force) force(x)
 if (pryr::is_promise(x)) promise_info(x)
 else "not a promise"
 }
> str(f(log(-1), force=FALSE))
List of 4
 $ code  : language log(-1)
 $ env   :
 $ evaled: logi FALSE
 $ value : NULL
> str(f(log(-1), force=TRUE))
List of 4
 $ code  : language log(-1)
     $ env   : NULL
 $ evaled: logi TRUE
 $ value : num NaN
Warning message:
In log(-1) : NaNs produced

Can you give a concrete example of what you are try to accomplish?

Bill Dunlap
TIBCO Software
wdunlap tibco.com <http://tibco.com> <http://tibco.com>


On Fri, Jul 28, 2017 at 3:04 PM, Benjamin Tyner
<bty...@g

Re: [Rd] force promises inside lapply

[Benjamin Tyner]

Thanks Bill. I think my confusion may have been in part due to my 
conflating two distinct meanings of the term "evaluate"; the help for 
force says it "forces the evaluation of a function argument" whereas the 
help for eval says it "evaluates the ... argument ... and returns the 
computed value". I found it helpful to compare:


   > lapply(list(a=1,b=2,c=3), function(x){ force(substitute(x)) })
   $a
   X[[i]]

   $b
   X[[i]]

   $c
   X[[i]]

versus

   > lapply(list(a=1,b=2,c=3), function(x){ eval(substitute(x)) })
   Error in eval(substitute(x)) : object 'X' not found

Now for the context my question arose in: given a function

   loader <- function(package, quietly = TRUE) {

   wrapper <- if (quietly) suppressPackageStartupMessages else `{`

   expr <- substitute(wrapper(library(package = package)))

   eval(expr)
   }

prior to R version 3.2, one could do things like

lapply(c("MASS", "boot"), loader)

but not anymore (which is fine; I agree that one should not depend on 
lapply's implementation details).


Regards,
Ben

On 07/28/2017 06:53 PM, William Dunlap wrote:
1: substitute(), when given an argument to a function (which will be a 
promise) gives you the unevaluated expression given as the argument:


>  L <- list(a=1, b=2, c=3)
> str(lapply(L, function(x) substitute(x)))
List of 3
 $ a: language X[[i]]
 $ b: language X[[i]]
 $ c: language X[[i]]

The 'X' and 'i' are in a frame constructed by lapply and you are not 
really supposed to depend on the precise form of those expressions.


2: An evaluated promise is still a promise: it has the 'evaled' field 
set to TRUE and the 'value' field set to the result of evaluating 
'code' in 'env'.


> f <- function(x, force) {
 if (force) force(x)
 if (pryr::is_promise(x)) promise_info(x)
 else "not a promise"
 }
> str(f(log(-1), force=FALSE))
List of 4
 $ code  : language log(-1)
 $ env   :
 $ evaled: logi FALSE
 $ value : NULL
> str(f(log(-1), force=TRUE))
List of 4
 $ code  : language log(-1)
 $ env   : NULL
 $ evaled: logi TRUE
 $ value : num NaN
Warning message:
In log(-1) : NaNs produced

Can you give a concrete example of what you are try to accomplish?

Bill Dunlap
TIBCO Software
wdunlap tibco.com <http://tibco.com>

On Fri, Jul 28, 2017 at 3:04 PM, Benjamin Tyner <bty...@gmail.com 
<mailto:bty...@gmail.com>> wrote:


Hi,

I thought I understood the change to lapply semantics resulting
from this,

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16093
<https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16093>

However, would someone care to explain why this does not work?

   > L <- list(a=1, b=2, c=3)
   > str(lapply(L, function(x){ y <- substitute(x); force(x);
eval(y) }))
   Error in eval(y) : object 'X' not found

Basically, my primary goal is to achieve the same result as,

   > str(lapply(L, function(x){ eval.parent(substitute(x)) }))
   List of 3
$ a: num 1
$ b: num 2
$ c: num 3

but without having to resort to eval.parent as that seems to rely
on an implementation detail of lapply.

My secondary goal is to understand why force(x) does not actually
force the promise here,

   > str(lapply(L, function(x){ force(x); pryr::is_promise(x) }))
   List of 3
$ a: logi TRUE
$ b: logi TRUE
$ c: logi TRUE
,
Regards
Ben

__
R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
<https://stat.ethz.ch/mailman/listinfo/r-devel>




__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] force promises inside lapply

[Benjamin Tyner]

Hi,

I thought I understood the change to lapply semantics resulting from this,

   https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16093

However, would someone care to explain why this does not work?

   > L <- list(a=1, b=2, c=3)
   > str(lapply(L, function(x){ y <- substitute(x); force(x); eval(y) }))
   Error in eval(y) : object 'X' not found

Basically, my primary goal is to achieve the same result as,

   > str(lapply(L, function(x){ eval.parent(substitute(x)) }))
   List of 3
$ a: num 1
$ b: num 2
$ c: num 3

but without having to resort to eval.parent as that seems to rely on an 
implementation detail of lapply.


My secondary goal is to understand why force(x) does not actually force 
the promise here,


   > str(lapply(L, function(x){ force(x); pryr::is_promise(x) }))
   List of 3
$ a: logi TRUE
$ b: logi TRUE
$ c: logi TRUE
,
Regards
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] package load altering RNG state

[Benjamin Tyner]

Martin,

Outstanding! This is a most welcome enhancement.

Regards

Ben


On 02/07/2017 09:59 AM, Martin Maechler wrote:

Henric Winell <nilsson.hen...@gmail.com>
 on Tue, 7 Feb 2017 13:37:42 +0100 writes:

 > Hi, On 2017-02-07 13:12, Benjamin Tyner wrote:

 >> Hello
 >>
 >> When loading a package, I'm wondering if it's frowned
 >> upon for the package to alter the state of the random
 >> number generator? I guess not, since the parallel package
 >> does it?

 > Surprisingly it is not frowned upon, but it *is* a
 > nuisance.  I brought it up a couple of years ago

 > 
http://r.789695.n4.nabble.com/parallel-package-changes-Random-seed-td4686321.html

 > along with a patch, but was told off...

 >>
 >> > set.seed(6860) > old.seed <- .GlobalEnv$.Random.seed >
 >> library(parallel) > new.seed <- .GlobalEnv$.Random.seed >
 >> identical(old.seed, new.seed) [1] FALSE
 >>
 >> I ask because, I found myself writing a custom wrapper
 >> around library() to restore the original RNG state, in
 >> order to increase reproducibility.  But now wondering if
 >> others would welcome such a feature to be added to base
 >> R? Either something very general like
 >>
 >> preserveRNGstate(library(parallel))
 >>
 >> or perhaps an specific enhancement to library itself?

 > I would very much welcome a change, but in the light of
 > things it doesn't seem likely.

 > Henric Winell

Sometimes things change ... and not always for the worse.  I've
found a version of your original patch idea which is very
efficient nice (in my eyes) and still leaves
system.time(loadNamespace("parallel"))
to round to 0, i.e. needing less than 1 ms.

--> in R-devel svn rev 72136

Martin


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] package load altering RNG state

[Benjamin Tyner]

Hello

When loading a package, I'm wondering if it's frowned upon for the 
package to alter the state of the random number generator? I guess not, 
since the parallel package does it?


   > set.seed(6860)
   > old.seed <- .GlobalEnv$.Random.seed
   > library(parallel)
   > new.seed <- .GlobalEnv$.Random.seed
   > identical(old.seed, new.seed)
   [1] FALSE

I ask because, I found myself writing a custom wrapper around library() 
to restore the original RNG state, in order to increase reproducibility. 
But now wondering if others would welcome such a feature to be added to 
base R? Either something very general like


   preserveRNGstate(library(parallel))

or perhaps an specific enhancement to library itself?

Regards
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] For integer vectors, `as(x, "numeric")` has no effect.

[Benjamin Tyner]

Perhaps it is not that surprising, given that

> mode(1L)
[1] "numeric"

and

> is.numeric(1L)
[1] TRUE

On the other hand, this is curious, to say the least:

> is.double(as(1L, "double"))
[1] FALSE


Here's the surprising behavior:

x <- 1L
xx <- as(x, "numeric")
class(xx)
## [1] "integer"

It occurs because the call to `as(x, "numeric")` dispatches the coerce
S4 method for the signature `c("integer", "numeric")`, whose body is
copied in below.

 function (from, to = "numeric", strict = TRUE)
 if (strict) {
 class(from) <- "numeric"
 from
 } else from

This in turn does nothing, even when strict=TRUE, because that
assignment to class "numeric" has no effect:

 x <- 10L
 class(x) <- "numeric"
 class(x)
 [1] "integer"

Is this the desired behavior for `as(x, "numeric")`?


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] could not find function anyNA when building tools package in R 3.1.1

[Benjamin Tyner]

Hello,

When building R from source, during the part where the 'tools' package
is built, I get:

make[6]: Entering directory `/home/btyner/R-3.1.1/src/library/tools/src'
make[6]: Leaving directory `/home/btyner/R-3.1.1/src/library/tools/src'
make[5]: Leaving directory `/home/btyner/R-3.1.1/src/library/tools/src'
make[4]: Leaving directory `/home/btyner/R-3.1.1/src/library/tools'
Error in .deparseOpts(control) : could not find function anyNA
Calls: ::: ... tryCatch - tryCatchList - tryCatchOne - Anonymous
Execution halted
make[3]: *** [all] Error 1
make[3]: Leaving directory `/home/btyner/R-3.1.1/src/library/tools'
make[2]: *** [R] Error 1
make[2]: Leaving directory `/home/btyner/R-3.1.1/src/library'
make[1]: *** [R] Error 1
make[1]: Leaving directory `/home/btyner/R-3.1.1/src'
make: *** [R] Error 1

wondering if anyone has seen this before or has any suggestions as to
what the problem could be? Here is some platform information:

uname -m = x86_64
uname -r = 2.6.32-358.18.1.el6.x86_64
uname -s = Linux

Happy to provide more info as needed.

Regards
Ben


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] crossprod(): g77 versus gfortran

[Benjamin Tyner]

Hi

I've got two builds of R, one using g77 (version 3.4.6) and the other
using gfortran (version 4.1.2). The two builds are otherwise identical
as far as I can tell. The one which used g77 performs crossprod()s
roughly twice as fast as the gfortran one. I'm wondering if this rings a
bell with anyone, and if so, are you aware of any configure settings
which will improve the performance when using gfortran. This is on RHEL 5.

Regards
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] built-in NAMED(obj) from within R

[Benjamin Tyner]

Hello,

Is it possible to retrieve the 'named' field within the header (sxpinfo) 
of a object, without resorting to a debugger, external code, etc? In 
other words is there a built-in equivalent to the getnamed subroutine 
described here:


http://tolstoy.newcastle.edu.au/R/e2/devel/07/09/4496.html

The goal is to ascertain whether a copy of an object has been made.

Thanks,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] built-in NAMED(obj) from within R

[Benjamin Tyner]

Thanks Matthew and Hadley; that is exactly what I was looking for, and them 
some.

Regards,
Ben


/ The goal is to ascertain whether a copy of an object has been made.
//
// Then :
//
// ?tracemem
//
// One demonstration of using both together is here :
//
// http://stackoverflow.com/a/10312843/403310
/
Note that tracemem detects duplications (in a technical R sense), not
copies - i.e.

x - 1:10
tracemem(x)
x[11] - 11L

although all the individual elements are copied, the vector is not
duplicated.  This confused me at first, so I thought I'd point it out.

Hadley

--
RStudio / Rice University
http://had.co.nz/


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] suggestion for subset.data.frame()

[Benjamin Tyner]

Hello,

Currently, subset.data.frame() does

   if (missing(subset))
   r - TRUE
   else {

perhaps better might be:

   if (missing(subset))
   r - rep(TRUE, nrow(x))
   else {

The rationale being, if 'x' has zero rows and 'subset' is missing, we 
don't want to end up with more rows than we started with:


 subset(head(iris, 0), select = -Species)
  Sepal.Length Sepal.Width Petal.Length Petal.Width
   NA   NA  NA   NA  NA

Thoughts?
-Ben
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] warning or error upon type/storage mode coercion?

[Benjamin Tyner]

Hi,

I'm aware that the language definition states R objects are often 
coerced to different types during computations. Two questions:


1. Is it possible to configure the R environment so that, for example, 
coercion from (say) numeric to integer will throw a warning or an error? 
I realize that in the base R code alone, there are thousands of calls to 
as.integer() which would trigger such an event, so this would not be a 
very practical configuration...


2. So, assuming the answer to (1) is a resounding no, does anyone care 
to state an opinion regarding the philosophical or historical rationale 
for why this is the case in R/S, whereas certain other interpreted 
languages offer the option to perform strict type checking? Basically, 
I'm trying to explain to someone from a perl background why the 
(apparent) lack of a use strict; use warnings; equivalent is not a 
hindrance to writing bullet-proof R code.


Thanks,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Mksetup() limited to hashing with 32 bits

[Benjamin Tyner]

The MKsetup() in unique.c throws an error if the vector to be hashed is 
longer than (2^32)/8:


   if(n  0 || n  536870912) /* protect against overflow to -ve */
   error(_(length %d is too large for hashing), n);

I occasionally work with vectors longer than this on 64-bit builds. Would it be 
too much to ask that R can take advantage of all 64 bits for hashing when 
compiled as such?

Thanks
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] bug (PR#13570)

[Benjamin Tyner]

Many thanks Brian for tracking this down. Was it fixed by

 c next line is not in current dloess
  goto 7

in ehg136? If this needs to be in the netlib version as well, we should 
inform Eric Grosse.


While we're at it, there are a few more inconsistencies (not nearly as 
serious as PR#13570 so I hesitate to call them bugs) regarding the 
definition of leaf cell membership (certain .lt. should be .le. ) in 
ehg128, ehg137, and ehg138 (not currently used); it seems I neglected to 
mention these to Eric. If you are interested in these I can submit a 
patch and will notify Eric as well.


Finally, perhaps now is as good a time as any to point out that in the 
documentation, the bit about cross-terms in


   \item{drop.square}{for fits with more than one predictor and
   \code{degree=2}, should the quadratic term (and cross-terms) be
   dropped for particular predictors?

is incorrect -- cross terms are not dropped in this implementation of 
loess.


Thanks again,
Ben

Prof Brian Ripley wrote:
I've found the discrepancy, so the patched code from current dloess is 
now available in R-patched and R-devel.


On Fri, 6 Mar 2009, Prof Brian Ripley wrote:


On Thu, 5 Mar 2009, Benjamin Tyner wrote:


Hi

Nice to hear from you Ryan. I also do not have the capability to 
debug on windows; however, there is a chance that the behavior you 
are seeing is caused by the following bug noted in my thesis 
(available on ProQuest; email me if you don't have access):


When lambda = 0 there are no local slopes to aid the blending 
algorithm, yet the
interpolator would still assume they were available, and thus use 
arbitrary values
from memory. This had implications for both fit and tr[L] 
computation. In the
updated code these are set equal to zero which seems the best 
automatic rule when

lambda = 0. [lambda refers to degree]

I submitted a bug fix to Eric Grosse, the maintainer of the netlib 
routines; the fixed lines of fortran are identified in the comments 
at (just search for my email address):


http://www.netlib.org/a/loess

These fixes would be relatively simple to incorporate into R's 
version of loessf.f


The fixes from dloess even more simply, since R's code is based on 
dloess. Thank you for the suggestion.


Given how tricky this is to reproduce, I went back to my example 
under valgrind.  If I use the latest dloess code, it crashes, but by 
selectively importing some of the differences I can get it to work.


So it looks as if we are on the road to a solution, but something in 
the current version (not necessarily in these changes) is 
incompatible with the current R code and I need to dig further (not 
for a few days).


Alternatively, a quick check would be for someone to compile the 
source package at 
https://centauri.stat.purdue.edu:98/loess/loess_0.4-1.tar.gz and 
test it on windows. Though this package incorporates this and a few 
other fixes, please be aware that it the routines are converted to C 
and thus there is a slight performance hit compared to the fortran.


Hope this helps,
Ben


[...]

--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595





__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] bug (PR#13570)

[Benjamin Tyner]

Hi

Nice to hear from you Ryan. I also do not have the capability to debug 
on windows; however, there is a chance that the behavior you are seeing 
is caused by the following bug noted in my thesis (available on 
ProQuest; email me if you don't have access):


When lambda = 0 there are no local slopes to aid the blending 
algorithm, yet the
interpolator would still assume they were available, and thus use 
arbitrary values
from memory. This had implications for both fit and tr[L] computation. 
In the
updated code these are set equal to zero which seems the best automatic 
rule when

lambda = 0. [lambda refers to degree]

I submitted a bug fix to Eric Grosse, the maintainer of the netlib 
routines; the fixed lines of fortran are identified in the comments at 
(just search for my email address):


http://www.netlib.org/a/loess

These fixes would be relatively simple to incorporate into R's version 
of loessf.f


Alternatively, a quick check would be for someone to compile the source 
package at https://centauri.stat.purdue.edu:98/loess/loess_0.4-1.tar.gz 
and test it on windows. Though this package incorporates this and a few 
other fixes, please be aware that it the routines are converted to C and 
thus there is a slight performance hit compared to the fortran.


Hope this helps,
Ben

Ryan Hafen wrote:

That is true - good point.

lp1 - predict(loess(y ~ x, degree=0))
lp2 - predict(loess(y ~ x, degree=0, 
control=loess.control(surface=direct)))

sort(abs(lp1-lp2))

It appears that the interpolating fit is correct at the vertices.  I 
know when degree=1, the interpolation uses the slopes of the local 
fits to get a better approximation.  Perhaps it's still trying to do 
this with degree=0 but the slopes aren't available.  And we have just 
been lucky in the past with uninitialized values?  If this is the 
problem it would probably be very simple to fix and I'd love to see 
degree=0 stay.  I will see if I can figure it out.



On Mar 5, 2009, at 6:01 PM, Greg Snow wrote:


I see the same problem on Windows XP.

But if I run loess with surface='direct' then the results are 
correct.  So it looks like the problem comes from the 
smoothing/interpolating, not the main loess algorithm.


--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.s...@imail.org
801.408.8111



-Original Message-
From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r-
project.org] On Behalf Of Ryan Hafen
Sent: Thursday, March 05, 2009 7:43 AM
To: Prof Brian Ripley
Cc: Uwe Ligges; Berwin A Turlach; r-de...@stat.math.ethz.ch; Peter
Dalgaard
Subject: Re: [Rd] bug (PR#13570)


On Mar 5, 2009, at 7:59 AM, Prof Brian Ripley wrote:


On Thu, 5 Mar 2009, Peter Dalgaard wrote:


Prof Brian Ripley wrote:

Undortunately the example is random, so not really reproducible
(and I
see nothing wrong on my Mac). However, Linux valgrind on R-devel is
showing a problem:

==3973== Conditional jump or move depends on uninitialised value(s)
==3973==at 0xD76017B: ehg141_ (loessf.f:532)
==3973==by 0xD761600: lowesa_ (loessf.f:769)
==3973==by 0xD736E47: loess_raw (loessc.c:117)

(The uninitiialized value is in someone else's code and I suspect
it was
either never intended to work or never tested.)  No essential
change has
been made to the loess code for many years.

I would not have read the documentation to say that degree = 0 was

a

reasonable value. It is not to my mind 'a polynomial surface', and
loess() is described as a 'local regression' for degree 1 or 2 in
the
reference.  So unless anyone wants to bury their heads in that
code I
think a perfectly adequate fix would be to disallow degree = 0.
(I vaguely recall debating allowing in the code ca 10 years ago.)


The code itself has

 if (!match(degree, 0:2, 0))
 stop('degree' must be 0, 1 or 2)

though. Local fitting of a constant essentially becomes kernel
smoothing, right?


I do know the R code allows it: the question is whether it is worth
the effort of finding the problem(s) in the underlying c/dloess
code, whose manual (and our reference) is entirely about 1 or 2.  I
am concerned that there may be other things lurking in the degree=0
case if it was never tested (in the netlib version: I am sure it was
only minmally tested through my R interface).

I checked the original documentation on netlib and that says

29  DIM dimension of local regression
  1   constant
  d+1 linear   (default)
  (d+2)(d+1)/2quadratic
  Modified by ehg127 if cdegtdeg.

which seems to confirm that degree = 0 was intended to be allowed,
and what I dimly recall from ca 1998 is debating whether the R code
should allow that or not.

If left to me I would say I did not wish to continue to support
degree = 0.


True.  There are plenty of reasons why one wouldn't want to use
degree=0 anyway.  And I'm sure there are plenty of other simple ways
to achieve the same effect.

I ran into the 

[Rd] data frame with NA in names

[Benjamin Tyner]

When calling read.table() and supplying colClasses and header=FALSE,
it is possible for the resulting data frame to have NA in the names.
(The warning not all columns named in 'colClasses' exist is
returned)

Here is such a data frame:

x - structure(list(a = 1:10,
NA = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA)),
.Names = c(a, NA_character_),
row.names = c(NA, -10L),
class = data.frame)

One undesirable property of such a data frame is that subset() has
trouble with it:

subset(x, a  5)

Error in `[.data.frame`(x, r, vars, drop = drop) :
  undefined columns selected


Is this a bug in read.table() ?

Thanks
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] suggestion for read.table

[Benjamin Tyner]

R-devel,

IMHO, in readtable.R, the scan command


if (nlines  1)
for (i in seq_along(col))
col[i] - length(scan(file, what = , sep = sep,
  quote = quote,
  nlines = 1, quiet = TRUE, skip = 0,
  strip.white = strip.white,
  blank.lines.skip = blank.lines.skip,
  comment.char = comment.char,
  allowEscapes = allowEscapes))
cols - max(col1, col)


should have fill = fill added. Otherwise, when fill = TRUE, it is
possible for the warning

warning(header and 'col.names' are of different lengths)

to be inadvertantly triggered further down.

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] suggestion for read.table

[Benjamin Tyner]

Correction: the warning would be triggered by

warning(cols = , cols,  != length(data) = , length(data), domain = NA)

Ben

On 9/20/07, Benjamin Tyner [EMAIL PROTECTED] wrote:
 R-devel,

 IMHO, in readtable.R, the scan command


 if (nlines  1)
 for (i in seq_along(col))
 col[i] - length(scan(file, what = , sep = sep,
   quote = quote,
   nlines = 1, quiet = TRUE, skip = 0,
   strip.white = strip.white,
   blank.lines.skip = blank.lines.skip,
   comment.char = comment.char,
   allowEscapes = allowEscapes))
 cols - max(col1, col)


 should have fill = fill added. Otherwise, when fill = TRUE, it is
 possible for the warning

 warning(header and 'col.names' are of different lengths)

 to be inadvertantly triggered further down.

 Ben


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] hasNA() / anyNA()?

[Benjamin Tyner]

Why not

hasNA - function(x) !is.na(match(NA, x))

-Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] suggestion with ambiguous object component replacement

[Benjamin Tyner]

?[- says

When replacing (that is using indexing on the lhs of an assignment)
NA does not select any element to be replaced. As there is ambiguity
as to whether an element of the rhs should be used or not, this is
only allowed if the rhs value is of length one (so the two
interpretations would have the same outcome).

This is quite reasonable to me; for example,

letters[NA] - z

does not throw an error but

letters[NA] - c(z,z)

does. However I wonder why assignments like

letters[c(2,2)] - c(b1,b2)

are allowed, since at least to me it is not clear at all that b2 is
the intended replacement. I would think that at the very least this
should throw a warning. I apologize if this has already been
discussed.

Thanks
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] possible bug in 'scan'

[Benjamin Tyner]

R-devel,

When I run the following code on the attached file,

tmp - scan(C:/temp.csv,
   what=list(character,numeric),
   sep=,)

Then tmp[[2]] is a character vector. My impression from the help file
is that it should be a numeric as specified by 'what'


sessionInfo()

R version 2.5.0 (2007-04-23)
i386-pc-mingw32

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics  grDevices utils datasets
methods   base

other attached packages:
latticegdata
0.15-5  2.3.1

My apologies if this is due to my misunderstanding.
Ben
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] possible bug in 'scan'

[Benjamin Tyner]

How embarrassing -- works like a charm.

On 6/26/07, Greg Snow [EMAIL PROTECTED] wrote:
 The what argument looks at what the elements are, not the word they say.
 Try this:

  tmp - scan(C:/temp.csv,
  what=list(,0),
  sep=,)

 Hope this helps,

 --
 Gregory (Greg) L. Snow Ph.D.
 Statistical Data Center
 Intermountain Healthcare
 [EMAIL PROTECTED]
 (801) 408-8111



  -Original Message-
  From: [EMAIL PROTECTED]
  [mailto:[EMAIL PROTECTED] On Behalf Of Benjamin Tyner
  Sent: Tuesday, June 26, 2007 9:59 AM
  To: r-devel@r-project.org
  Subject: [Rd] possible bug in 'scan'
 
  R-devel,
 
  When I run the following code on the attached file,
 
  tmp - scan(C:/temp.csv,
  what=list(character,numeric),
  sep=,)
 
  Then tmp[[2]] is a character vector. My impression from the
  help file is that it should be a numeric as specified by 'what'
 
   sessionInfo()
  R version 2.5.0 (2007-04-23)
  i386-pc-mingw32
 
  locale:
  LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
  States.1252;LC_MONETARY=English_United
  States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
 
  attached base packages:
  [1] stats graphics  grDevices utils datasets
  methods   base
 
  other attached packages:
   latticegdata
  0.15-5  2.3.1
 
  My apologies if this is due to my misunderstanding.
  Ben
 



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] possible bug in co.intervals when overlap=0

[Benjamin Tyner]

I apologize if this is just a misunderstanding on my part, but I was 
under the impression that the intervals returned by co.intervals should 
cover all the observations. Yet

x-1:10
z-co.intervals(x,overlap=0)

In R, z equals
  [,1] [,2]
[1,]  0.5  1.5
[2,]  2.5  3.5
[3,]  3.5  4.5
[4,]  5.5  6.5
[5,]  7.5  8.5
[6,]  8.5  9.5

Note in particular the second and last element of x are not covered by 
any of the intervals. In fact S-PLUS gives
 [,1] [,2]
[1,]12
[2,]33
[3,]45
[4,]67
[5,]88
[6,]9   10


Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] array indexes in C

[Benjamin Tyner]

Hin-Tak  Tamas,

For example, see what I've done in 
http://www.stat.purdue.edu/~btyner/postage_0.1-1.tar.gz

I am by no means a C guru but it works with my compiler. The relevant 
lines are

(in postage.c:)
void postage(int *lambda, int *D, int *tau, int r[*tau][*D])
{

(in postage.R:)
  r - .C(postage,
  as.integer(lambda),
  as.integer(D),
  as.integer(tau0),
  r = integer(tau0 * D),
  PACKAGE=postage)$r

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] array indexes in C

[Benjamin Tyner]

Tamas,

You could write convenience functions, but I have used the C99 mechanism 
for variable length arrays with no problems calling from R. One thing 
you have to keep in mind though is that (as far as I know) the 
dimensions must be passed before the array reference. So for example,

r - .C(foo,
  as.integer(ni),
  as.integer(nj),
  x = double(ni * nj),
  ...)

with your function defined as

void foo(int *ni, int *nj, double x[*ni][*nj])
{
...

Then in C you can access elements of x via x[3][4], for example.

Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] unloadNamespace inside .Last.lib

[Benjamin Tyner]

In my package's zzz.R, I put

.Last.lib - function(libpath) {
unloadNamespace(mypackage)
}

and I exported .Last.lib in NAMESPACE, with the intent that detaching 
the package would also cause the name space to be unloaded. However, the 
result of detach(package:mypackage) is then

Error: evaluation nested too deeply: infinite recursion / 
options(expressions=)?environment: namespace:mypackage
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
environment: namespace:mypackage
Error in library.dynam.unload(mypackage, libpath) :
shared library 'mypackage' was not loaded
Error in as.character(environment) : cannot coerce to vector
Error in unregisterNamespace(nsname) : name space not registered
Error in detach(pos) : detaching package:base is not allowed
Error in detach(pos) : detaching package:base is not allowed
Error in detach(pos) : detaching package:base is not allowed


With a hundred more or so of the last error. What is the correct way (if 
any) to unload a name space using detach() alone ?

Thanks,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] prefix 'R_' when calling external code

[Benjamin Tyner]

I noticed that in

https://svn.r-project.org/R/trunk/src/library/stats/R/loess.R

that we are now calling R_loess_raw, R_lowesw, R_lowesp, etc. I'm 
interested to know what is the benefit/reason for the 'R_', as I am 
unfamiliar with this prefix and do not see it mentioned in 'Writing R 
Extensions.'

Thanks,
Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] custom strip in lattice ignoring plotmath expressions for all but style = 1 (PR#8733)

[Benjamin Tyner]

Sorry, so used to it being bundled that I didn't realize lattice was a 
contributed package.

Ben

Deepayan Sarkar wrote:



1. Bugs in packages should be reported to the maintainer, not R-bugs.

2. This already works as it should in R 2.3.0 alpha

-Deepayan
  


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel