from:"Kenny Bell"

[Rd] Possible bug in predict.lm when x is a poly

2021-03-29 Thread Kenny Bell

Hi all,

As always, thank you all for your incredible work maintaining and improving
R.

mdl <- lm(data = mtcars,
  mpg ~ I(poly(disp, 2)))

predict(mdl, newdata = data.frame(disp = c(120, 120)))
#> Error in poly(disp, 2): 'degree' must be less than number of unique
points

predict(mdl, newdata = data.frame(disp = c(120, 121, 122)))
#> 1 2 3
#> 43.937856 12.617762  3.716257

The predict function seems to require a sufficiently high number of unique
values in newdata when the RHS is a poly. Of course, I would have expected
the output here to be:

#> 1 2
#> 43.937856 43.937856

If people agree, I can submit this to bugzilla.

Kind regards,
Kenny

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

2020-05-25 Thread Kenny Bell

I also tried just upgrading to 20.04 and that seemed to fix it.

On Tue, May 26, 2020 at 5:38 PM Kenny Bell  wrote:

> I can also confirm that the latest  https://hub.docker.com/_/r-base has
> the correct behaviour.
>
> On Tue, May 26, 2020 at 5:05 PM Kenny Bell  wrote:
>
>> I was actually able to reproduce this on a relatively fresh install of
>> 18.04 (a virtualbox). Paul, did you run apt update && apt upgrade before
>> trying to reproduce?
>>
>> On Tue, May 26, 2020 at 4:36 PM Kenny Bell  wrote:
>>
>>> Hi Paul,
>>>
>>> I tried downgrading to R 3.4.4 and I still see the problem. I also have
>>> a conda environment that doesn't exhibit the problem so it's something
>>> environmental.
>>>
>>> Any tips for possible solutions/troubleshooting would be appreciated!
>>>
>>> Cheers,
>>> Kenny
>>>
>>> On Tue, May 26, 2020 at 3:09 PM Paul Murrell 
>>> wrote:
>>>
>>>>
>>>> I am not seeing that problem on my 18.04 ...
>>>>
>>>>  > sessionInfo()
>>>> R version 4.0.0 Patched (2020-05-12 r78431)
>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>> Running under: Ubuntu 18.04.4 LTS
>>>>
>>>> Matrix products: default
>>>> BLAS:   /home/pmur002/R/R-4-0-branch/BUILD/lib/libRblas.so
>>>> LAPACK: /home/pmur002/R/R-4-0-branch/BUILD/lib/libRlapack.so
>>>>
>>>> locale:
>>>>   [1] LC_CTYPE=en_NZ.UTF-8   LC_NUMERIC=C
>>>>   [3] LC_TIME=en_NZ.UTF-8LC_COLLATE=en_NZ.UTF-8
>>>>   [5] LC_MONETARY=en_NZ.UTF-8LC_MESSAGES=en_NZ.UTF-8
>>>>   [7] LC_PAPER=en_NZ.UTF-8   LC_NAME=C
>>>>   [9] LC_ADDRESS=C   LC_TELEPHONE=C
>>>> [11] LC_MEASUREMENT=en_NZ.UTF-8 LC_IDENTIFICATION=C
>>>>
>>>> attached base packages:
>>>> [1] stats graphics  grDevices utils datasets  methods   base
>>>>
>>>> loaded via a namespace (and not attached):
>>>> [1] compiler_4.0.0
>>>>
>>>> Paul
>>>>
>>>> On 26/05/20 2:21 pm, Kenny Bell wrote:
>>>> > Hi all,
>>>> >
>>>> > I have found after upgrading to R 4.0.0 (among other upgrades so this
>>>> may
>>>> > not be the cause) that the degree symbol doesn't work correctly on
>>>> Ubuntu
>>>> > 18.04. Googling brought me to this thread that appears related.
>>>> >
>>>> > I tried running:
>>>> > cairo_pdf()
>>>> > plot.new(); text(0.5,0.5, bquote(120*degree*N), cex=5)
>>>> > dev.off()
>>>> >
>>>> > and the ubuntu plot has the degree symbol vertically in the center of
>>>> the
>>>> > line. The Windows one correctly shows as superscript.
>>>> >
>>>> > Anyone else see this behaviour?
>>>> >
>>>> > Cheers,
>>>> > Kenny
>>>> >
>>>> > On Fri, Apr 10, 2020 at 3:36 AM Nicolas Mailhot via R-devel <
>>>> > r-devel@r-project.org> wrote:
>>>> >
>>>> >> Le mercredi 08 avril 2020 à 02:55 -0700, Gabriel Becker a écrit :
>>>> >>> Hi Paul,
>>>> >>
>>>> >> Hi Gabriel,
>>>> >>
>>>> >> Thanks a lot for the testing.
>>>> >>
>>>> >>> The various font family settings seem to work too, from what I can
>>>> >>> tell. Both font families you suggested, however, Helvetica and Apple
>>>> >>> Symbols (the s is important) have significantly incomplete coverage
>>>> >>> with PUA on.
>>>> >>
>>>> >> That is to be expected, the AMS symbol dump in PUA space was a quick
>>>> >> hack to make pre-unicode symbols available in an unicode world,
>>>> pending
>>>> >> their normalisation.
>>>> >>
>>>> >> That standardisation is long past (IIRC it occured by unicode 3.2
>>>> >> released in March 2002), so no newly created/updated font family is
>>>> >> going to place those symbols in PUA anymore.
>>>> >>
>>>> >> Now adding the AMS symbols to new fonts has been slow, due to the
>>>> large
>>>> >> amount of software hardcoding Symbol (and equivallent) and masking
>>>> the
>>>> >> actual glyph userbase to font makers. It will accelerate with more
>>>> apps
>>>> >> expecting plain unicode by default.
>>>> >>
>>>> >> Thanks for the testing!
>>>> >>
>>>> >> Regards,
>>>> >>
>>>> >> --
>>>> >> Nicolas Mailhot
>>>> >>
>>>> >> __
>>>> >> R-devel@r-project.org mailing list
>>>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>> >>
>>>> >>
>>>> >> __
>>>> >> R-devel@r-project.org mailing list
>>>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>> --
>>>> Dr Paul Murrell
>>>> Department of Statistics
>>>> The University of Auckland
>>>> Private Bag 92019
>>>> Auckland
>>>> New Zealand
>>>> 64 9 3737599 x85392
>>>> p...@stat.auckland.ac.nz
>>>> http://www.stat.auckland.ac.nz/~paul/
>>>>
>>>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

2020-05-25 Thread Kenny Bell

I can also confirm that the latest  https://hub.docker.com/_/r-base has the
correct behaviour.

On Tue, May 26, 2020 at 5:05 PM Kenny Bell  wrote:

> I was actually able to reproduce this on a relatively fresh install of
> 18.04 (a virtualbox). Paul, did you run apt update && apt upgrade before
> trying to reproduce?
>
> On Tue, May 26, 2020 at 4:36 PM Kenny Bell  wrote:
>
>> Hi Paul,
>>
>> I tried downgrading to R 3.4.4 and I still see the problem. I also have a
>> conda environment that doesn't exhibit the problem so it's something
>> environmental.
>>
>> Any tips for possible solutions/troubleshooting would be appreciated!
>>
>> Cheers,
>> Kenny
>>
>> On Tue, May 26, 2020 at 3:09 PM Paul Murrell 
>> wrote:
>>
>>>
>>> I am not seeing that problem on my 18.04 ...
>>>
>>>  > sessionInfo()
>>> R version 4.0.0 Patched (2020-05-12 r78431)
>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>> Running under: Ubuntu 18.04.4 LTS
>>>
>>> Matrix products: default
>>> BLAS:   /home/pmur002/R/R-4-0-branch/BUILD/lib/libRblas.so
>>> LAPACK: /home/pmur002/R/R-4-0-branch/BUILD/lib/libRlapack.so
>>>
>>> locale:
>>>   [1] LC_CTYPE=en_NZ.UTF-8   LC_NUMERIC=C
>>>   [3] LC_TIME=en_NZ.UTF-8LC_COLLATE=en_NZ.UTF-8
>>>   [5] LC_MONETARY=en_NZ.UTF-8LC_MESSAGES=en_NZ.UTF-8
>>>   [7] LC_PAPER=en_NZ.UTF-8   LC_NAME=C
>>>   [9] LC_ADDRESS=C   LC_TELEPHONE=C
>>> [11] LC_MEASUREMENT=en_NZ.UTF-8 LC_IDENTIFICATION=C
>>>
>>> attached base packages:
>>> [1] stats graphics  grDevices utils datasets  methods   base
>>>
>>> loaded via a namespace (and not attached):
>>> [1] compiler_4.0.0
>>>
>>> Paul
>>>
>>> On 26/05/20 2:21 pm, Kenny Bell wrote:
>>> > Hi all,
>>> >
>>> > I have found after upgrading to R 4.0.0 (among other upgrades so this
>>> may
>>> > not be the cause) that the degree symbol doesn't work correctly on
>>> Ubuntu
>>> > 18.04. Googling brought me to this thread that appears related.
>>> >
>>> > I tried running:
>>> > cairo_pdf()
>>> > plot.new(); text(0.5,0.5, bquote(120*degree*N), cex=5)
>>> > dev.off()
>>> >
>>> > and the ubuntu plot has the degree symbol vertically in the center of
>>> the
>>> > line. The Windows one correctly shows as superscript.
>>> >
>>> > Anyone else see this behaviour?
>>> >
>>> > Cheers,
>>> > Kenny
>>> >
>>> > On Fri, Apr 10, 2020 at 3:36 AM Nicolas Mailhot via R-devel <
>>> > r-devel@r-project.org> wrote:
>>> >
>>> >> Le mercredi 08 avril 2020 à 02:55 -0700, Gabriel Becker a écrit :
>>> >>> Hi Paul,
>>> >>
>>> >> Hi Gabriel,
>>> >>
>>> >> Thanks a lot for the testing.
>>> >>
>>> >>> The various font family settings seem to work too, from what I can
>>> >>> tell. Both font families you suggested, however, Helvetica and Apple
>>> >>> Symbols (the s is important) have significantly incomplete coverage
>>> >>> with PUA on.
>>> >>
>>> >> That is to be expected, the AMS symbol dump in PUA space was a quick
>>> >> hack to make pre-unicode symbols available in an unicode world,
>>> pending
>>> >> their normalisation.
>>> >>
>>> >> That standardisation is long past (IIRC it occured by unicode 3.2
>>> >> released in March 2002), so no newly created/updated font family is
>>> >> going to place those symbols in PUA anymore.
>>> >>
>>> >> Now adding the AMS symbols to new fonts has been slow, due to the
>>> large
>>> >> amount of software hardcoding Symbol (and equivallent) and masking the
>>> >> actual glyph userbase to font makers. It will accelerate with more
>>> apps
>>> >> expecting plain unicode by default.
>>> >>
>>> >> Thanks for the testing!
>>> >>
>>> >> Regards,
>>> >>
>>> >> --
>>> >> Nicolas Mailhot
>>> >>
>>> >> __
>>> >> R-devel@r-project.org mailing list
>>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>>> >>
>>> >>
>>> >> __
>>> >> R-devel@r-project.org mailing list
>>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>> --
>>> Dr Paul Murrell
>>> Department of Statistics
>>> The University of Auckland
>>> Private Bag 92019
>>> Auckland
>>> New Zealand
>>> 64 9 3737599 x85392
>>> p...@stat.auckland.ac.nz
>>> http://www.stat.auckland.ac.nz/~paul/
>>>
>>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

2020-05-25 Thread Kenny Bell

I was actually able to reproduce this on a relatively fresh install of
18.04 (a virtualbox). Paul, did you run apt update && apt upgrade before
trying to reproduce?

On Tue, May 26, 2020 at 4:36 PM Kenny Bell  wrote:

> Hi Paul,
>
> I tried downgrading to R 3.4.4 and I still see the problem. I also have a
> conda environment that doesn't exhibit the problem so it's something
> environmental.
>
> Any tips for possible solutions/troubleshooting would be appreciated!
>
> Cheers,
> Kenny
>
> On Tue, May 26, 2020 at 3:09 PM Paul Murrell 
> wrote:
>
>>
>> I am not seeing that problem on my 18.04 ...
>>
>>  > sessionInfo()
>> R version 4.0.0 Patched (2020-05-12 r78431)
>> Platform: x86_64-pc-linux-gnu (64-bit)
>> Running under: Ubuntu 18.04.4 LTS
>>
>> Matrix products: default
>> BLAS:   /home/pmur002/R/R-4-0-branch/BUILD/lib/libRblas.so
>> LAPACK: /home/pmur002/R/R-4-0-branch/BUILD/lib/libRlapack.so
>>
>> locale:
>>   [1] LC_CTYPE=en_NZ.UTF-8   LC_NUMERIC=C
>>   [3] LC_TIME=en_NZ.UTF-8LC_COLLATE=en_NZ.UTF-8
>>   [5] LC_MONETARY=en_NZ.UTF-8LC_MESSAGES=en_NZ.UTF-8
>>   [7] LC_PAPER=en_NZ.UTF-8   LC_NAME=C
>>   [9] LC_ADDRESS=C   LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_NZ.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] stats graphics  grDevices utils datasets  methods   base
>>
>> loaded via a namespace (and not attached):
>> [1] compiler_4.0.0
>>
>> Paul
>>
>> On 26/05/20 2:21 pm, Kenny Bell wrote:
>> > Hi all,
>> >
>> > I have found after upgrading to R 4.0.0 (among other upgrades so this
>> may
>> > not be the cause) that the degree symbol doesn't work correctly on
>> Ubuntu
>> > 18.04. Googling brought me to this thread that appears related.
>> >
>> > I tried running:
>> > cairo_pdf()
>> > plot.new(); text(0.5,0.5, bquote(120*degree*N), cex=5)
>> > dev.off()
>> >
>> > and the ubuntu plot has the degree symbol vertically in the center of
>> the
>> > line. The Windows one correctly shows as superscript.
>> >
>> > Anyone else see this behaviour?
>> >
>> > Cheers,
>> > Kenny
>> >
>> > On Fri, Apr 10, 2020 at 3:36 AM Nicolas Mailhot via R-devel <
>> > r-devel@r-project.org> wrote:
>> >
>> >> Le mercredi 08 avril 2020 à 02:55 -0700, Gabriel Becker a écrit :
>> >>> Hi Paul,
>> >>
>> >> Hi Gabriel,
>> >>
>> >> Thanks a lot for the testing.
>> >>
>> >>> The various font family settings seem to work too, from what I can
>> >>> tell. Both font families you suggested, however, Helvetica and Apple
>> >>> Symbols (the s is important) have significantly incomplete coverage
>> >>> with PUA on.
>> >>
>> >> That is to be expected, the AMS symbol dump in PUA space was a quick
>> >> hack to make pre-unicode symbols available in an unicode world, pending
>> >> their normalisation.
>> >>
>> >> That standardisation is long past (IIRC it occured by unicode 3.2
>> >> released in March 2002), so no newly created/updated font family is
>> >> going to place those symbols in PUA anymore.
>> >>
>> >> Now adding the AMS symbols to new fonts has been slow, due to the large
>> >> amount of software hardcoding Symbol (and equivallent) and masking the
>> >> actual glyph userbase to font makers. It will accelerate with more apps
>> >> expecting plain unicode by default.
>> >>
>> >> Thanks for the testing!
>> >>
>> >> Regards,
>> >>
>> >> --
>> >> Nicolas Mailhot
>> >>
>> >> __
>> >> R-devel@r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>> >>
>> >>
>> >> __
>> >> R-devel@r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> --
>> Dr Paul Murrell
>> Department of Statistics
>> The University of Auckland
>> Private Bag 92019
>> Auckland
>> New Zealand
>> 64 9 3737599 x85392
>> p...@stat.auckland.ac.nz
>> http://www.stat.auckland.ac.nz/~paul/
>>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

2020-05-25 Thread Kenny Bell

Hi Paul,

I tried downgrading to R 3.4.4 and I still see the problem. I also have a
conda environment that doesn't exhibit the problem so it's something
environmental.

Any tips for possible solutions/troubleshooting would be appreciated!

Cheers,
Kenny

On Tue, May 26, 2020 at 3:09 PM Paul Murrell 
wrote:

>
> I am not seeing that problem on my 18.04 ...
>
>  > sessionInfo()
> R version 4.0.0 Patched (2020-05-12 r78431)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 18.04.4 LTS
>
> Matrix products: default
> BLAS:   /home/pmur002/R/R-4-0-branch/BUILD/lib/libRblas.so
> LAPACK: /home/pmur002/R/R-4-0-branch/BUILD/lib/libRlapack.so
>
> locale:
>   [1] LC_CTYPE=en_NZ.UTF-8   LC_NUMERIC=C
>   [3] LC_TIME=en_NZ.UTF-8LC_COLLATE=en_NZ.UTF-8
>   [5] LC_MONETARY=en_NZ.UTF-8LC_MESSAGES=en_NZ.UTF-8
>   [7] LC_PAPER=en_NZ.UTF-8   LC_NAME=C
>   [9] LC_ADDRESS=C   LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_NZ.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_4.0.0
>
> Paul
>
> On 26/05/20 2:21 pm, Kenny Bell wrote:
> > Hi all,
> >
> > I have found after upgrading to R 4.0.0 (among other upgrades so this may
> > not be the cause) that the degree symbol doesn't work correctly on Ubuntu
> > 18.04. Googling brought me to this thread that appears related.
> >
> > I tried running:
> > cairo_pdf()
> > plot.new(); text(0.5,0.5, bquote(120*degree*N), cex=5)
> > dev.off()
> >
> > and the ubuntu plot has the degree symbol vertically in the center of the
> > line. The Windows one correctly shows as superscript.
> >
> > Anyone else see this behaviour?
> >
> > Cheers,
> > Kenny
> >
> > On Fri, Apr 10, 2020 at 3:36 AM Nicolas Mailhot via R-devel <
> > r-devel@r-project.org> wrote:
> >
> >> Le mercredi 08 avril 2020 à 02:55 -0700, Gabriel Becker a écrit :
> >>> Hi Paul,
> >>
> >> Hi Gabriel,
> >>
> >> Thanks a lot for the testing.
> >>
> >>> The various font family settings seem to work too, from what I can
> >>> tell. Both font families you suggested, however, Helvetica and Apple
> >>> Symbols (the s is important) have significantly incomplete coverage
> >>> with PUA on.
> >>
> >> That is to be expected, the AMS symbol dump in PUA space was a quick
> >> hack to make pre-unicode symbols available in an unicode world, pending
> >> their normalisation.
> >>
> >> That standardisation is long past (IIRC it occured by unicode 3.2
> >> released in March 2002), so no newly created/updated font family is
> >> going to place those symbols in PUA anymore.
> >>
> >> Now adding the AMS symbols to new fonts has been slow, due to the large
> >> amount of software hardcoding Symbol (and equivallent) and masking the
> >> actual glyph userbase to font makers. It will accelerate with more apps
> >> expecting plain unicode by default.
> >>
> >> Thanks for the testing!
> >>
> >> Regards,
> >>
> >> --
> >> Nicolas Mailhot
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> p...@stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

2020-05-25 Thread Kenny Bell

Hi all,

I have found after upgrading to R 4.0.0 (among other upgrades so this may
not be the cause) that the degree symbol doesn't work correctly on Ubuntu
18.04. Googling brought me to this thread that appears related.

I tried running:
cairo_pdf()
plot.new(); text(0.5,0.5, bquote(120*degree*N), cex=5)
dev.off()

and the ubuntu plot has the degree symbol vertically in the center of the
line. The Windows one correctly shows as superscript.

Anyone else see this behaviour?

Cheers,
Kenny

On Fri, Apr 10, 2020 at 3:36 AM Nicolas Mailhot via R-devel <
r-devel@r-project.org> wrote:

> Le mercredi 08 avril 2020 à 02:55 -0700, Gabriel Becker a écrit :
> > Hi Paul,
>
> Hi Gabriel,
>
> Thanks a lot for the testing.
>
> > The various font family settings seem to work too, from what I can
> > tell. Both font families you suggested, however, Helvetica and Apple
> > Symbols (the s is important) have significantly incomplete coverage
> > with PUA on.
>
> That is to be expected, the AMS symbol dump in PUA space was a quick
> hack to make pre-unicode symbols available in an unicode world, pending
> their normalisation.
>
> That standardisation is long past (IIRC it occured by unicode 3.2
> released in March 2002), so no newly created/updated font family is
> going to place those symbols in PUA anymore.
>
> Now adding the AMS symbols to new fonts has been slow, due to the large
> amount of software hardcoding Symbol (and equivallent) and masking the
> actual glyph userbase to font makers. It will accelerate with more apps
> expecting plain unicode by default.
>
> Thanks for the testing!
>
> Regards,
>
> --
> Nicolas Mailhot
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


Rplot001_ubuntu.pdf
Description: Adobe PDF document


Rplot001_windows.pdf
Description: Adobe PDF document
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Model object, when generated in a function, saves entire environment when saved

2020-01-29 Thread Kenny Bell

Reviving an old thread. I haven't noticed this be a problem for a while
when saving RDS's which is great. However, I noticed the problem again when
saving `qs` files (https://github.com/traversc/qs) which is an RDS
replacement with a fast serialization / compression system.

I'd like to get an idea of what change was made within R to address this
issue for `saveRDS`. My thought is that this will help the author of the
`qs` package do something similar. I have had a browse through the release
notes for the last few years (Ctrl-F-ing "environment") and couldn't see it.

Many thanks for any help and best wishes to all.

The following code uses R 3.6.2 and requires you to run
install.packages("qs") first:

save_size_qs <- function (object) {
  tf <- tempfile(fileext = ".qs")
  on.exit(unlink(tf))
  qs::qsave(object, file = tf)
  file.size(tf)
}

save_size_rds <- function (object) {
  tf <- tempfile(fileext = ".rds")
  on.exit(unlink(tf))
  saveRDS(object, file = tf)
  file.size(tf)
}

normal_lm <- function(){
  junk <- 1:1e+08
  lm(Sepal.Length ~ Sepal.Width, data = iris)
}

normal_ggplot <- function(){
  junk <- 1:1e+08
  ggplot2::ggplot()
}

clean_lm <- function () {
  junk <- 1:1e+08
  # Run the lm in its own environment
  env <- new.env(parent = globalenv())
  env$subset <- subset
  with(env, lm(Sepal.Length ~ Sepal.Width, data = iris))
}

# The qs save size includes the junk but the rds does not
save_size_qs(normal_lm())
#> [1] 848396
save_size_rds(normal_lm())
#> [1] 4163
save_size_qs(normal_ggplot())
#> [1] 857446
save_size_rds(normal_ggplot())
#> [1] 12895


# Both exclude the junk when separating the lm into its own environment
save_size_qs(clean_lm())
#> [1] 6154
save_size_rds(clean_lm())
#> [1] 4255


On Thu, Jul 28, 2016 at 7:31 AM Kenny Bell  wrote:

> Thanks so much for all this.
>
> The first solution is what I'm going with as I want the terms object to
> come along so that predict still works.
>
> On Wed, Jul 27, 2016 at 12:28 PM, William Dunlap via R-devel <
> r-devel@r-project.org> wrote:
>
>> Another solution is to only save the parts of the model object that
>> interest you.  As long as they don't include the formula (which is
>> what drags along the environment it was created in), you will
>> save space.  E.g.,
>>
>> tfun2 <- function(subset) {
>>junk <- 1:1e6
>>list(subset=subset, lm(Sepal.Length ~ Sepal.Width, data=iris,
>> subset=subset)$coef)
>> }
>>
>> saveSize(tfun2(1:4))
>> #[1] 152
>>
>>
>>
>> Bill Dunlap
>> TIBCO Software
>> wdunlap tibco.com
>>
>> On Wed, Jul 27, 2016 at 11:19 AM, William Dunlap 
>> wrote:
>>
>> > One way around this problem is to make a new environment whose
>> > parent environment is .GlobalEnv and which contains only what the
>> > the call to lm() requires and to compute lm() in that environment.
>>  E.g.,
>> >
>> > tfun1 <- function (subset)
>> > {
>> > junk <- 1:1e+06
>> > env <- new.env(parent = globalenv())
>> > env$subset <- subset
>> > with(env, lm(Sepal.Length ~ Sepal.Width, data = iris, subset =
>> subset))
>> > }
>> > Then we get
>> >> saveSize(tfun1(1:4)) # see below for def. of saveSize
>> >[1] 910
>> > instead of the 2129743 bytes in the save file when using the naive
>> method.
>> >
>> > saveSize <- function (object) {
>> > tf <- tempfile(fileext = ".RData")
>> > on.exit(unlink(tf))
>> > save(object, file = tf)
>> > file.size(tf)
>> > }
>> >
>> >
>> >
>> > Bill Dunlap
>> > TIBCO Software
>> > wdunlap tibco.com
>> >
>> > On Wed, Jul 27, 2016 at 10:48 AM, Kenny Bell 
>> wrote:
>> >
>> >> In the below, I generate a model from an environment that isn't
>> >> .GlobalEnv with a large object that is unrelated to the model
>> >> generation. It seems to save the irrelevant object unnecessarily. In
>> >> my actual use case, I am running and saving many models in a loop that
>> >> each use a single large data.frame (that gets collapsed into a small
>> >> data.frame for estimation), so removing it isn't an option.
>> >>
>> >> In the case where the model exists in .GlobalEnv, everything is
>> >> peachy. So replicating whatever happens when saving the model that was
>> >> generated in .GlobalEnv at the return() stage of the function call
>> >

[Rd] Writing files with long paths on Windows fails even after enabling long paths

2019-07-23 Thread Kenny Bell

Hello all,

We now can have long file paths on Windows by setting the LongPathsEnabled
key to 1 at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
in the Registry Editor. I can successfully rename files to have long paths
in File Explorer.

R still isn't able to take advantage of the change, however:

write(1, paste0(paste(sample(letters, 265, TRUE), collapse = ''), '.txt'))
#> Warning in file(file, ifelse(append, "a", "w")): cannot open file
#>
'mbpbzpgivtyhuugkkbcfxlvdzhsujaivaissyhivwqdxixmbnzctxlplazgtewpiioyxielkbohmvejcdkhcjmqtjyieymccrfquiwepngrseroiolayuucevztmqemnlnvfsshyhhggakxfemzzovwikusqtfeslwevxqmvufuyyvgvwwubmmpzvlcwaiwhipumrzphyeziwibisxszqrouekuonzjlaztdtonuupniqmxsaifpvkcldftkgnkozntgndwfu.txt':
#> No such file or directory
#> Error in file(file, ifelse(append, "a", "w")): cannot open the connection

I have read that long paths will be enabled by default in some future
release of Windows.

Hope this helps,
Kenny

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Problem with compiling OpenBLAS to work with R

2019-02-27 Thread Kenny Bell

This person has had apparent success - you could follow what they did or
just download their product (with appropriate caution downloading a random
.exe).

https://github.com/thequackdaddy/R-OpenBLAS

On Thu, Feb 28, 2019 at 6:28 AM Erin Hodgess 
wrote:

> Hello!
>
> I'm not sure if this is the right place to post this, so apologies
> in advance if I'm not in the right list.
>
> I downloaded the OpenBLAS and am following Avraham Adler's great
> instructions.  However, when I run make, things go well to a certain point,
> and then go bad:
>
> make
> [snip]
>
> touch cygopenblas_haswellp-r0.3.5.a
> make -j 1 -C test all
> make[1]: Entering directory
> '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> gfortran -O2 -Wall -frecursive -m64 -mavx2   -o sblat1 sblat1.o
> ../cygopenblas_haswellp-r0.3.5.a  -L/usr/lib/gcc/x86_64-pc-msys/7.3.0
> -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib/../lib
> -L/usr/lib/../lib
> -L/usr/lib/gcc/x86_64-pc-msys/7.3.0/../../../../x86_64-pc-msys/lib
> -L/usr/lib/w32api  -lmsys-2.0
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001088.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_destroy'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001090.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_init'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001091.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_lock'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001094.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_trylock'
> D:/msys64/usr/lib/../lib/libpthread.a(t-d001095.o):fake:(.text+0x2):
> undefined reference to `__imp_pthread_mutex_unlock'
> collect2.exe: error: ld returned 1 exit status
> make[1]: *** [Makefile:134: sblat1] Error 1
> make[1]: Leaving directory
> '/home/erinm/OPB_HOME/xianyi-OpenBLAS-eebc189/test'
> make: *** [Makefile:124: tests] Error 2
>
>
> I think it has something to do with the threads/pthreads but am not sure
> how to fix it.  Any suggestions much appreciated.
>
> Thanks,
> Sincerely,
> Erin
>
> Erin Hodgess, PhD
> mailto: erinm.hodg...@gmail.com
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Request: make `var` a generic

2019-01-20 Thread Kenny Bell

Hi all,

I am running into problems using `summary.lm` as calling `var` on a `units`
object does not retain the `units` attribute.

The units package addresses this issue for `mean` by implementing
`mean.units`:

units:::mean.units
function (x, ...)
{
.as.units(NextMethod(), units(x))
}

This is, however, not possible for `var` (as it's not a generic), which
breaks `summary.lm`:

```
library(units)
#> udunits system database from C:/R/Library/3.5/units/share/udunits
iris$Sepal.Length <- set_units(iris$Sepal.Length, "cm")

summary(lm(Sepal.Length ~ Sepal.Width,
   data = iris))
#> Error in Ops.units(mean(f)^2, var(f)): both operands of the expression
should be "units" objects
```

Created on 2019-01-21 by the [reprex package](https://reprex.tidyverse.org)
(v0.2.1.9000)

Apologies if `stats` package emails shouldn't go here.

Cheers,
Kenny

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Is this a bug in `[`?

2018-08-04 Thread Kenny Bell

This should more clearly illustrate the issue:

c(1, 2, 3, 4)[-seq_len(4)]
#> numeric(0)
c(1, 2, 3, 4)[-seq_len(3)]
#> [1] 4
c(1, 2, 3, 4)[-seq_len(2)]
#> [1] 3 4
c(1, 2, 3, 4)[-seq_len(1)]
#> [1] 2 3 4
c(1, 2, 3, 4)[-seq_len(0)]
#> numeric(0)
Created on 2018-08-05 by the reprex package (v0.2.0.9000).

On Sun, Aug 5, 2018 at 3:58 AM Rui Barradas  wrote:

>
>
> Às 15:51 de 04/08/2018, Iñaki Úcar escreveu:
> > El sáb., 4 ago. 2018 a las 15:32, Rui Barradas
> > () escribió:
> >>
> >> Hello,
> >>
> >> Maybe I am not understanding how negative indexing works but
> >>
> >> 1) This is right.
> >>
> >> (1:10)[-1]
> >> #[1]  2  3  4  5  6  7  8  9 10
> >>
> >> 2) Are these right? They are at least surprising to me.
> >>
> >> (1:10)[-0]
> >> #integer(0)
> >>
> >> (1:10)[-seq_len(0)]
> >> #integer(0)
> >>
> >>
> >> It was the last example that made me ask, seq_len(0) whould avoid an
> >> if/else or something similar.
> >
> > I think it's ok, because there is no negative zero integer, so -0 is 0.
>
> Ok, this makes sense, I should have thought about that.
>
> >
> > 1.0/-0L # Inf
> > 1.0/-0.0 # - Inf
> >
> > And the same can be said for integer(0), which is the result of
> > seq_len(0): there is no negative empty integer.
>
> I'm not completely convinced about this one, though.
> I would expect -seq_len(n) to remove the first n elements from the
> vector, therefore, when n == 0, it would remove none.
>
> And integer(0) is not the same as 0.
>
> (1:10)[-0] == (1:10)[0] == integer(0) # empty
>
> (1:10)[-seq_len(0)] == (1:10)[-integer(0)]
>
>
> And I have just reminded myself to run
>
> identical(-integer(0), integer(0))
>
> It returns TRUE so my intuition is wrong, R is right.
> End of story.
>
> Thanks for the help,
>
> Rui Barradas
>
> >
> > Iñaki
> >
> >>
> >>
> >> Thanks in advance,
> >>
> >> Rui Barradas
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R Compilation gets stuck on Windows 64

2018-02-09 Thread Kenny Bell

You can see how the appveyor build works here:

https://github.com/rwinlib/base

I suggest that you work off the build process in the  rwinlib repository so
you are starting from something that you know works and already
incorporates the set of dependencies you need.

On Fri, Feb 9, 2018, 5:33 AM Avraham Adler  wrote:

> On Fri, Feb 9, 2018 at 2:16 AM, Indrajit Sen Gupta 
> wrote:
> > Hi Avraham,
> >
> > A quick question - I realized I did not have Perl installed. So I
> installed
> > ActiveState Perl right now. Also I see I need texinfo and texi2any. I was
> > able to installed texinfo from here:
> > http://gnuwin32.sourceforge.net/packages/texinfo.htm. But not sure
> where to
> > get texi2any. Can you guide me in this step?
>
> It is in the ZIP file "texinfo5.zip" here [1]. Unzip that entire file
> into a directory and use that as your texinfo directory. That works
> for me.
>
> Avi
>
> [1] http://www.stats.ox.ac.uk/pub/Rtools/
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] saveRDS() overwrites file when object is not found

2018-02-07 Thread Kenny Bell

I ran into this behaviour when accidentally running a line of code that I
shouldn't have.

When saving over an rds with an object that's not found, I would have
expected saveRDS to not touch the file.

saveRDS(iris, "test.rds")
file.size("test.rds")
#> [1] 1080
saveRDS(no_object_here, "test.rds")
#> Error in saveRDS(no_object_here, "test.rds"): object 'no_object_here'
not found
file.size("test.rds")
#> [1] 20
file.remove("test.rds")

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Best practices in developing package: From a single file

2018-01-30 Thread Kenny Bell

In response to Duncan regarding the use of roxygen2 from the point of view
of a current user, I believe the issue he brings up is one of correlation
rather than causation.

Writing my first piece of R documentation was made much easier by using
roxygen2, and it shallowed the learning curve substantially.

What Duncan may be observing is a general tendency of roxygen2 users to
write overly concise documentation. However, I believe this to be caused by
an omitted variable - likely the tendency of roxygen2 users to want outputs
quickly. I can't see anything in roxygen2 that might suggest a causal link
but I'd be interested in hearing specific examples.

FWIW, I've also heard the same documentation criticism leveled at R in
general (mostly from Stata and MATLAB users).

Kenny

On Wed, Jan 31, 2018, 10:12 AM Dirk Eddelbuettel  wrote:

>
> Mehmet,
>
> That is a loaded topic, not unlikely other topics preoccupying us these
> days.
>
> There is package.skeleton() in base R as you already mentioned. It drove me
> bonkers that it creates packages which then fail R CMD check, so I wrote a
> wrapper package (pkgKitten) with another helper function (kitten()) which
> calls the base R helper and then cleans up it---but otherwise remains
> faithful to it.
>
> These days pkgKitten defaults to creating per-package top-level help page
> that just references content from DESCRIPTION via a set of newer Rd macros
> as
> I find that helps keeping them aligned. The most recent example of mine is
>   https://github.com/eddelbuettel/prrd/blob/master/man/prrd-package.Rd
> I use either this function or the RStudio template helper all the time.
>
> And similarly, other people written similar helpers. You may get other
> pointers.
>
> And every couple of months someone writes a new tutorial about how to
> write a
> first package. Then social media goes gaga and we get half a dozen blog
> posts
> where someone celebrates finding said tutorial, reading it and following
> through with a new package.
>
> And many of us taught workshops on creating packages. There is a lot of
> material out here, though lots of this material seems to be entirely
> ignorant
> of what came before it.
>
> And there has been lots, including Fritz's tutorial from a decade ago:
>
> https://epub.ub.uni-muenchen.de/6175/  as well as on CRAN as
> https://cran.r-project.org/doc/contrib/Leisch-CreatingPackages.pdf
>
> So I'd recommend you just experiment and set up your own helpers. After all
> the rule still holds: Anything you do more than three times should be a
> function, and every function should be in a package. So customize _your_
> function to create your package.
>
> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

2017-12-16 Thread Kenny Bell

Julia using OpenBLAS is *very* reassuring.

I agree that having it included as an options(...) feature should be OK.

On Sun, Dec 17, 2017, 3:22 PM Juan Telleria  wrote:

> Julia Programming Language uses also OpenBlas, and it is actively
> maintained with bugs being fixed as I have checked it out:
>
> http://www.openblas.net/Changelog.txt
>
> So I still see it ok to be included as an options(...) feature (by default
> off, just for safety), over other Blas libraries.
>
> R could not use Intel MKL for legal reasons (I think), because as long
> that R ships with GPL libraries, shipping R by default with Non-GPL is
> illegal.
>
> Cheers,
> Juan
>
> El 17/12/2017 2:50 a. m., "Avraham Adler" 
> escribió:
>
>> On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell  wrote:
>> > It seems like many of the multi-threaded BLASes have some sort of
>> > fundamental problem preventing use in the way Juan suggests:
>> >
>> >  - Dirk's vignette states that ATLAS "fixes the number of cores used at
>> > compile-time and cannot vary this setting at run-time", so any
>> > user-friendly implementation for R would have to compile ATLAS for 1-16
>> > threads to allow the user to switch at run-time. This might dramatically
>> > affect install times.
>> >
>> >  - MKL seems like it's been outright rejected in the past based on not
>> > being "free-enough".
>> >
>> >  - OpenBLAS causes crashes.
>> >
>> > Has anyone tried ExBLAS for use with R?
>> >
>> > On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder <
>> > peter.langfel...@gmail.com> wrote:
>> >
>> >> I would be very cautious about OpenBLAS in particular...  from time to
>> >> time I get complains from users that compiled code calculations in my
>> >> WGCNA package crash or produce wrong answers with large data, and they
>> >> all come from OpenBLAS users. I am yet to reproduce any of their
>> >> crashes when using MKL and ATLAS BLAS implementations.
>> >>
>> >> Just my 2 cents...
>> >>
>> >> Peter
>>
>> I've been building R on Windows 64 bit with OpenBLAS for years and my
>> builds pass check-devel. For a while in the past it failed one check
>> as the tolerance was 5e-5 and with my build of OpenBLAS the error was
>> 5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall
>> correctly. I provide descriptions here [1], but I haven't gone so far
>> as to post compiled Rblas.dlls just yet. My personal build sets 4
>> threads when compiling OpenBLAS itself as I'm currently on a quad-core
>> SandyBridge. In tests I ran a few years ago, both single and multi
>> threaded BLAS compile and then can be compiled into R with no issues
>> (on my platforms, at least). Most matrix operations performed better
>> with multi-threaded except for R's eigenvalue decomposition, to the
>> nest of my recollection.
>>
>> Avi
>>
>> [1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

2017-12-16 Thread Kenny Bell

It seems like many of the multi-threaded BLASes have some sort of
fundamental problem preventing use in the way Juan suggests:

 - Dirk's vignette states that ATLAS "fixes the number of cores used at
compile-time and cannot vary this setting at run-time", so any
user-friendly implementation for R would have to compile ATLAS for 1-16
threads to allow the user to switch at run-time. This might dramatically
affect install times.

 - MKL seems like it's been outright rejected in the past based on not
being "free-enough".

 - OpenBLAS causes crashes.

Has anyone tried ExBLAS for use with R?

On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder <
peter.langfel...@gmail.com> wrote:

> I would be very cautious about OpenBLAS in particular...  from time to
> time I get complains from users that compiled code calculations in my
> WGCNA package crash or produce wrong answers with large data, and they
> all come from OpenBLAS users. I am yet to reproduce any of their
> crashes when using MKL and ATLAS BLAS implementations.
>
> Just my 2 cents...
>
> Peter
>
> On Sat, Dec 16, 2017 at 12:28 PM, Kenny Bell  wrote:
> > Hi R-devel list,
> >
> > OpenBLAS is readily available for unix-likes:
> >
> > https://cloud.r-project.org/web/packages/gcbd/vignettes/gcbd.pdf
> >
> > However, my questions are:
> >
> > 1) Would R-devel consider using OpenBLAS for the main distribution of R
> for
> > all platforms including Windows?
> > 2) If so, would R-devel set the default multi-thread level to the number
> of
> > (real) cores on a machine?
> >
> > My sense is there're a lot of wasted resources on laptops and personal
> > desktops around the world that could be utilised by such a switch. I
> > believe most unix-like users don't know about OpenBLAS and are blissfully
> > ignorant of the available speed gains. It seems to be quite difficult
> for a
> > typical Windows user to get this done today.
> >
> > Thanks heaps,
> > Kenny
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

2017-12-16 Thread Kenny Bell

It seems that reproducibility across systems is also an issue with
multithreaded BLASes:

https://hal.archives-ouvertes.fr/hal-01202396/file/exblas.pdf

On Sun, Dec 17, 2017 at 11:50 AM, Keith O'Hara  wrote:

> On point 1):
>
> The standard approach seems to favor the reference BLAS for reasons other
> than speed.
>
> For example, vecLib, Apple's multi-threaded BLAS library, is not the
> default choice for macOS binaries due to concerns about 'precision'. See:
>
> https://cloud.r-project.org/bin/macosx/RMacOSX-FAQ.html#Whic
> h-BLAS-is-used-and-how-can-it-be-changed_003f
>
> This doesn't appear to be Mac- or vecLib-specifc. R-Core seem wary of
> non-reference BLAS implementations for several reasons:
>
> 'External BLAS implementations often make less use of extended-precision
> floating-point registers and will almost certainly re-order computations.
> This can result in less accuracy than using the internal BLAS, and may
> result in different solutions, e.g. different signs in SVD and
> eigendecompositions.'
>
> https://cran.r-project.org/doc/manuals/r-devel/R-admin.html#BLAS
>
> I'm not sure how pervasive a problem this is, though.
>
> Keith
>
>
> On Sat, Dec 16, 2017 at 4:01 PM, Dirk Eddelbuettel  wrote:
>
>>
>> Kenny,
>>
>> On 17 December 2017 at 09:28, Kenny Bell wrote:
>> | Hi R-devel list,
>> |
>> | OpenBLAS is readily available for unix-likes:
>> |
>> | https://cloud.r-project.org/web/packages/gcbd/vignettes/gcbd.pdf
>>
>> Please consider re-reading this vignette of mine. BLAS is an interface,
>> OpenBLAS is but one implementation.  R has allowed you to switch between
>> different implementations for a long time (if you used a shared library
>> build), and illustrating / measuring the possible performance differences
>> is
>> the whole point of the gcbd benchmarking package.
>>
>> | However, my questions are:
>> |
>> | 1) Would R-devel consider using OpenBLAS for the main distribution of R
>> for
>> | all platforms including Windows?
>> | 2) If so, would R-devel set the default multi-thread level to the
>> number of
>> | (real) cores on a machine?
>>
>> It's complicated. If you fork at the process-level (with package parallel
>> or
>> one of the many other mechansim) and then also used multi-threaded BLAS
>> you
>> can starve yourself for resources quickly.
>>
>> | My sense is there're a lot of wasted resources on laptops and personal
>> | desktops around the world that could be utilised by such a switch. I
>> | believe most unix-like users don't know about OpenBLAS and are
>> blissfully
>> | ignorant of the available speed gains. It seems to be quite difficult
>> for a
>> | typical Windows user to get this done today.
>>
>> Many things a developer / power-user would do are very difficult on
>> Windows. It is one of the charms of the platform. On the other hand you do
>> get a few solitaire games so I guess everybody is happy.
>>
>> Dirk
>>
>> | Thanks heaps,
>> | Kenny
>> |
>> |   [[alternative HTML version deleted]]
>> |
>> | __
>> | R-devel@r-project.org mailing list
>> | https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] OpenBLAS in everyday R?

2017-12-16 Thread Kenny Bell

On Sun, Dec 17, 2017 at 10:01 AM, Dirk Eddelbuettel  wrote:

>
> Kenny,
>
> On 17 December 2017 at 09:28, Kenny Bell wrote:
> | Hi R-devel list,
> |
> | OpenBLAS is readily available for unix-likes:
> |
> | https://cloud.r-project.org/web/packages/gcbd/vignettes/gcbd.pdf
>
> Please consider re-reading this vignette of mine. BLAS is an interface,
> OpenBLAS is but one implementation.  R has allowed you to switch between
> different implementations for a long time (if you used a shared library
> build), and illustrating / measuring the possible performance differences
> is
> the whole point of the gcbd benchmarking package.
>

Understood. To be clear, my suggestion is to change the *default* BLAS
implementation to multithreaded OpenBLAS.


> | However, my questions are:
> |
> | 1) Would R-devel consider using OpenBLAS for the main distribution of R
> for
> | all platforms including Windows?
> | 2) If so, would R-devel set the default multi-thread level to the number
> of
> | (real) cores on a machine?
>
> It's complicated. If you fork at the process-level (with package parallel
> or
> one of the many other mechansim) and then also used multi-threaded BLAS you
> can starve yourself for resources quickly.
>
>
This indeed was my experience if not being careful when using MRO, which
also has a multithreaded algebra library as the default. However, the
overall speed benefits far outweighed the costs. MRO helped to overcome
this problem with an interface to change the number of threads to use. In
MRO, it is on the user to switch this before using any explicit parallel
functionality. Another question, does using multithreaded OpenBLAS mess
with RcppParallel?


> | My sense is there're a lot of wasted resources on laptops and personal
> | desktops around the world that could be utilised by such a switch. I
> | believe most unix-like users don't know about OpenBLAS and are blissfully
> | ignorant of the available speed gains. It seems to be quite difficult
> for a
> | typical Windows user to get this done today.
>
> Many things a developer / power-user would do are very difficult on
> Windows. It is one of the charms of the platform. On the other hand you do
> get a few solitaire games so I guess everybody is happy.
>
> Dirk
>
> | Thanks heaps,
> | Kenny
> |
> |   [[alternative HTML version deleted]]
> |
> | __
> | R-devel@r-project.org mailing list
> | https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] OpenBLAS in everyday R?

2017-12-16 Thread Kenny Bell

Hi R-devel list,

OpenBLAS is readily available for unix-likes:

https://cloud.r-project.org/web/packages/gcbd/vignettes/gcbd.pdf

However, my questions are:

1) Would R-devel consider using OpenBLAS for the main distribution of R for
all platforms including Windows?
2) If so, would R-devel set the default multi-thread level to the number of
(real) cores on a machine?

My sense is there're a lot of wasted resources on laptops and personal
desktops around the world that could be utilised by such a switch. I
believe most unix-like users don't know about OpenBLAS and are blissfully
ignorant of the available speed gains. It seems to be quite difficult for a
typical Windows user to get this done today.

Thanks heaps,
Kenny

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?

2017-10-29 Thread Kenny Bell

User here: incorporating Intel's MKL, as MRO does, would be a very welcome
addition.

I was an MRO user before and it improved my experience with medium data
immensely.

They did, however, leave behind bugs here and there, especially related to
development with Rcpp, so I switched back to vanilla R.

On Mon, Oct 30, 2017, 9:42 AM Juan Telleria  wrote:

> Dear R Developers,
>
> First of all, I would like to thank you Jeroen Ooms for taking the binary
> Window Builds from Duncan. I firmly believe that the R Community will
> benefit a lot from his work.
>
> However, the debate I would like to open is about if some of Microsoft R
> Open Code shall be ported from R Open to Mainstream R.
>
> There are some beneficts in R Open such as multithreaded performance:
> https://mran.microsoft.com/documents/rro/multithread/
>
> Maybe, the R Consortium, and in particular, Microsoft R Team, could
> collaborate, if appropriate, in such duty.
>
> Thank you,
> Juan Telleria
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Parallel compression support for saving to rds/rdata files?

2016-12-15 Thread Kenny Bell

Hi,

I have tried to follow the instructions in the ``save`` documentation and
it doesn't seem to work (see below):

mydata <- do.call(rbind, rep(iris, 1))
con <- pipe("pigz -p8 > fname.gz", "wb");
save(mydata, file = con); close(con) # This runs

R.utils::gunzip("fname.gz", "fname.RData", overwrite = TRUE)
load("fname.RData") # Error: error reading from connection

First question: Should the above work?

Second question: Is it possible to make this dummy friendly by allowing
"pigz" as an option for ``compress`` in saveRDS and save? And in such a way
that the decompressing is hidden from the user like normal?

Thanks!
Kenny


-- 
Kendon Bell
Email: km...@berkeley.edu
Phone: (510) 612-3375

Ph.D. Candidate
Department of Agricultural & Resource Economics
University of California, Berkeley

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Spam messages

2016-12-02 Thread Kenny Bell

Have others received spam messages after posting to this list?

The two problem emails are hodgesdonna...@yahoo.com and
amykristen4...@octbm.com.

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] unlist strips date class

2016-12-02 Thread Kenny Bell

Is this a bug?

> unlist(list(as.Date("2015-01-01")))
[1] 16436

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] What happened to Ross Ihaka's proposal for a Common Lisp based R successor?

2016-08-05 Thread Kenny Bell

Is it conceivable that Julia could be ported to use R syntax in a way that
would allow the vastly larger numbers of R programmers to seamlessly
switch? Or equivalently, could an iteration of R itself do this?

On Fri, Aug 5, 2016, 9:00 AM Hadley Wickham  wrote:

> When it was being actively worked on, it had the advantage of existing.
>
> Hadley
>
> On Fri, Aug 5, 2016 at 10:48 AM, Kenny Bell  wrote:
> > Why is the described system preferable to Julia?
> >
> > On Fri, Aug 5, 2016, 4:50 AM peter dalgaard  wrote:
> >
> >>
> >> On 05 Aug 2016, at 06:41 , Andrew Judson  wrote:
> >>
> >> > I read this paper
> >> > <https://www.stat.auckland.ac.nz/~ihaka/downloads/Compstat-2008.pdf>
> and
> >> > haven't been able to find out what happened - I have seen some
> sporadic
> >> > mention in message groups but nothing definitive. Does anyone know?
> >>
> >> Presumably Ross does...
> >>
> >> You get a hint if you go one level up and look for the newest file:
> >>
> >> https://www.stat.auckland.ac.nz/~ihaka/downloads/New-System.pdf
> >>
> >>
> >> --
> >> Peter Dalgaard, Professor,
> >> Center for Statistics, Copenhagen Business School
> >> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> >> Phone: (+45)38153501
> >> Office: A 4.23
> >> Email: pd@cbs.dk  Priv: pda...@gmail.com
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>
> --
> http://hadley.nz
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] What happened to Ross Ihaka's proposal for a Common Lisp based R successor?

2016-08-05 Thread Kenny Bell

Why is the described system preferable to Julia?

On Fri, Aug 5, 2016, 4:50 AM peter dalgaard  wrote:

>
> On 05 Aug 2016, at 06:41 , Andrew Judson  wrote:
>
> > I read this paper
> >  and
> > haven't been able to find out what happened - I have seen some sporadic
> > mention in message groups but nothing definitive. Does anyone know?
>
> Presumably Ross does...
>
> You get a hint if you go one level up and look for the newest file:
>
> https://www.stat.auckland.ac.nz/~ihaka/downloads/New-System.pdf
>
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Model object, when generated in a function, saves entire environment when saved

2016-07-27 Thread Kenny Bell

Thanks so much for all this.

The first solution is what I'm going with as I want the terms object to
come along so that predict still works.

On Wed, Jul 27, 2016 at 12:28 PM, William Dunlap via R-devel <
r-devel@r-project.org> wrote:

> Another solution is to only save the parts of the model object that
> interest you.  As long as they don't include the formula (which is
> what drags along the environment it was created in), you will
> save space.  E.g.,
>
> tfun2 <- function(subset) {
>junk <- 1:1e6
>list(subset=subset, lm(Sepal.Length ~ Sepal.Width, data=iris,
> subset=subset)$coef)
> }
>
> saveSize(tfun2(1:4))
> #[1] 152
>
>
>
> Bill Dunlap
> TIBCO Software
> wdunlap tibco.com
>
> On Wed, Jul 27, 2016 at 11:19 AM, William Dunlap 
> wrote:
>
> > One way around this problem is to make a new environment whose
> > parent environment is .GlobalEnv and which contains only what the
> > the call to lm() requires and to compute lm() in that environment.
>  E.g.,
> >
> > tfun1 <- function (subset)
> > {
> > junk <- 1:1e+06
> > env <- new.env(parent = globalenv())
> > env$subset <- subset
> > with(env, lm(Sepal.Length ~ Sepal.Width, data = iris, subset =
> subset))
> > }
> > Then we get
> >> saveSize(tfun1(1:4)) # see below for def. of saveSize
> >[1] 910
> > instead of the 2129743 bytes in the save file when using the naive
> method.
> >
> > saveSize <- function (object) {
> > tf <- tempfile(fileext = ".RData")
> > on.exit(unlink(tf))
> > save(object, file = tf)
> > file.size(tf)
> > }
> >
> >
> >
> > Bill Dunlap
> > TIBCO Software
> > wdunlap tibco.com
> >
> > On Wed, Jul 27, 2016 at 10:48 AM, Kenny Bell  wrote:
> >
> >> In the below, I generate a model from an environment that isn't
> >> .GlobalEnv with a large object that is unrelated to the model
> >> generation. It seems to save the irrelevant object unnecessarily. In
> >> my actual use case, I am running and saving many models in a loop that
> >> each use a single large data.frame (that gets collapsed into a small
> >> data.frame for estimation), so removing it isn't an option.
> >>
> >> In the case where the model exists in .GlobalEnv, everything is
> >> peachy. So replicating whatever happens when saving the model that was
> >> generated in .GlobalEnv at the return() stage of the function call
> >> would fix this problem.
> >>
> >> I was referred to this list from r-bugs. First time r-devel poster.
> >>
> >> Hope this helps,
> >>
> >> Kendon
> >>
> >> ```
> >> tmp_fun <- function(x){
> >>   iris_big <- lapply(1:1, function(x) iris)
> >>   lm(Sepal.Length ~ Sepal.Width, data = iris)
> >> }
> >>
> >> out <- tmp_fun(1)
> >> object.size(out)
> >> # 48008
> >> save(out, file = "tmp.RData", compress = FALSE)
> >> file.size("tmp.RData")
> >> # 57196752 - way too big
> >>
> >> # Works fine when in .GlobalEnv
> >> iris_big <- lapply(1:1, function(x) iris)
> >> out <- lm(Sepal.Length ~ Sepal.Width, data = iris)
> >>
> >> object.size(out)
> >> # 48008
> >> save(out, file = "tmp.RData", compress = FALSE)
> >> file.size("tmp.RData")
> >> # 16641 - good size.
> >> ```
> >>
> >> [[alternative HTML version deleted]]
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Model object, when generated in a function, saves entire environment when saved

2016-07-27 Thread Kenny Bell

In the below, I generate a model from an environment that isn't
.GlobalEnv with a large object that is unrelated to the model
generation. It seems to save the irrelevant object unnecessarily. In
my actual use case, I am running and saving many models in a loop that
each use a single large data.frame (that gets collapsed into a small
data.frame for estimation), so removing it isn't an option.

In the case where the model exists in .GlobalEnv, everything is
peachy. So replicating whatever happens when saving the model that was
generated in .GlobalEnv at the return() stage of the function call
would fix this problem.

I was referred to this list from r-bugs. First time r-devel poster.

Hope this helps,

Kendon

```
tmp_fun <- function(x){
  iris_big <- lapply(1:1, function(x) iris)
  lm(Sepal.Length ~ Sepal.Width, data = iris)
}

out <- tmp_fun(1)
object.size(out)
# 48008
save(out, file = "tmp.RData", compress = FALSE)
file.size("tmp.RData")
# 57196752 - way too big

# Works fine when in .GlobalEnv
iris_big <- lapply(1:1, function(x) iris)
out <- lm(Sepal.Length ~ Sepal.Width, data = iris)

object.size(out)
# 48008
save(out, file = "tmp.RData", compress = FALSE)
file.size("tmp.RData")
# 16641 - good size.
```

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Possible bug in predict.lm when x is a poly

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Re: [Rd] [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Re: [Rd] Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Re: [Rd] Model object, when generated in a function, saves entire environment when saved

[Rd] Writing files with long paths on Windows fails even after enabling long paths

Re: [Rd] Problem with compiling OpenBLAS to work with R

[Rd] Request: make `var` a generic

Re: [Rd] Is this a bug in `[`?

Re: [Rd] R Compilation gets stuck on Windows 64

[Rd] saveRDS() overwrites file when object is not found

Re: [Rd] Best practices in developing package: From a single file

Re: [Rd] OpenBLAS in everyday R?

Re: [Rd] OpenBLAS in everyday R?

Re: [Rd] OpenBLAS in everyday R?

Re: [Rd] OpenBLAS in everyday R?

[Rd] OpenBLAS in everyday R?

Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?

[Rd] Parallel compression support for saving to rds/rdata files?

[Rd] Spam messages

[Rd] unlist strips date class

Re: [Rd] What happened to Ross Ihaka's proposal for a Common Lisp based R successor?

Re: [Rd] What happened to Ross Ihaka's proposal for a Common Lisp based R successor?

Re: [Rd] Model object, when generated in a function, saves entire environment when saved

[Rd] Model object, when generated in a function, saves entire environment when saved

27 matches

Site Navigation

Mail list logo

Footer information