Re: [Rd] R 4.0.0 build error with sysdata.rda on ppc64el architecture
Hum, at least it is not Apple, so maybe you can attach a debugger to the running process? (gdb -p process_id or something like that --- haven't actually done it for a decade). Then at least we can get a stack trace and a clue about where it is looping. Diddling optimization options can also sometimes provide a clue. -pd > On 29 Apr 2020, at 01:17 , Dirk Eddelbuettel wrote: > > > The R 4.0.0 package migration on Debian is being held back by a failed build > on ppc64el [1]. We can see from the history of builds logs [2] that it used > to build, briefly failed, worked again and then failed leading to R 4.0.0's > release. (And my bad for missing how the alpha1/alpha2/beta/rc builds failed.) > > I have however neither changed anything, nor did I ever have to accomodate > ppc64el (as it has happened with other platforms in the past). > > The automated build gets killed after 150 mins at > > make[7]: Entering directory '/<>/src/library/tools/src' > mkdir -p -- ../../../../library/tools/libs > make[7]: Leaving directory '/<>/src/library/tools/src' > make[6]: Leaving directory '/<>/src/library/tools/src' > make[5]: Leaving directory '/<>/src/library/tools' > make[5]: Entering directory '/<>/src/library/tools' > installing 'sysdata.rda' > E: Build killed with signal TERM after 150 minutes of inactivity > > as can be seen in [3]. The Debian wiki has pointers for getting a shell > account on such platforms [4] (and that is not limited to Debianers but a > 'Minipower' service). I now have one such account on the VM farm at Unicamp > [5] in Brazil. It uses OpenStack (slick, never used it before) and I just > provisioned a reasonably beefy machine, booted from one of the available OSs > (Ubuntu 20.04), installed the build-dependencies and ... am now hanging at > the exact same spot: > > make[7]: Entering directory '/home/ubuntu/git/r-base/src/library/tools/src' > mkdir -p -- ../../../../library/tools/libs > make[7]: Leaving directory '/home/ubuntu/git/r-base/src/library/tools/src' > make[6]: Leaving directory '/home/ubuntu/git/r-base/src/library/tools/src' > make[5]: Leaving directory '/home/ubuntu/git/r-base/src/library/tools' > make[5]: Entering directory '/home/ubuntu/git/r-base/src/library/tools' > installing 'sysdata.rda' > > So at least it reproduces. But how do we go about addressing this? Why would > it be looping infinitely trying to assemble sysdata.rda? > > Any hints or suggestions or debug flags I should set? > > Thanks in advance for any pointers, Dirk > > > [1] https://buildd.debian.org/status/package.php?p=r-base&suite=experimental > [2] https://buildd.debian.org/status/logs.php?pkg=r-base&arch=ppc64el > [3] > https://buildd.debian.org/status/fetch.php?pkg=r-base&arch=ppc64el&ver=4.0.0-1&stamp=1587737274&raw=0 > [4] https://wiki.debian.org/ppc64el > [5] https://openpower.ic.unicamp.br/minicloud/ > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] grid 4.0 generates wrong results when adding two complex units by sum()
Hi, In grid 4.0, adding two complex units by `sum()` seems to give wrong results. In the following example, `u1 + u2` gives the correct result, but `sum(u1, u2)` also `sum(unit.c(u1, u2))` give the wrong results. ``` library(grid) u1 = 0.4*sum(unit(1, "inch"), unit(1, "mm")) u2 = 0.1*sum(unit(1, "inch"), unit(1, "mm")) u1 # [1] 0.4*sum(1inches, 1mm) u2 # [1] 0.1*sum(1inches, 1mm) # this is correct u1 + u2 # [1] 0.5*sum(1inches, 1mm) # but this is wrong, it should return `sum(0.4inches, 0.4mm, 0.1inches, 0.1mm)`, right? sum(u1, u2) # [1] sum(0.4inches, 0.1mm, 1inches, 1mm) sum(unit.c(u1, u2)) # this is also wrong # [1] sum(0.4inches, 0.1mm, 1inches, 1mm) ``` Session info: ``` sessionInfo() # R version 4.0.0 (2020-04-24) # Platform: x86_64-apple-darwin17.0 (64-bit) # Running under: macOS Catalina 10.15.4 # # Matrix products: default # BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib # LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib # # locale: # [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 # # attached base packages: # [1] grid stats graphics grDevices utils datasets methods # [8] base # # other attached packages: # [1] colorout_1.2-2 # # loaded via a namespace (and not attached): # [1] compiler_4.0.0 ``` Thanks! Zuguang Gu [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] mclapply returns NULLs on MacOS when running GAM
> PS. Simon, I think your explicit comment on mcparallel() & friends is
very helpful for many people and developers. It clearly tells
developers to never use mclapply() as the only path through their
code. I'm quite sure not everyone has been or is aware of this. Now
it's clear. Thank you.
I second that, IMO that should land somewhere in manual.
On Wed, Apr 29, 2020 at 6:40 AM Henrik Bengtsson
wrote:
>
> On Tue, Apr 28, 2020 at 9:00 PM Shian Su wrote:
> >
> > Thanks Simon,
> >
> > I will take note of the sensible default for core usage. I’m trying to
> > achieve small scale parallelism, where tasks take 1-5 seconds and make
> > fuller use of consumer hardware. Its not a HPC-worthy computation but even
> > laptops these days come with 4 cores and I don’t see a reason to not make
> > use of it.
> >
> > The goal for the current piece of code I’m working on is to bootstrap many
> > smoothing fits to generate prediction intervals, this is quite easy to
> > write using mclapply. When you say native with threads, OpenMP, etc… are
> > you referring to at the C/C++ level? From my understanding most parallel
> > packages in R end up calling multicore or snow deep down.
> >
> > I think one of the great advantages of mclapply is that it defaults to
> > lapply when running on a single thread, this makes it much easier to
> > maintain code with optional parallelism. I’m already running into trouble
> > with the fact that PSOCK doesn’t seem to retain loaded packages in spawned
> > processes. I would love to know if there reliable options in R that allow a
> > similar interface to mclapply but use a different and more RStudio-stable
> > mode of parallelisation?
>
> If you use parLapply(cl, ...) and gives the end-users the control over
> the cluster 'cl' object (e.g. via an argument), then they have the
> option to choose from the different types of clusters that cl <-
> parallel::makeCluster(...) can create, notably PSOCK, FORK and MPI
> cluster but the framework support others.
>
> The 'foreach' framework takes this separation of *what* to parallelize
> (which you decide as a developer) and *how* to parallel (which the
> end-user decides) further by so called foreach adaptors aka parallel
> backends. With foreach, users have plently of doNnn packages to pick
> from, doMC, doParallel, doMPI, doSnow, doRedis, and doFuture. Several
> of these parallel backends build on top of the core functions provided
> by the 'parallel' package. So, with foreach your users can use forked
> parallel processing if they want and, or something else (selected at
> the top of their script).
>
> (Disclaimer: I'm the author) The 'future' framework tries to take this
> developer-end-user separation one step further and with a lower level
> API - future(), value(), resolved() - for which different parallel
> backends have been implemented, e.g. multicore, multisession
> ("PSOCK"), cluster (any parallel::makeCluster() cluster), callr,
> batchtools (HPC job schedulers), etc. All these have been tested to
> conform to the Future API specs, so we know our parallel code works
> regardless of which of these backends the user picks. Now, based on
> these basic future low-level functions, other higher level APIs have
> been implemented. For instance, the future.apply packages provides
> futurized version of all base R apply functions, e.g. future_lapply(),
> future_vapply(), future_Map(), etc. You can basically take you
> lapply(...) code and replace it with future_lapply(...) and things
> will just work. So, try replacing your current mclapply() with
> future_lapply(). If you/the user uses the 'multicore' backend - set
> by plan(multicore) at top of script, you'll get basically what
> mclapply() provides. If plan(multisession) is used, the you basically
> get what parLapply() does. The difference is that you don't have to
> worry about globals and packages. If you like the foreach-style of
> map-reduce, you can use futures via the doFuture backend. If you like
> the purrr-style of map-reduce, you can use the 'furrr' package. So,
> and I'm obviously biased, if you pick the future framework, you'll
> leave yourself and end-users with more options going forward.
>
> Clear as mud?
>
> /Henrik
>
> PS. Simon, I think your explicit comment on mcparallel() & friends is
> very helpful for many people and developers. It clearly tells
> developers to never use mclapply() as the only path through their
> code. I'm quite sure not everyone has been or is aware of this. Now
> it's clear. Thank you.
>
> >
> > Thanks,
> > Shian
> >
> > > On 29 Apr 2020, at 1:33 pm, Simon Urbanek
> > > wrote:
> > >
> > > Do NOT use mcparallel() in packages except as a non-default option that
> > > user can set for the reasons Henrik explained. Multicore is intended for
> > > HPC applications that need to use many cores for computing-heavy jobs,
> > > but it does not play well with RStudio and more importantly you don't
> > > know the resource availabl
Re: [Rd] grid 4.0 generates wrong results when adding two complex units by sum()
Confirmed. This is fixed now in R-devel and R-patched. Thanks for the report! Paul On 29/04/20 10:05 pm, Gu, Zuguang wrote: Hi, In grid 4.0, adding two complex units by `sum()` seems to give wrong results. In the following example, `u1 + u2` gives the correct result, but `sum(u1, u2)` also `sum(unit.c(u1, u2))` give the wrong results. ``` library(grid) u1 = 0.4*sum(unit(1, "inch"), unit(1, "mm")) u2 = 0.1*sum(unit(1, "inch"), unit(1, "mm")) u1 # [1] 0.4*sum(1inches, 1mm) u2 # [1] 0.1*sum(1inches, 1mm) # this is correct u1 + u2 # [1] 0.5*sum(1inches, 1mm) # but this is wrong, it should return `sum(0.4inches, 0.4mm, 0.1inches, 0.1mm)`, right? sum(u1, u2) # [1] sum(0.4inches, 0.1mm, 1inches, 1mm) sum(unit.c(u1, u2)) # this is also wrong # [1] sum(0.4inches, 0.1mm, 1inches, 1mm) ``` Session info: ``` sessionInfo() # R version 4.0.0 (2020-04-24) # Platform: x86_64-apple-darwin17.0 (64-bit) # Running under: macOS Catalina 10.15.4 # # Matrix products: default # BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib # LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib # # locale: # [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 # # attached base packages: # [1] grid stats graphics grDevices utils datasets methods # [8] base # # other attached packages: # [1] colorout_1.2-2 # # loaded via a namespace (and not attached): # [1] compiler_4.0.0 ``` Thanks! Zuguang Gu [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Dr Paul Murrell Department of Statistics The University of Auckland Private Bag 92019 Auckland New Zealand 64 9 3737599 x85392 p...@stat.auckland.ac.nz http://www.stat.auckland.ac.nz/~paul/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R 4.0.0 build error with sysdata.rda on ppc64el architecture
On 29 April 2020 at 11:22, peter dalgaard wrote: | Hum, at least it is not Apple, so maybe you can attach a debugger to the running process? (gdb -p process_id or something like that --- haven't actually done it for a decade). Then at least we can get a stack trace and a clue about where it is looping. Diddling optimization options can also sometimes provide a clue. (Missed this earlier as the conversation moved off-list.) And to keep the list abreast, this appears to be related to the long double issue on powerpc where needed an extra #define to ensure compilation. That commit is the difference in a bisection as I was able to demonstrate. The issue can also be circumvented by disabling long double support on the platform, but hopefully a better fix can be found. Bryan Lewis was eagle-eyed on this and very helpful. The issue is now back in the hands of R Core and I and others will await the news. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] "not a valid win32 application" with rtools40-x86_65.exe on Windows 10
Hello, All: "00install.out" from "R CMD check Ecfun_0.2-4.tar.gz" includes: Error: package or namespace load failed for 'Ecfun': .onLoad failed in loadNamespace() for 'rJava', details call: inDL(x, as.logical(local), as.logical(now), ...) error: unable to load shared object 'c:/Program Files/R/R-4.0.0/library/rJava/libs/i386/rJava.dll': LoadLibrary failure: ^1 is not a valid win32 application This was after installing R 4.0.0 and "rtools40-x86_64.exe" under Windows 10 Pro 64-bit. Suggestions? Thanks, Spencer Graves sessionInfo() R version 4.0.0 (2020-04-24) Platform: x86_64-64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CCTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_4.0.0 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] "not a valid win32 application" with rtools40-x86_65.exe on Windows 10
Are you missing the 32-bit Java JDK? Cheers, S > On 30/04/2020, at 4:37 PM, Spencer Graves wrote: > > Hello, All: > > > "00install.out" from "R CMD check Ecfun_0.2-4.tar.gz" includes: > > > Error: package or namespace load failed for 'Ecfun': > .onLoad failed in loadNamespace() for 'rJava', details > call: inDL(x, as.logical(local), as.logical(now), ...) > error: unable to load shared object 'c:/Program > Files/R/R-4.0.0/library/rJava/libs/i386/rJava.dll': > LoadLibrary failure: ^1 is not a valid win32 application > > > This was after installing R 4.0.0 and "rtools40-x86_64.exe" under > Windows 10 Pro 64-bit. > > > Suggestions? > Thanks, > Spencer Graves > > > sessionInfo() > R version 4.0.0 (2020-04-24) > Platform: x86_64-64-mingw32/x64 (64-bit) > Running under: Windows 10 x64 (build 18362) > > Matrix products: default > > locale: > [1] LC_COLLATE=English_United States.1252 > [2] LC_CCTYPE=English_United States.1252 > [3] LC_MONETARY=English_United States.1252 > [4] LC_NUMERIC=C > [5] LC_TIME=English_United States.1252 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > loaded via a namespace (and not attached): > [1] compiler_4.0.0 > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
