Re: [Rd] Possible Bug: file.exists() Function. Due to UTF-8 Encoding differences on Windows between R 4.0.1 and R 3.6.3?
Thank you Kevin, just checked that the error is solved in the latest development version of "renv", and now it works as expected with R 4.0.1: https://github.com/rstudio/renv/commit/976ae7af6dc348af30eaf2893d886f132a76aba0 Sorry for posting in r-devel, I was not sure if it was a R or "renv" error due to different behaviour in different versions of R 4.0.1 and R 3.6.3 for conversion from UTF16-LE to UTF-8 encoding. Will provide a better reproducible example next time and traceback the error with options(error=recover)) to make sure. Thanks, Juan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Possible Bug: file.exists() Function. Due to UTF-8 Encoding differences on Windows between R 4.0.1 and R 3.6.3?
Dear R Developers,
I am having an issue with the renv package and R 4.0.1, which I
suspect is related to base R and not the renv package itself, as with
R 3.6.3 such an "error" does not appear.
The error is raised by a file.exists() path, and path
"C:\Users\J-tel\Documents", which in R 3.6.3 is read correctly, but in
R 4.0.1 fails (Probably because of the "-" symbol), and I suspect it
might be related with the new UTF-8 usage on Windows 10?
(https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-on-windows/index.html)
I have also checked file.exists() function and its internals, and seem
not to have happened changes in the meanwhile within them:
https://github.com/wch/r-source/blob/0e3b3182f87a60af4b0293a5410dde680b910f49/src/library/base/R/files.R
https://github.com/search?q=SEXP%20attribute_hidden%20do_fileexists+repo:wch/r-source&type=Code
Error Details:
> renv::init()
Error in file.exists(children) :
file name conversion problem -- name too long?
> traceback()
14: file.exists(children)
13: renv_dependencies_find_dir_children(path, root)
12: renv_dependencies_find_dir(path, root)
11: FUN(X[[i]], ...)
10: lapply(path, renv_dependencies_find_impl, root = root)
9: renv_dependencies_find(path, root)
8: (function (path = getwd(), root = NULL, ..., progress = TRUE,
errors = c("reported", "fatal", "ignored"), dev = FALSE)
{
path <- renv_path_normalize(path, winslash = "/", mustWork = TRUE)
root <- root %||% renv_dependencies_root(path)
if (exists(path, envir = `_renv_dependencies`))
return(get(path, envir = `_renv_dependencies`))
renv_dependencies_begin(root = root)
on.exit(renv_dependencies_end(), add = TRUE)
dots <- list(...)
if (identical(dots[["quiet"]], TRUE)) {
progress <- FALSE
errors <- "ignored"
}
files <- renv_dependencies_find(path, root)
deps <- renv_dependencies_discover(files, progress, errors)
renv_dependencies_report(errors)
deps
})(path, progress = FALSE, errors = errors, dev = TRUE)
7: eval(call, envir = parent.frame(2))
6: eval(call, envir = parent.frame(2))
5: delegate(renv_dependencies_impl)
4: dependencies(path, progress = FALSE, errors = errors, dev = TRUE)
3: withCallingHandlers(dependencies(path, progress = FALSE, errors = errors,
dev = TRUE), renv.dependencies.error =
renv_dependencies_error_handler(message,
errors))
2: renv_dependencies_scope(project, action = "init")
1: renv::init()
> renv::diagnostics()
Diagnostics Report -- renv [0.10.0]
===
# Session Info ===
R version 4.0.1 (2020-06-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)
Matrix products: default
locale:
[1] LC_COLLATE=Spanish_Spain.1252 LC_CTYPE=Spanish_Spain.1252
[3] LC_MONETARY=Spanish_Spain.1252 LC_NUMERIC=C
[5] LC_TIME=Spanish_Spain.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] renv_0.10.0
loaded via a namespace (and not attached):
[1] compiler_4.0.1 rsconnect_0.8.16 htmltools_0.4.0 tools_4.0.1
[5] yaml_2.2.1 Rcpp_1.0.4.6 rmarkdown_2.2knitr_1.28
[9] xfun_0.14digest_0.6.25packrat_0.5.0rlang_0.4.6
[13] evaluate_0.14
# Project
Project path: "~/Test2"
# Status =
# Lockfile ===
This project has not yet been snapshotted: 'renv.lock' does not exist.
# Library
The project library "~/Test2/renv/library/R-4.0/x86_64-w64-mingw32"
does not exist.
# Dependencies ===
# User Profile ===
[no user profile detected]
# Settings ===
List of 6
$ external.libraries : chr(0)
$ ignored.packages : chr(0)
$ package.dependency.fields: chr [1:3] "Imports" "Depends" "LinkingTo"
$ snapshot.type: chr "implicit"
$ use.cache: logi TRUE
$ vcs.ignore.library : logi TRUE
# Options
List of 1
$ renv.verbose: logi TRUE
# Environment Variables ==
HOME= C:\Users\J-tel\OneDrive\Documents
LANG=
R_LIBS =
R_LIBS_SITE =
R_LIBS_USER = C:/Users/J-tel/OneDrive/Documents/R/win-library/4.0
# PATH ===
- C:\rtools40\usr\bin
- C:\Program Files\R\R-4.0.1\bin\x64
- C:\ProgramData\Miniconda3
- C:\ProgramData\Miniconda3\Library\mingw-w64\bin
- C:\ProgramData\Miniconda3\Library\usr\bin
- C:\ProgramData\Miniconda3\Library\bin
- C:\ProgramData\Miniconda3\Scripts
- C:\ProgramData\Oracle\Java\javapath
- C:\WINDOWS\system32
- C:\WINDOWS
- C:\WINDOWS\System32\Wbem
- C:\WINDOWS\System32\WindowsPowerShell\v1.0\
- C:\WINDOWS\System32\OpenSSH\
- C:\Program Files\MiKTeX 2.9\miktex\bin\x64\
- C:\ProgramData\Miniconda3\Scripts\conda.exe
# Cache ==
There a
Re: [Rd] [External] Feature Request: User Prompt + Message First Execution when "Managing Search Path Conflicts"
Got it, thank you for pointing out the solution then. Best, Juan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [External] Feature Request: User Prompt + Message First Execution when "Managing Search Path Conflicts"
Thank you Mr. Tierney!
Using globalCallingHandlers() to directly handle
"packageConflictError" is an excellent idea!
The benefits I see for such an implementation are:
* The patch would be contained within the Conflict Error Handler,
which should reduce any side effects with an eventual implementation.
* And by making its usage optional, by setting for example
options(conflicts.policy.ask = TRUE), in should neither affect any
packages nor other base code.
Hope it allows R Users to work in a more agile manner, and guide R
Students through best practices of variable conflict handling in an
educative manner.
Thanks,
Juan
> You can get what you are asking for now in R 4.0.0 with
> globalCallingHandlers and using the packageConflictError object that
> is signaled. This should get you started:
>
> ```
> options(conflicts.policy = "strict")
>
> packageConflictError
>
> handle_conflicts <- function(e) {
> cat(conditionMessage(e))
> opt <- readline(prompt="1: mask.ok; 2: exclude. Choose: ")
> if (opt == "1")
> conflictRules(e$package, mask.ok = as.character(unlist(e$conflicts)))
> else if (opt == "2")
> conflictRules(e$package, exclude = as.character(unlist(e$conflicts)))
> stop("unresolved conflicts") ## ideal invode a restart here
> }
>
> globalCallingHandlers(packageConflictError = handle_conflicts)
>
> library(dplyr)
> ```
>
> An IDE could provide a more sophisticated interface, like a dialog
> allowing separate choices for each conflict. But this is best left up
> to the IDE or the user.
>
> The one addition to library that might be worth considering is to
> provide a restart for the handler to invoke.
>
> Best,
>
> luke
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Feature Request: User Prompt + Message First Execution when "Managing Search Path Conflicts"
Dear R Developers,
###
# Context:
###
When managing Search Path Conflicts (See:
https://developer.r-project.org/Blog/public/2019/03/19/managing-search-path-conflicts/index.html),
with:
options(conflicts.policy = "strict")
We get the following behaviour when loading a package (Eg: dplyr):
library(dplyr)
## Error: Conflicts attaching package ‘dplyr’:
##
## The following objects are masked from ‘package:stats’:
##
## filter, lag
##
## The following objects are masked from ‘package:base’:
##
## intersect, setdiff, setequal, union
So we would have to solve the conflict by writing:
library(dplyr,
mask.ok = c("filter", "lag",
"intersect", "setdiff", "setequal",
"union"))
So my feature request proposals:
###
# Feature Request 1: Interactive Session
###
Would it be possible to raise an input prompt, which asks user for an
action to be taken as regards conflicts?
This would make the package loading process more dynamic when being
loaded for first time in an interactive session (Eg: R Notebook). An
example:
The first time the package is loaded:
options(conflicts.policy = "strict", conflicts.policy.ask = TRUE)
library(dplyr)
Executes iteratively the code, in order to ask the user for action
(See toy example):
opt <- readline(prompt="1: mask.ok; 2: exclude. Choose: ")
if (opt == "1"){
txt <- sprintf(
fmt = "conflictRules(pkg = '%s' , mask.ok = '%s')",
"package.name",
"variable.name"
)
eval(parse(text = txt))
message(txt)
} else if(opt == "2"){
txt <- sprintf(
fmt = "conflictRules(pkg = '%s' , exclude = '%s')",
"package.name",
"variable.name"
)
eval(parse(text = txt))
message(txt)
}
And afterwards, a message is printed with the selected
"conflictRules()" Configuration for the dynamic setup.
The user will only have to put the printed pre-configured
"conflictRules()" setup into his code, and when re-executing, the
input prompt asking for action will not be raised back again.
Such behaviour is similar in spirit to how 'conflicted' R package
works, which prints for example, for dplyr::filter :
Error: [conflicted] `filter` found in 2 packages.
Either pick the one you want with `::`
* dplyr::filter
* stats::filter
Or declare a preference with `conflict_prefer()`
* conflict_prefer("filter", "dplyr")
* conflict_prefer("filter", "stats")
Where the user will only have to copy "conflict_prefer("filter",
"dplyr")" and paste it into his script. The difference, is that with:
options(conflicts.policy = "strict", conflicts.policy.ask = TRUE),
such message is printed at package load.
I would put the required code within the library() function with the
following conditional:
...
if (length(conflicts)) {
if(getOption("conflicts.policy.ask") == TRUE){
...
}
}
...
###
# Feature Request 2: Source Execution
###
Another alternative, which could apply when executing the .R file from
source, would be to suggest the user a default conflictRules() setup:
options(conflicts.policy = "strict")
library(dplyr)
# Error: Conflicts attaching package ‘dplyr’:
#
# The following objects are masked from ‘package:stats’:
#
# filter, lag
#
# The following objects are masked from ‘package:base’:
#
# intersect, setdiff, setequal, union
#
# Declare preference with `conflictRules()` before loading:
# * conflictRules("dplyr", mask.ok = list(stats = TRUE, base = TRUE))
# * conflictRules("dplyr", mask.ok = list(stats = c("filter",
"lag"), base = c("intersect", "setdiff", "setequal", "union")))
# * conflictRules("dplyr", exclude = c("filter", "lag",
"intersect", "setdiff", "setequal", "union"))
In this case, the error message would have to be extended with
sensible suggested defaults.
Thanks,
Juan
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Support for Dashes in the Raw String Delimiter
Dear R Developers, As regards "Support for Dashes in the Raw String Delimiter" from commit: https://github.com/wch/r-source/commit/4d4781ad19890193d5eb458d71f18d7e53ee73c5 Would it be possible to support in addition to r"" Syntax, for not escaping backlash character in strings, also support """ """ (Python Like Syntax), for also allowing to have within the character string the closing sequence " symbol? See Python's article on string literals for further information: https://docs.python.org/2.0/ref/strings.html Thanks! [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] how to check as CRAN with alternative BLAS?
Give a try to "rhub": https://cran.r-project.org/web/packages/rhub/index.html https://cran.r-project.org/web/packages/rhub/vignettes/local-debugging.html Which allows to try different os, and system configurations :) Hope it works! El lunes, 30 de diciembre de 2019, Dirk Eddelbuettel escribió: > > On 29 December 2019 at 16:39, steven pav wrote: > | One of my packages is slated to be archived from CRAN due to failures > when > | the ATLAS BLAS is used. I am unable to replicate the error on my machine > | under R 3.6.1 using the atlas library from ubuntu (seems to be 3.10.2-9, > | while the good professor is using 3.10.3 per > | https://www.stats.ox.ac.uk/pub/bdr/Rblas/README.txt ). I also tried the > | rocker/r-base with R 3.6.2 > | and /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3 (Dockerfile here: > | https://gist.github.com/shabbychef/efaa048f0f715dcf89fd39f2e28a402d ), > but > | was unable to replicate the error there. Is there some way to *really* > test > | as CRAN-with-atlas? I would prefer a dockerized solution, or failing > that, > | a travis CI recipe. > > That CRAN checks cannot always be replicated outside of CRAN is a real > issue. > > This has been brought up before, and some of us wanted to see about > improving > it, possibly relying on Docker---but as you know life is short, spare time > is > generally missing, and other shiny things can distract us. So nothing to > report yet. Which is a bummer. > > Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] reverse dependency checks
https://builder.r-hub.io/ El martes, 3 de septiembre de 2019, Therneau, Terry M., Ph.D. via R-devel < r-devel@r-project.org> escribió: > I remember there was advice about a server that one could use for reverse > dependency > checks, but I forgot to write it down. (Or I did save the info and forgot > where I saved > it...) I have been doing the checks for survival myself, but the count > is getting out of > hand (663, not counting bioconductor). > > Any pointers? > > Terry Therneau > > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Recommended Reading: Advanced R Second Edition
Dear R Developers, After having fully read "Advanced R First Edition" , and having just bought my physical copy of "Advanced R Second Edition", I recommend that: Any community member interested in the development of R reads "Advanced R Second Edition", which explains R Language Core concepts cristal clear, and shows the motivation behind libraries such as "rlang", "purrr", "bench", "profvis", "sloop", "lobstr", above others. I'm sure you will learn something new and enjoy the Reading! Digital Book (Free): https://adv-r.hadley.nz/ Physical Book: https://www.amazon.com/Advanced-Second-Chapman-Hall-CRC/dp/0815384572/ref=mp_s_a_1_1?keywords=advanced+r+second+edition&qid=1563703482&s=gateway&sprefix=advanced+R+second+edition&sr=8-1 Best, Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Converting non-32-bit integers from python to R to use bit64: reticulate
Thank you Martin for giving to know and developing 'Rmpfr' library for unlimited size integers (GNU C GMP) and arbitrary precision floats (GNU C MPFR): https://cran.r-project.org/package=Rmpfr My question is: In the long term (For R3.7.0 or R3.8.0): Does it have sense that CMP substitutes INTSXP, and MPFR substitutes REALSXP code? With this we would achieve that an integer is always an integer, and a numeric double precision float always a numeric double precision float, without sometimes casting underneath. And would the R Community / R Ordinary Members would be willing to help R Core on such implementation (If has sense, and wants to be adopted)? Thank you all! :) > > If you are interested in using unlimited size integers, you > could use the CRAN R package 'gmp' which builds on the GMP = GNU > MP = GNU Multi Precision C library. > >https://cran.r-project.org/package=gmp > > (and for arbitrary precision "floats", see CRAN pkg 'Rmpfr' > built on package gmp, and both the GNU C libraries GMP and > MPFR: >https://cran.r-project.org/package=Rmpfr > ) > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Converting non-32-bit integers from python to R to use bit64: reticulate
Thank you Gabriel for valuable insights on the 64-bit integers topic. In addition, my statement was wrong, as Python3 seems to have unlimited (and variable) size integers. Here is related CPython Code: https://github.com/python/cpython/blob/master/Objects/longobject.c Division between Int-32 and Int-64 seems to only happen in Python2. Best, Juan El miércoles, 29 de mayo de 2019, Gabriel Becker escribió: > Hi Juan, > > Comments inline. > > On Wed, May 29, 2019 at 12:48 PM Juan Telleria Ruiz de Aguirre < > jtelleria.rproj...@gmail.com> wrote: > >> Dear R Developers, >> >> There is an interesting issue related to "reticulate" R package which >> discusses how to convert Python's non-32 bit integers to R, which has had >> quite an exhaustive discussion: >> >> https://github.com/rstudio/reticulate/issues/323 >> >> Python seems to handle integers differently from R, and is dependant on >> the >> system arquitecture: On 32 bit systems uses 32-bit integers, and on 64-bit >> systems uses 64-bit integers. >> >> So my question is: >> >> As regards R's C Interface, how costly would it be to convert INTSXP from >> 32 bits to 64 bits using C, on 64 bits Systems? Do the benefits surpass >> the >> costs? And should such development be handled from within R Core / >> Ordinary >> Members , or it shall be left to package maintainers? >> > > Well, I am not an R-core member, but I can mention a few things: > > 1. This seems like it would make the results of R code non-reproducible > between 32 and 64bit versions of R; at least some code would give different > results (at the very least in terms of when integer values overflow to NA, > which is documented behavior). > 2. Obviously all integer data would take twice as much memory, memory > bandwidth, space in caches, etc, even when it doesn't need it. > 3. Various places treat data /data pointers coming out of INTSXP and > LGLSXP objects the same within the internal R sources (as currently they're > both int/int*). Catching and fixing all those wouldn't be impossible, but > it would take at least some doing. > > For me personally 1 seems like a big problem, and 3 makes the conversion > more work than it might have seemed initially. > > As a related side note, as far as I understand what I've heard from R-core > members directly, the choice to not have multiple types of integers is > intentional and unlikely to change. > > Best, > ~G > > > > >> >> Thank you! :) >> >> [[alternative HTML version deleted]] >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Converting non-32-bit integers from python to R to use bit64: reticulate
Dear R Developers, There is an interesting issue related to "reticulate" R package which discusses how to convert Python's non-32 bit integers to R, which has had quite an exhaustive discussion: https://github.com/rstudio/reticulate/issues/323 Python seems to handle integers differently from R, and is dependant on the system arquitecture: On 32 bit systems uses 32-bit integers, and on 64-bit systems uses 64-bit integers. So my question is: As regards R's C Interface, how costly would it be to convert INTSXP from 32 bits to 64 bits using C, on 64 bits Systems? Do the benefits surpass the costs? And should such development be handled from within R Core / Ordinary Members , or it shall be left to package maintainers? Thank you! :) [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Managing Search Path Conflicts in R 3.6.0: Lazy Method Attachment into Search Path
Dear R Developers, R 3.6.0 is going to introduce new features for managing search path conflicts, explained in greater detail in the following article, and which are greatly welcome: https://developer.r-project.org/Blog/public/2019/03/19/managing-search-path-conflicts/index.html In addition, another Conflict Policy could also be adopted, which I will call "Lazy Method Attachment into the Search Path", understood as: * All non-conflicting methods are always attached into the Search path. * If a conflict occurs, method for a specific class object, is not attached, unless you are going to use it, and then the user is asked in the command prompt if you want it attached or not. For example, if I set: options(conflict.policy = "ask") library(dplyr) # All works fine. df %>% select(col1, col2) # Works. df %>% filter(col1 == "a") #> Do you want to attach data.frame.filter Method? y/n? This approach would be great for dynamic programming and R Scripting, and lay somewhere between R Core / @Luke Tierney's approach and "conflicted" package. Thank you! [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Suggestion: Make CRAN source URLs immutable
It would be nice to see such immutable package version links: E.g.: https://cran.r-project.org/package=httr&version=1.3.1 https://cran.r-project.org/package=httr&version=1.3.0 In the CRAN package web pages themselves, and specifically in the "Old Sources: httr Section": https://cran.r-project.org/web/packages/httr/index.html In order that package users become aware of it. But just as actually is also perfectly fine! Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Native 64 Integers
Dear R Developers, I would like to pick up back again the issue of 64 bits integers with R: http://r.789695.n4.nabble.com/Re-R-support-for-64-bit-integers-td2320024.html *** CURRENT SITUATION *** At the moment, as regards integers, all the following are the same type: * length of an R vector * R integer type * C int type (Fixed at 32 bits: In practice) * Fortran INTEGER type (Fixed at 32 bits: By Standard) *** OBJECTIVE *** Introducing 64-bit integers natively into "base R", notably if it was also allowed using them for indices. And, ideally, we would like: * length of an R vector. * R integer type. To become 64bit. This would allow to free ourselves from the increasingly relevant maximum-atomic-object-length = 2^31 problem. *** DIFFICULTIES *** a) If both the R length type and the R integer type become the same 64bit type and replace the current integer type -> Then every compiled package would have to change to declare the arguments as int64 (or long, on most 64bit systems) and INTEGER*8. b) If the R length type changes to something /different/ from the integer type then any compiled code has to be checked to see if C int arguments are lengths or integers, which is more work and more error-prone. c) On the other hand, changing the integer type to 64bit -> Will presumably make integer code run noticeably more slowly on 32bit systems. In any case, the changes could be postponed by having an option to .C/.Call forcing lengths and integers to be passed as 32-bit -> This would mean that: The code couldn't use large integers or large vectors, but it would keep working indefinitely. *** 2010 SOLUTION*** There were 2 possibilities at the time: a) Using 64-bit integers. b) Using "double precision integers": Solution Finally Chosen at 2010. Reason: In order that not all R packages using compiled code had to be patched extensively. *** BIT64 PACKAGE*** Nowdays, we have 'bit64' Package, which provides serializable S3 atomic 64bit (signed) integers (+-2^63). But this are not a replacement for 32bit integers, as integer64 are: * Not supported for subscripting. * Have different semantics when combined with double, e.g. integer64 + double => integer64. https://cran.r-project.org/web/packages/bit64/index.html *** PROPOSAL *** Instead of seeing 64 integers as a substitution to 32 bit integers, these could be included into base R as a new / additional data type, which co-exists with: a) Using 64-bit integers. b) Using "double precision integers". This new data type could: * Be based (ported) from "bit64" package: https://github.com/cran/bit64 * Allow to use int64 Data Type for Subscripting. * Have Coercion Rules such as: as.integer64() is.integer64() integer + integer64 => integer double + integer64 => double * Be included with a double "L". e.g.: 34783274893274892334279LL (This would be integer64, not double). By doing so, existing packages would not need to be recompiled, and could keep on working as already do. So we would not introduce backward incompatible change. *** FINAL KEY IDEA *** Take already developed "bit64" Package (https://github.com/cran/bit64) as base for building a new Integer64 Type System which co-exists natively in R with Integer32 (Just as "parallel" package was included in the past into base R for example), and build on top of it improvements. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] compairing doubles
Maybe a new Operator could be defined for a fast and easy double Comparison: `~~` `~~` <- function (e1, e2) all.equal(e1, e2) And document it properly. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Code Optimization: print.data.frame + as.data.frame(head(x, n = options("max.print")))
I polished a little bit more the function:
* Used: getOption("max.print")
* Added comment at the end: cat('[ reached getOption("max.print") --
omitted ', omitted,' rows ]')
function (x, ..., digits = NULL, quote = FALSE, right = TRUE,
row.names = TRUE)
{
n <- length(row.names(x))
if (length(x) == 0L) {
cat(sprintf(ngettext(n, "data frame with 0 columns and %d row",
"data frame with 0 columns and %d rows"), n), "\n",
sep = "")
}
else if (n == 0L) {
print.default(names(x), quote = FALSE)
cat(gettext("<0 rows> (or 0-length row.names)\n"))
}
else {
omitted <- nrow(x)-getOption("max.print")
x <- as.data.frame(head(x, n = getOption("max.print")))
m <- as.matrix(format.data.frame(x, digits = digits,
na.encode = FALSE))
if (!isTRUE(row.names))
dimnames(m)[[1L]] <- if (isFALSE(row.names))
rep.int("", n)
else row.names
print(m, ..., quote = quote, right = right)
if((nrow(x)-getOption("max.print"))>0){
cat('[ reached getOption("max.print") -- omitted ', omitted,' rows ]')
}
}
invisible(x)
}
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Code Optimization: print.data.frame + as.data.frame(head(x, n = options("max.print")))
Dear R Developers,
I would like to propose a simple optimization for print.data.frame
base function:
To add: x <- as.data.frame(head(x, n = options("max.print")))
This would prevent that, if for example, we have a 10GB data.frame
(e.g.: Instead of a data.table), and we accidentally print it, the R
Session does not "collapse", forcing us to press ESC or kill the
RSession.
function (x, ..., digits = NULL, quote = FALSE, right = TRUE,
row.names = TRUE)
{
n <- length(row.names(x))
if (length(x) == 0L) {
cat(sprintf(ngettext(n, "data frame with 0 columns and %d row",
"data frame with 0 columns and %d rows"), n), "\n",
sep = "")
}
else if (n == 0L) {
print.default(names(x), quote = FALSE)
cat(gettext("<0 rows> (or 0-length row.names)\n"))
}
else {
x <- as.data.frame(head(x, n = options("max.print")))
m <- as.matrix(format.data.frame(x, digits = digits,
na.encode = FALSE))
if (!isTRUE(row.names))
dimnames(m)[[1L]] <- if (isFALSE(row.names))
rep.int("", n)
else row.names
print(m, ..., quote = quote, right = right)
}
invisible(x)
}
Thank you.
Best,
Juan
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] CRAN: R 3.5.0 Not Available
In CRAN, R 3.5.0 is not available for download on "Previous Releases of R for Windows". ¿Could it be added? https://cran.r-project.org/ Thanks. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Quartz graphic device can be extremely slow in some cases
It might be related with this files: https://github.com/wch/r-source/blob/trunk/src/library/grDevices/src/devQuartz.c https://github.com/wch/r-source/blob/trunk/src/include/R_ext/QuartzDevice.h Could we port some of the UNIX optimizations to previous code? https://github.com/wch/r-source/blob/trunk/src/library/grDevices/R/unix/quartz.R __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Reminder: R Project Contributed Documentation Still Frozen
Dear R Developers, Just I would like to remind that R Contributed Documentation is still frozen: https://cran.r-project.org/other-docs.html It would certainly be nice to have an official R Project Contributed Documentation Site where we can all contribute :) Kind regards, Juan Telleria *P.D.: I have already finished editing my first R Machine Learning Cheatsheet, and hope to publish it to RStudio's Contributed Cheatsheet Section in short time :) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R Lapack – why a subset?
> Is the cost really so high as to preclude adding the remaining Lapack > routines to Rlapack? Updating Lapack Libraries shall not break compatibility, and rather provide bug fixes I guess. > the reason is that there would be a significant extra maintenance burden > consisting of things that is not being used by R itself. I agree with Keith O'Hara when she says that *"R is often used as a front-end for libraries that do, so Rlapack-related linking errors are arising more and more"*. Maybe we could propose R-Core some SVN patches by ourselves for R-alpha that update LAPLACK libraries and see how it looks like... :S > http://www.netlib.org/lapack/archives/ > https://github.com/wch/r-source/tree/trunk/src/modules/lapack And in SVN: > svn checkout https://svn.r-project.org/R/trunk/ R-devel > svn update > svn diff > patch.diff __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] 64-bit integer type warning on windows
It does not answer direcly your question, but have you tried "bit64" CRAN package :) https://cran.r-project.org/web/packages/bit64/index.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
I attach the Github Flow for teams and projects with regular deployments: https://guides.github.com/pdfs/githubflow-online.pdf https://guides.github.com/introduction/flow/ Tips: * Always Do pull requests based on branches (never on the master). * Keep your Fork Synchronized with the Upstream Repository. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
> So I created a branch within my fork, and committed the change there. But > Github provides no way to create a pull request that only includes the new > stuff! > Every attempt I made would have included everything from both bug fixes. I have been doing some tests, and I think that this issue can be easily addressed with Bitbucket GUI (https://bitbucket.org/product), which is free for Open Source Projects (https://www.atlassian.com/software/views/open-source-license-request) and seems easy to use (So it follows the K.I.S.S. principle). Just another alternative instead of Gitlab for self-hosting... no worries. Best, Juan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
Yes, indeed Gitlab GUI Core Code is Open Source (Libre / Community Edition): https://gitlab.com/gitlab-org/gitlab-ce > But his instructions required command-line git, and my main claim is that > Github is not good enough to do the kinds of things I want to do and R Core > would need to do. > > My other claim is that git is too hard to use. I'm sure that Git Command Line Recipe Documentation can solve this issue, Gitlab, in particular, has a wiki in which this kind of issues could be documented. Also Git cheat-sheets might prove useful. In addition, any feature request could be done in Gitlab Issue Section (See above), or if that does not still does not convince, other options could be considered, such as Bitbucket (https://bitbucket.org/), etc. In addition, the Git Repository: * Could be self-hosted in the University Servers (Just as SVN actually is nowadays). * Be accessed either by the Command Line or the Graphical User Interface (As users prefer). The main reason motivating the move to the GIT Repository, as said before, is that it would to allow individual users or companies from the R Consortium to do pull requests based on issues for improving base R code. Indeed, in some years from now I would like to help to improve base R myself, maybe re-writing some parts of the code in C++, fixing bugs, or who knows :) Kind regards, Juan Telleria __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
In case it's useful: A) Git Cheatsheet: ADVANCED (GitHub) https://www.google.es/url?sa=t&source=web&rct=j&url=https://services.github.com/on-demand/downloads/github-git-cheat-sheet.pdf&ved=2ahUKEwjkhYmdt_bYAhXIwBQKHXWdBfoQFjAAegQIERAB&usg=AOvVaw3RoN21aynhcDHqKncV31el B) Git Cheatsheet: BASICS (GitHub) https://www.google.es/url?sa=t&source=web&rct=j&url=https://education.github.com/git-cheat-sheet-education.pdf&ved=2ahUKEwjkhYmdt_bYAhXIwBQKHXWdBfoQFjABegQIEhAB&usg=AOvVaw2D3W2R0fwoOBi8YrhZYLFJ C) Git Cheatsheet (Atlassian) https://www.google.es/url?sa=t&source=web&rct=j&url=https://www.atlassian.com/dam/jcr:8132028b-024f-4b6b-953e-e68fcce0c5fa/atlassian-git-cheatsheet.pdf&ved=2ahUKEwjkhYmdt_bYAhXIwBQKHXWdBfoQFjACegQIExAB&usg=AOvVaw2TdLUeHteS2YU_G9u_-6nP [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
I do not mind investing as much time as necessary :-) > If you ever need to document issues / coding recipes related to GIT / SVN: > > * I could pick the commands from e-mails. > * Any documentation you send me. > * Or books (http://shop.oreilly.com/product/0636920022862.do), web pages, etc.. > > And create a wiki / documentation page in any platform, in order to help. > > Best, > Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why R should never move to git
If you ever need to document issues / coding recipes related to GIT / SVN: * I could pick the commands from e-mails. * Any documentation you send me. * Or books (http://shop.oreilly.com/product/0636920022862.do), web pages, etc.. And create a wiki / documentation page in any platform, in order to help. Best, Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
Dear R Developers, I attach a complete summary of all the Know-How that has been generated during Dec 2017 & Jan 2018 in R-devel Mailing List related to the Dynamic Change of: - Regular Single Threaded BLAS. - And a Multi-threaded BLAS Implementation (e.g.: OpenBLAS): https://kbrproject.miraheze.org/wiki/BLAS_Libraries Kind regards, Juan Telleria [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
> In order to document the generated Know-How of Optional BLAS Libraries > Implementation and Tests (For example: OpenBLAS), I have created a > Mediawiki based wiki page in which anyone can document and discuss any > issues he/she encounters: > > https://kbrproject.miraheze.org/wiki/Main_Page/BLAS > > I will myself document all observations and attach the papers that > have been mentioned in R-devel related to such topic. Will take me some time... Maybe I can finish it at the weekend, That there is more info that expected. Juan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
In order to document the generated Know-How of Optional BLAS Libraries Implementation and Tests (For example: OpenBLAS), I have created a Mediawiki based wiki page in which anyone can document and discuss any issues he/she encounters: https://kbrproject.miraheze.org/wiki/Main_Page/BLAS I will myself document all observations and attach the papers that have been mentioned in R-devel related to such topic. Hope its useful. Best, Juan Telleria __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Community Feedback: Git Repository for R-Devel
# R-devel Archives: “Why R-project source code is not on Github" [Summary: Aug 2014] Key Citations (Pros and Cons) from R-devel Archives # GIT PROS 1. [Simon Urbanek R-devel Aug 21, 2014] Github just makes it much easier to create and post patches to the project - it has nothing to do with write access - typically on Github the community has no write access, either. 2. [Simon Urbanek R-devel Aug 21, 2014] Using pull requests is certainly much less fragile than e-mails and patches are based on forked branches, so you can directly build the patched version if you want without manually applying the patch - and you see the whole history so you can pick out things logically. 3. [Simon Urbanek R-devel Aug 21, 2014] You can comment on individual patches to discuss them and even individual commits - often leading to a quick round trip time of revising it. 4. [Yihui Xie-2 R-devel Aug 22, 2014] Sometimes the patches are not worth emails back and forth, such as the correction of typos. I cannot think of anything else that is more efficient than being able to discuss the patch right in the lines of diff's. 5. [Gaurav Sehrawat R-devel Aug 24, 2014] Bridging gap between web2.0 and web1.0 development methodologies & thus passing code to younger generation . 6. [Jeroen Ooms. R-devel Aug 24, 2014] By now all activity of r-base [1] cran [2] and r-forge [3] is continuously mirrored on Github, which already gives unprecedented insight in developments. At least several r-core members [4,5,6,7,8] have been spotted on Github, and this years useR2014 website [9] was developed and hosted completely on Github. It seems like a matter of time until the benefits outweigh the cost of a migration, even to the more conservative stakeholders. 7. [Spencer Graves-2 R-devel Aug 24, 2014] We could use Git without Github (Gitlab, …) 8. [Spencer Graves-2 R-devel Aug 24, 2014] It should be easy and cheap for someone to program a server to make daily backup copies of whatever we want from Github. This could provide an insurance policy in case events push the group to leave Github. 9. [Brian Rowe R-devel Aug 24, 2014] One thing to note about git vs svn is that each git repository is a complete repository containing the full history, so despite github acting as a central repository, it is not the same as a central svn repository. In svn the central repository is typically the only repository with a complete revision history, but that is not the case with git. 10. [Simon Urbanek R-devel Aug 25, 2014] There is no point in using git alone (Github actually supports direct SVN access as well). 11. [Simon Urbanek R-devel Aug 25, 2014] Github: The whole point are the collaborative features. # GIT CONS 1. [Marc Schwartz R-devel Aug 21, 2014] Since the current SVN based system works well for them (R Core) and provides restricted write access that they can control, there is no motivation to move to an alternative version control system unless they would find it to be superior for their own development processes. 2. [Jeroen Ooms. R-devel Aug 24, 2014] These things take time 3. [Jeroen Ooms. R-devel Aug 24, 2014] However moving development of a medium sized, 20 year old open source project is not trivial. You are dealing with a large commit history and many contributors that all have to overhaul their familiar tools and development practices overnight. 4. [Jeroen Ooms. R-devel Aug 24, 2014] There is also the infrastructure of nightly builds and CRAN r-devel package checking that relies on the svn. 5. [Jeroen Ooms. R-devel Aug 24, 2014] Moreover moving to Github means changes in communications, for example replacing the current bug tracking system to Github "issues". 6. [Jeroen Ooms. R-devel Aug 24, 2014] In addition, several members are skeptical about putting source code in the hands of a for-profit US company, and other legal issues. 7. [Jeroen Ooms. R-devel Aug 24, 2014] The most critical piece of making such a transition is a detailed proposal outlining what the migration would involve, the cost/benefits, a planning, and someone that is willing to take the lead. Only on the basis of such a serious proposal you can have a discus
Re: [Rd] Community Feedback: Git Repository for R-Devel
I attach a basic State of Art: ## # State of Art Analysis of Git vs SVN ## Scopus Keywords: GIT AND SVN ## # 1. How Do Centralized (SVN) and Distributed Version Control (GIT) Systems Impact Software Changes? (22 Citations; Published: 2014) ## 1.1 Paper Conclusions We found that the use of CVCS and DVCS have observable effects on developers, teams and processes. The most surprising findings are that (i) the size of commits in DVCS was smaller than in CVCS, (ii) developers split commits (group changes by intent) more often in DVCS, and (iii) DVCS commits are more likely to reference issue tracking labels. These show that DVCS contain higher quality commits compared to CVCS due to their smaller size, cohesive changes and the presence of issue tracking labels. The survey provided valuable information on why developers prefer one paradigm versus the other. DVCS are preferred because of killer features, such as the ability of committing locally. In contrast CVCS are preferred for their ease of use and faster learning curve. 1.2 Full Paper http://dig.cs.illinois.edu/papers/ICSE14_Caius.pdf ## # 2 Version Control with Git (Book: J Loeliger, M McCullough – 2012) ## 2.1 Book Introduction ***The Birth of Git*** Often, when there is discord between a tool and a project, the developers simply create a new tool. Indeed, in the world of software, the temptation to create new tools can be deceptively easy and inviting. In the face of many existing version control systems, the decision to create another shouldn’t be made casually. However, given a critical need, a bit of insight, and a healthy dose of motivation, forging a new tool can be exactly the right course. Git, affectionately termed “the information manager from hell” by its creator is such a tool. Although the precise circumstances and timing of its genesis are shrouded in political wrangling within the Linux Kernel community, there is no doubt that what came from that fire is a well-engineered version control system capable of supporting worldwide development of software on a large scale. Prior to Git, the Linux Kernel was developed using the commercial BitKeeper VCS, which provided sophisticated operations not available in then-current, free software version control systems such as RCS and CVS. However, when the company that owned BitKeeper placed additional restrictions on its “free as in beer” version in the spring of 2005, the Linux community realized that BitKeeper was no longer a viable solution. Linus looked for alternatives. Eschewing commercial solutions, he studied the free software packages but found the same limitations and flaws that led him to reject them previously. What was wrong with the existing VCS systems? What were the elusive missing features or characteristics that Linus wanted and couldn’t find? ***Facilitate distributed development*** There are many facets to “distributed development,” and Linus wanted a new VCS that would cover most of them. It had to allow parallel as well as independent and simultaneous development in private repositories without the need for constant synchronization with a central repository, which could form a development bottleneck. It had to allow multiple developers in multiple locations even if some of them were offline temporarily. ***Scale to handle thousands of developers*** It isn’t enough just to have a distributed development model. Linus knew that thousands of developers contribute to each Linux release, so any new VCS had to handle a very large number of developers, whether they were working on the same or on different parts of a common project. And the new VCS had to be able to integrate all of their work reliably. ***Perform quickly and efficiently*** Linus was determined to ensure that a new VCS was fast and efficient. In order to support the sheer volume of update operations that would be made on the Linux Kernel alone, he knew that both individual update operations and network transfer operations would have to be very fast. To save space and thus transfer time, compression and “delta” techniques would be needed. Using a distributed model instead of a centralized model also ensure
Re: [Rd] R CMD check warning about compiler warning flags
I repeat it for all the reason I gave to Duncan on a personal E-mail, It is a lot of text, and I might be wrong, but I attach it in case it is useful: A) I feel Version Control (SVN) is great when there is a little group of people maintaining the code (RCore ~ 20); but Git could allow a bigger group of people (for example ~200) working simultaneously on perfect and polish the code without overlapping each other, and RCore (~20) check that code, and do commits. This would allow an even more robust base language foundation, or improvements such as writing some base code more efficiently in C++, etc. I will put a SQL Transactions simile for further explain this issue: * SVN is like a Table-Lock. * And GIT is a Row-Lock in the Table. Table-Lock works well with very few connections, and as there are only few connections, users are happy with it. But it prevents other client connections to query that table at the same time, and does not allow high concurrency. B) I feel that Version Control is a little bit obscure for people who are not from R-Core Mailing List, as: * We cannot see what gets updated with each R-devel specific version. * And have communication tools flexible enough to suggest a little patch (changes in a few lines of code all scattered over R Code) by ourselves other than E-mails. ¿What if our fixes are very sparse? Communication and Patches for Collaboration are difficult and tedious for Non-Core Contributors. C) Reverting a whole version can cause that bugs that where already fixed, to be reverted (I've seen it once in R). D) Making a whole new version for a single change in 1 line of code is inefficient, and time consuming. In GitHub it would take few seconds. E) Google Trends indicates that GIT is getting more and more used than SVN by other developments. If it works for others, why should not work for R? F) It would also be important to know what the opinion of key R contributors outside R-Core is, for example: * Hadley Wickham. * Other important R Package Authors. If they would feel more willing to help on R Core Language Development with GitHub, Gitlab, etc. Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Community Feedback: Git Repository for R-Devel
Thank you Mark, this is what I was looking for. On Sunday I will read again in detail previous discussion's facts, and attach the pros and cons here, so that they remain for the future, and the topic can be closed. Juan El 4 ene. 2018 11:06 a. m., "Mark van der Loo" escribió: > This question has been discussed before on this list: > http://r.789695.n4.nabble.com/Why-R-project-source-code-is- > not-on-Github-td4695779.html > > See especially Jeroen's answer. > > Best, > Mark > > Op do 4 jan. 2018 om 01:11 schreef Juan Telleria : > >> UNBIASED FACTS: >> • Bugzilla & R-devel Mailing Lists: Remain unchanged: Understood as >> Ticketing platforms for bug pull requests on the R-devel Git Repository. >> >> • Git Repository Options: >> A) Github (Cloud with Automated backups from GitHub to CRAN Server): >> https://github.com >> B) Gitlab (Selfhosted on CRAN): https://about.gitlab.com >> C) Phabricator (Selfhosted on CRAN): https://www.phacility.com >> D) Microsoft Codeplex: https://www.codeplex.com >> E) Others: Unknown >> >> GOOGLE TRENDS: >> https://trends.google.com/trends/explore?date=all&q=Git,Svn,Github,Gitlab >> >> EXAMPLE >> Git Repository on Core Python: https://github.com/python >> >> PERSONAL OPINION / MOTIVATION: >> I think that moving efforts in this direction is important because it >> would >> allow a true Open Source Innovation & Open Collaboration in R between: >> * R Community. >> * And R-Core. >> For: >> * R Bug Fixes. >> * And Core Feature Wishlist. >> As anyone would be able to: >> * Check the unassigned bugs in Bugzilla (apart from R-Core). >> * And propose bugs fixes by themselves as Pull requests (by mentioning the >> Bug ID of Bugzilla or the Mailing Lists). >> >> This would allow that _individuals_ either from Universities or Companies >> interested in the Development of R: >> * apart of donating economical resources to the R Foundation. >> * could help to maintain core R Code by themselves. >> Which aligns with the true spirit of R, which shall be done from >> contributing individuals, for individuals themselves. >> >> It would also allow to put the focus on the precise lines of code changed >> with each Commit, and revert changes in an easy way, without verbose >> E-mails: Tidy, Clean, Maintainable, and Fast. >> >> At last, I noticed R-devel Archives do not have an E-mail Id (Unique >> Unsigned Integer), so it would be a good idea to add one for pull requests >> if Git was adopted. >> >> Juan >> >> [[alternative HTML version deleted]] >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Community Feedback: Git Repository for R-Devel
UNBIASED FACTS: • Bugzilla & R-devel Mailing Lists: Remain unchanged: Understood as Ticketing platforms for bug pull requests on the R-devel Git Repository. • Git Repository Options: A) Github (Cloud with Automated backups from GitHub to CRAN Server): https://github.com B) Gitlab (Selfhosted on CRAN): https://about.gitlab.com C) Phabricator (Selfhosted on CRAN): https://www.phacility.com D) Microsoft Codeplex: https://www.codeplex.com E) Others: Unknown GOOGLE TRENDS: https://trends.google.com/trends/explore?date=all&q=Git,Svn,Github,Gitlab EXAMPLE Git Repository on Core Python: https://github.com/python PERSONAL OPINION / MOTIVATION: I think that moving efforts in this direction is important because it would allow a true Open Source Innovation & Open Collaboration in R between: * R Community. * And R-Core. For: * R Bug Fixes. * And Core Feature Wishlist. As anyone would be able to: * Check the unassigned bugs in Bugzilla (apart from R-Core). * And propose bugs fixes by themselves as Pull requests (by mentioning the Bug ID of Bugzilla or the Mailing Lists). This would allow that _individuals_ either from Universities or Companies interested in the Development of R: * apart of donating economical resources to the R Foundation. * could help to maintain core R Code by themselves. Which aligns with the true spirit of R, which shall be done from contributing individuals, for individuals themselves. It would also allow to put the focus on the precise lines of code changed with each Commit, and revert changes in an easy way, without verbose E-mails: Tidy, Clean, Maintainable, and Fast. At last, I noticed R-devel Archives do not have an E-mail Id (Unique Unsigned Integer), so it would be a good idea to add one for pull requests if Git was adopted. Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R CMD check warning about compiler warning flags
However, and hope not to be off-topic, a git repository (github, gitlab, codeplex, etc., not just solely github) could constitute a tidy approach, and make things easier to R Core :) By putting the focus on version control, the line of changes made with each commit (With the possibility to reverse changes), and not verbose e-mails. Juan I strongly disagree. Are you aware that github is a commercial >> company, github inc. [1] ? >> What about gitlab? or Microsoft's codeplex? There are other services >> similar to github, why github? >> What happens if github goes out of business? >> >> R-project should be maintained in the academic network and under >> auspices of universities. >> >> >> [*] GitHub, Inc. >>https://en.wikipedia.org/wiki/GitHub >> > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R CMD check warning about compiler warning flags
M... I see... thank you Suzen. Juan > I strongly disagree. Are you aware that github is a commercial > company, github inc. [1] ? > What about gitlab? or Microsoft's codeplex? There are other services > similar to github, why github? > What happens if github goes out of business? > > R-project should be maintained in the academic network and under > auspices of universities. > > > [*] GitHub, Inc. >https://en.wikipedia.org/wiki/GitHub > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Collaborative Wiki: Mediawiki
Dear R Developers, I create this e-mail, linked to "Collaborative Wiki for fostering Open Innovation in R", for going straight into the point: After studing multiple wikies (Confluence, xwiki, Mediawiki) for a collaborative approach to R Documentation, I finally came up that Mediawiki was the best for managing documentation for being: - 100% Open Source. - Simple, tidy, neat interface. - Documentation Focused (no forums, no other things). - Being the one Wikipedia usses is really an assurance that it will never go away, and each second spent on it is worthy. - Anyone can collaborate, update files up to 40 MB, but there is a strict version control on the wiki :), just as wikipedia has. So I requested on Miraheze (https://meta.miraheze.org/wiki/Miraheze) Community Driven Wiki Farms that they set up a wiki as a *Sandbox for testing* (Not for real use in production): https://kbrproject.miraheze.org/wiki/Main_Page You can test it an play with it whatever you want; and assess if it could constitute a good option for managing contributed R documentation. Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R CMD check warning about compiler warning flags
Maybe I'm new, and forgive my ignorance, but maybe in the future (~ X years from now) the R Project could be managed entirely from github, by doing pull requests and only R Core having commit rights... Would make the forking process also easier... And could be a good roadmap. But we're not using git, we're just sending emails back and forth. I don't > need to run svn to get some useful information from the number 73909. > Winston's web page at https://github.com/wch/r-source/commit/2e80059 does > display the number 73909 if you know where to look, but his email doesn't > contain it. > > I don't know about other svn users, but I'd be perfectly content to read > something like "This appears to have been added in rev 73909 (diff shown > here: https://github.com/wch/r-source/commit/2e80059)." > > Duncan Murdoch [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Wish List: base::source() + Add Execution Time Argument
I did not know "progress" package existed, thank you Iñaki. However, something like that would be nice to have by default in source(), just something to add to R's "wish list", so that everybody can benefit from it without extra-packages, as most of us I suppose we will spend all day simply doing Ctrl + Run :) Thank you, Juan 2017-12-21 15:20 GMT+01:00 Iñaki Úcar : > 2017-12-21 15:05 GMT+01:00 Juan Telleria : > > But by statement in the source file, I mean, for knowing during the > > execution how much time is taking, without having to wait till the end. > > What's the ultimate purpose? Are you looking for a profiler (there are > some of them on CRAN) or some kind of progress bar (something like the > 'progress' package would be useful)? > > Iñaki > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Wish List: base::source() + Add Execution Time Argument
But by statement in the source file, I mean, for knowing during the execution how much time is taking, without having to wait till the end. 2017-12-21 13:06 GMT+01:00 Iñaki Úcar : > 2017-12-21 12:46 GMT+01:00 Juan Telleria : > > Dear R Developers, > > > > Adding to source() base function a Timer which indicates the execution > time > > of the source code would be a very well welcome feature, and in my > opinion > > not difficult to implement as an additional funtion argument. > > > > The source(timing = TRUE) function shall execute internally the following > > code for each statement: > > > > old <- Sys.time() # get start time at the beginning of source() > > # source code > > # print elapsed time > > new <- Sys.time() - old # calculate difference > > print(new) # print in nice format > > system.time(source(...)) does what you want. > > Iñaki > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Wish List: base::source() + Add Execution Time Argument
Dear R Developers, Adding to source() base function a Timer which indicates the execution time of the source code would be a very well welcome feature, and in my opinion not difficult to implement as an additional funtion argument. The source(timing = TRUE) function shall execute internally the following code for each statement: old <- Sys.time() # get start time at the beginning of source() # source code # print elapsed time new <- Sys.time() - old # calculate difference print(new) # print in nice format Thank you. Kind regards, Juan Telleria [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Collaborative Wiki for fostering Open Innovation in R
I have found Apache JSPWiki solution, which might work and is open source: https://jspwiki.apache.org In addition, I will test xwiki, and give my insights about the tool on r-devel, as it might just be a resolutive tool for collaboration thanks to it's extensible architecture. Kind regards, Juan Telleria [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Collaborative Wiki for fostering Open Innovation in R
Dear R Developers, I am writing in order to open a constructive debate whether if it is worth it that R Project adopts the Apache Software Foundation & Wikipedia Model for Open Documentation by using a Collaborative Wiki & Document Store (As a complement to Bugzilla and Mailing Lists), for fostering collaboration in R Development. The objective would be to promote innovation and collaboration between: • R Contributors. • R Core. • And R Community. For: • R Development. • R Contributed Documentation. • R Internals Wiki. The Wikis Section could be structured in different Sub-Wikis (or Pages) with different levels of permissions: • Some only editable by R Core. • Others only editable by R Core, and R Contributors. • And others editable by any member of R Community. And have different Categories, such as: • R Learning Resources. • R Cheatsheets. • R Development & Key Internals. • R Contributed Documentation. • R Help. • etc. A key point would be that the wikis are installed, hosted and accesible through CRAN Servers, as a way to give some collaborative GUI to CRAN hosted documentation. Three collaborative wikis worth to consider would be: • xwiki SAS (Open Source): http://www.xwiki.org • Atlassian Confluence (Free for Open Source Projects): https://www.atlassian.com/software/confluence • MediaWiki: www.mediawiki.org I personally prefer xwiki for being 100% Open Source, and being editable with a Markdown Syntax; but we use Confluence at work and I am also very happy with it; and MediaWiki is the one Wikipedia uses, but seems difficult to use. Thank you & best, Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
Julia Programming Language uses also OpenBlas, and it is actively maintained with bugs being fixed as I have checked it out: http://www.openblas.net/Changelog.txt So I still see it ok to be included as an options(...) feature (by default off, just for safety), over other Blas libraries. R could not use Intel MKL for legal reasons (I think), because as long that R ships with GPL libraries, shipping R by default with Non-GPL is illegal. Cheers, Juan El 17/12/2017 2:50 a. m., "Avraham Adler" escribió: > On Sat, Dec 16, 2017 at 7:41 PM, Kenny Bell wrote: > > It seems like many of the multi-threaded BLASes have some sort of > > fundamental problem preventing use in the way Juan suggests: > > > > - Dirk's vignette states that ATLAS "fixes the number of cores used at > > compile-time and cannot vary this setting at run-time", so any > > user-friendly implementation for R would have to compile ATLAS for 1-16 > > threads to allow the user to switch at run-time. This might dramatically > > affect install times. > > > > - MKL seems like it's been outright rejected in the past based on not > > being "free-enough". > > > > - OpenBLAS causes crashes. > > > > Has anyone tried ExBLAS for use with R? > > > > On Sun, Dec 17, 2017 at 1:03 PM, Peter Langfelder < > > peter.langfel...@gmail.com> wrote: > > > >> I would be very cautious about OpenBLAS in particular... from time to > >> time I get complains from users that compiled code calculations in my > >> WGCNA package crash or produce wrong answers with large data, and they > >> all come from OpenBLAS users. I am yet to reproduce any of their > >> crashes when using MKL and ATLAS BLAS implementations. > >> > >> Just my 2 cents... > >> > >> Peter > > I've been building R on Windows 64 bit with OpenBLAS for years and my > builds pass check-devel. For a while in the past it failed one check > as the tolerance was 5e-5 and with my build of OpenBLAS the error was > 5.4e-5 or 5.7e-5, but that was changed around R 3.3, if I recall > correctly. I provide descriptions here [1], but I haven't gone so far > as to post compiled Rblas.dlls just yet. My personal build sets 4 > threads when compiling OpenBLAS itself as I'm currently on a quad-core > SandyBridge. In tests I ran a few years ago, both single and multi > threaded BLAS compile and then can be compiled into R with no issues > (on my platforms, at least). Most matrix operations performed better > with multi-threaded except for R's eigenvalue decomposition, to the > nest of my recollection. > > Avi > > [1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
Multi-threaded Math Libraries (Trough OpenBlas), taking into account that today's laptops have a minimum of 2-4 cores, are an important topic, and in my opinion, shall be included in R for the general interest. I think that the way to go would be to create a configuration setting in R's options(OpenBlas=TRUE|FALSE) which enables or disables OpenBlas in an easy way, which by default shall be in FALSE (In order to avoid issues with parallel libraries). The key point would be that each time you open a new R session, a 1 liner informative comment arises that tells you: a) Whether OpenBlas is enabled or disabled. b) And how many cores it uses (Setting also configurable through options(...)) In a shape just as Microsoft R Open does. Kind regards, Juan Telleria El 17/12/2017 12:31 a. m., "Juan Telleria" escribió: > Multi-threaded Math Libraries (Trough OpenBlas), taking into account that > today's laptops have a minimum of 2-4 cores, are an important topic, and in > my opinion, shall be included in R for the general interest. > > I think that the way to go would be to create a configuration setting in > R's options(OpenBlas= TRUE|FALSE) which enables or disables OpenBlas in an > easy way, which by default shall be in FALSE (In order to avoid issues with > parallel libraries). > > The key point would be that each time you open a new R session, a 1 liner > informative comment arises that tells you: > a) Whether OpenBlas is enabled or disabled. > b) And how many cores it uses (Setting also configurable through > options(...)) > > In a shape just as Microsoft R Open does. > > Kind regards, > Juan Telleria > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] OpenBLAS in everyday R?
Multi-threaded Math Libraries (Trough OpenBlas), taking into account that today's laptops have a minimum of 2-4 cores, are an important topic, and in my opinion, shall be included in R for the general interest. I think that the way to go would be to create a configuration setting in R's options(OpenBlas= TRUE|FALSE) which enables or disables OpenBlas in an easy way, which by default shall be in FALSE (In order to avoid issues with parallel libraries). The key point would be that each time you open a new R session, a 1 liner informative comment arises that tells you: a) Whether OpenBlas is enabled or disabled. b) And how many cores it uses (Setting also configurable through options(...)) In a shape just as Microsoft R Open does. Kind regards, Juan Telleria [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?
OpenBlas seems to have a performance similar to Intel MKL, as is stated in
Wikipedia:
https://en.m.wikipedia.org/wiki/OpenBLAS
However, I would suggest debating this topic in the R Consortium and the R
Foundation, for taking the best decision possible for the future or R.
Thank you all,
Juan
El 31/10/2017 2:55 p. m., "Iñaki Úcar" escribió:
2017-10-31 14:34 GMT+01:00 Juan Telleria :
> So as long as I can read, OpenBlas, for Windows, might be a worth
> considering option:
> http://www.openblas.net
>
> But Intel MKL also seems to be free*:
> https://software.intel.com/en-us/articles/free-mkl
install.packages("rmsfact")
sub(".*because ", "", rmsfact::rmsfact(8))
Iñaki
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?
So as long as I can read, OpenBlas, for Windows, might be a worth considering option: http://www.openblas.net But Intel MKL also seems to be free*: https://software.intel.com/en-us/articles/free-mkl Thank you, Juan El 30/10/2017 6:45 p. m., "Avraham Adler" escribió: > [Sent offlist accidentally] > > What concerns me first and foremost is that the licensure would have > to be ironclad (including for commercial use like vanilla R now) as > well as ensuring that R remains completely FLOSS. Anything “added” to > R has to be a no-strings-attached gift to R. > > Also, I would think that it would have to play nice with existing > workflows (like OpenBLAS instead of MKL) unless there is such a > benefit that it is worth breaking compatibility. > > Avi > > On Sun, Oct 29, 2017 at 6:01 PM, Kenny Bell wrote: > > User here: incorporating Intel's MKL, as MRO does, would be a very > welcome > > addition. > > > > I was an MRO user before and it improved my experience with medium data > > immensely. > > > > They did, however, leave behind bugs here and there, especially related > to > > development with Rcpp, so I switched back to vanilla R. > > > > On Mon, Oct 30, 2017, 9:42 AM Juan Telleria > wrote: > > > >> Dear R Developers, > >> > >> First of all, I would like to thank you Jeroen Ooms for taking the > binary > >> Window Builds from Duncan. I firmly believe that the R Community will > >> benefit a lot from his work. > >> > >> However, the debate I would like to open is about if some of Microsoft R > >> Open Code shall be ported from R Open to Mainstream R. > >> > >> There are some beneficts in R Open such as multithreaded performance: > >> https://mran.microsoft.com/documents/rro/multithread/ > >> > >> Maybe, the R Consortium, and in particular, Microsoft R Team, could > >> collaborate, if appropriate, in such duty. > >> > >> Thank you, > >> Juan Telleria > >> > >> [[alternative HTML version deleted]] > >> > >> __ > >> R-devel@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-devel > >> > > > > [[alternative HTML version deleted]] > > > > __ > > R-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Debate: Shall some of Microsoft R Open Code be ported to mainstream R?
Dear R Developers, First of all, I would like to thank you Jeroen Ooms for taking the binary Window Builds from Duncan. I firmly believe that the R Community will benefit a lot from his work. However, the debate I would like to open is about if some of Microsoft R Open Code shall be ported from R Open to Mainstream R. There are some beneficts in R Open such as multithreaded performance: https://mran.microsoft.com/documents/rro/multithread/ Maybe, the R Consortium, and in particular, Microsoft R Team, could collaborate, if appropriate, in such duty. Thank you, Juan Telleria [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] source() + dbSendQuery() Returns Incorrect Results - Solved with: dbExecute()
Issue Description and Expected Result When used source() with dbSendQuery(), wrong results are obtained from the Database, probably due to a precision loose at some point. Database MariaDB Solution It was solved by means of using source() + dbExecute(), instead of dbSendQuery(). Question Although not Standard, ¿Shall dbSendQuery() default behavior be modified to avoid this issue?. [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Duncan's retirement: who's taking over Rtools?
The Apache Foundation has a whole article stating that open source projects shall be managed and used independently of any commercial interest: https://community.apache.org/projectIndependence.html So Microsoft could donate, if interested, money or other resources for such specific role, but always taking special care of company interest independence. Juan El 29/9/2017 2:23 p. m., "Juan Telleria" escribió: > I agree with Moshe. > > It is important to maintain the independence of R as a programming > language by itselft, even if it could benefit from Microsoft work (C++ Base > Code, etc.), it is better in my opinion to keep it independent. > > Also, Duncan work and know-how shall be transferred to the next future R > Core Developer which will be in charge of Duncan's roles. And, if > appropriate, transfer the scripts Duncan might use, the documentation, etc. > > Thank you, > Juan > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Duncan's retirement: who's taking over Rtools?
I agree with Moshe. It is important to maintain the independence of R as a programming language by itselft, even if it could benefit from Microsoft work (C++ Base Code, etc.), it is better in my opinion to keep it independent. Also, Duncan work and know-how shall be transferred to the next future R Core Developer which will be in charge of Duncan's roles. And, if appropriate, transfer the scripts Duncan might use, the documentation, etc. Thank you, Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R Configuration Variable: Maximum Memory Allocation per R Instance
Very very interesting, if it is ok with it, I will post these observations to Stack Overflow so that they are useful to other R Programmers, and make more research on the topic, putting it all together. I could even do a small article with my research. I think this is a critical point if you want to have an R Instance and a RDBMS in the same Server. Thank you, Juan 2017-09-17 14:14 GMT+02:00 Jeroen Ooms : > On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria > wrote: > > Dear R Developers, > > > > In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have > the > > innodb_buffer_pool_size or the key_buffer_size for setting the maximum > > amount of RAM which can be used by a Server Instance. > > Memory is not controlled by R itself because packages may malloc() > directly. However most operating systems have features to limit > resources of a given process. The CRAN package 'unix' has wrappers for > posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This > works pretty well, however I found that the way memory is managed and > counted varies a lot per OS and malloc implementation. > > You can also set rlimits on a single evaluation via the rlimit > parameter in sys::eval_safe(). > > > [1] http://pubs.opengroup.org/onlinepubs/009695399/ > functions/getrlimit.html > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R Configuration Variable: Maximum Memory Allocation per R Instance
This variables already exist as I have been said, and are: * memory.size * memory.limit R Documentation: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/memory.size.html Thank you, Juan El 17/9/2017 12:39 a. m., "Juan Telleria" escribió: > Dear R Developers, > > In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the > innodb_buffer_pool_size or the key_buffer_size for setting the maximum > amount of RAM which can be used by a Server Instance: > > ¿Would it be possible to create an R Configuration Variable which fixes > the maximum amount of RAM memory to be used as Commit / Dynamic Memory > Allocation? > > Thank you. > Juan > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R Configuration Variable: Maximum Memory Allocation per R Instance
Dear R Developers, In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the innodb_buffer_pool_size or the key_buffer_size for setting the maximum amount of RAM which can be used by a Server Instance: ¿Would it be possible to create an R Configuration Variable which fixes the maximum amount of RAM memory to be used as Commit / Dynamic Memory Allocation? Thank you. Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Suggestion: Create On-Disk Dataframes
Dear R Developers, I would like to suggest the creation of a new S4 object class for On-Disk data.frames which do not fit in RAM memory, which could be called disk.data.frame() It could be based in rsqlite for example (By translating R syntax to SQL syntax for example), and the syntax and way of working of the disk.data.frame() class could be exactly the same than with data.frame objects. When the session is of, is such disk.data.frames are not saved, and implicit DROP TABLE could be done in all the schemas created in rsqlite. Nowadays, with the SSD disk drives such new data.frame() class could have sense, specially when dealing with Big Data. It is true that this new class might be slower than regular data.frame, data.table or tibble classes, but we would be able to handle much more data, even if it is at cost of speed. Also with data sampling, and use of a regular odbc connection we could do all the work, but for people who do not know how to use RDBMS or specific purpose R packages for this job, this could work. Another option would be to base this new S4 class on feather files, but maybe making it with rsqlite is simply easier. A GitHub project could be created for such purpose, so that all the community can contribute (included me :D ). Thank you, Juan [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
