[R-pkg-devel] RIOT 2020: canceled
After carefully considering the current situation regarding COVID-19, we decided to cancel this year's RIOT workshop. Best regards, Stepan On 25. 02. 20 17:47, Stepan wrote: I hope you don’t mind us using this mailing list for a small advertisement, but we think it is relevant for this group: We'd like to invite you to RIOT 2020 - the 5rd workshop on R Implementation, Optimization and Tooling [1]. It will take place co-located with, and during, useR! 2020 in St. Louis on July 8th. RIOT is an excellent venue for deep technical discussions about R implementations, tools, optimizations and R extensions, and will be very interesting for anyone interested in what’s under the hood of R. The RIOT workshop topics include experience reports and technical details of novel R extensions. For example: ALTREP implementations, approaches to parallelization of R, integration with other (parallel, distributed, ...) computing engines, etc. Regards, Stepan Sindelar, Lukas Stadler (Oracle Labs), Jan Vitek (Northeastern), Alexander Bertram (BeDataDriven) [1] http://riotworkshop.github.io/ __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] RIOT 2020
After carefully considering the current situation regarding COVID-19, we decided to cancel this year's RIOT workshop. Best regards, Stepan On 25. 02. 20 17:38, Stepan wrote: I hope you don’t mind us using this mailing list for a small advertisement, but we think it is most relevant for this group: We'd like to invite you to RIOT 2020 - the 5rd workshop on R Implementation, Optimization and Tooling [1]. It will take place co-located with, and during, useR! 2020 in St. Louis on July 8th. RIOT is an excellent venue for deep technical discussions about R implementations, tools, optimizations and R extension, and will be very interesting for anyone interested in what’s under the hood of R. Regards, Stepan Sindelar, Lukas Stadler (Oracle Labs), Jan Vitek (Northeastern), Alexander Bertram (BeDataDriven) [1] http://riotworkshop.github.io __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[R-pkg-devel] RIOT 2020
I hope you don’t mind us using this mailing list for a small advertisement, but we think it is relevant for this group: We'd like to invite you to RIOT 2020 - the 5rd workshop on R Implementation, Optimization and Tooling [1]. It will take place co-located with, and during, useR! 2020 in St. Louis on July 8th. RIOT is an excellent venue for deep technical discussions about R implementations, tools, optimizations and R extensions, and will be very interesting for anyone interested in what’s under the hood of R. The RIOT workshop topics include experience reports and technical details of novel R extensions. For example: ALTREP implementations, approaches to parallelization of R, integration with other (parallel, distributed, ...) computing engines, etc. Regards, Stepan Sindelar, Lukas Stadler (Oracle Labs), Jan Vitek (Northeastern), Alexander Bertram (BeDataDriven) [1] http://riotworkshop.github.io/ __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[Rd] RIOT 2020
I hope you don’t mind us using this mailing list for a small advertisement, but we think it is most relevant for this group: We'd like to invite you to RIOT 2020 - the 5rd workshop on R Implementation, Optimization and Tooling [1]. It will take place co-located with, and during, useR! 2020 in St. Louis on July 8th. RIOT is an excellent venue for deep technical discussions about R implementations, tools, optimizations and R extension, and will be very interesting for anyone interested in what’s under the hood of R. Regards, Stepan Sindelar, Lukas Stadler (Oracle Labs), Jan Vitek (Northeastern), Alexander Bertram (BeDataDriven) [1] http://riotworkshop.github.io/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Get memory address of an R data frame
On 09. 01. 20 15:41, lille stor wrote:
I believe this could be done without creating side effects (e.g.
crash) as we are just talking about changing values.
that is exactly the issue that my last two points warn about. Example:
a <- mtcars
.Call("my_innocent_function", a)
Would you expect that mtcars data.frame would be altered after this code
is executed? What if some existing code relies on mtcars always
containing the same data, which is a perfectly valid assumption given R
specification.
If what you are trying to do is to have mutable data frame, then this
goes against the philosophy of R. You can get mutability with
environments and other R types that are intentionally mutable and their
mutability is documented.
You can get data.frame mutability with the data.table package, but the
tricks it's doing under the hood may bite back. In its source code you
can also see how these things can be done, but unless you really need
to, I would advise against implementing this yourself.
Best,
Stepan
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Get memory address of an R data frame
Hello Lille, raw data of a data.frame (or more precisely a list, because data.frame is just a list with "data.frame" class) is an array of R specific data structures (SEXP), so a generic C function will not be able to work with them. As a per-processing step, you may allocate an array for the pointers to the raw data of the columns yourself (there will be hopefully only a few compared to the size of the columns themselves). For this you'll need functions VECTOR_ELT to access the columns and DATAPTR to get their raw data (eventually TYPEOF to find out their type). Note that this won't work for a data frame that contains another list. If this memory layout doesn't work for you, then you may need to copy the whole data frame. If you want to update the data from C, then keep in mind that 1) R vectors have value semantics and you should not be altering raw data of any vector unless you know that its not referenced from anywhere else -- otherwise you should make a copy, alter that copy instead and return it as the result from your C function. 2) R has generational garbage collector, so it *must* know about references between R objects and so you should use SET_VECTOR_ELT to update the data of a list (some would say that you can update the raw data if you really understand how the GC and R internals work, I would say: just don't) Best, Stepan On 09. 01. 20 12:48, lille stor wrote: Hello, I would like for my C function to be able to manipulate some values stored in an R data frame. To achieve this, a need the (real) memory address where the R data frame stores its data (hopefully in a contiguous way). Then, from R, I call the C function and passing this memory address as a parameter. The question: how can we get the memory address of the R data frame? Thank you! L. __ R-devel@r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel=DwICAg=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=neKFCw86thQe2E2-61NAgpDMw4cC7oD_tUTTzraOkQM=ob3rEYy-Pk9cOE-VcE6_0TaHPYjGJ4kHYZru_jqXf38=AV2V5CyECZzyfSMZdViD_co5mAGurLNEu4jhA_CTDsk= __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Deparsing raw vectors with names
Hello,
deparse(structure(as.raw(1), .Names=c('a'))) gives "as.raw(c(a = 0x01))"
in 3.5.1 and later (actually tested on 3.5.1, 3.6.1 and devel). If you
execute as.raw(c(a = 0x01)), you get a raw vector without the names.
If the stripping of the names is the correct behavior of as.raw (I would
think it is), then perhaps deparse should use the old behavior for raw
vectors. On R-3.4.1 it gives: "structure(as.raw(0x01), .Names = 'a')".
Regards,
Stepan
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] "if" function in pure R?
Hello Alexandre,
there are two sides of your question it seems to me:
- there is no possibility to extend the R parser with new syntax. The R
parser knows to internally "rewrite"(*) things such as
if (a>b) print(1) else print(2)
into
`if`(a>b, print(1), print(2))
The parser has a fixed set of functions that get some special treatment
like this.
- there is possibility to implement control flow constructs thanks to
the lazy evaluation of arguments. An example of such function is ifelse
from base.
Best,
Stepan
(*) "rewrite" is illustrative, the exact internal working of this is not
important for this discussion
On 27/05/2019 08:16, Alexandre Courtiol wrote:
Thanks a lot Jiefei,
I had thought of defining a binary operator (inspired by pipes) or simply
using an additional condition in the if() calls [e.g. if(foo & fn(bar))
doSomeThing; with fn(bar) returning a logical], but both are workaround
that I do not find as elegant as a proper control-flow construct.
Thus two questions remain:
- is it possible to create a control-flow construct in pure R?
- if yes, how?
Anyone with more insights?
Thanks
On Mon, 27 May 2019 at 04:27, King Jiefei wrote:
Hi Alexandre,
I'm not an R expert so this is only my personal thought:
I don't think you can achieve what you want exactly. A possible solution
would be defining a binary operator %*%, where you can replace the asterisk
with any function name you want. The function %*% is special since it has
two arguments, left operand and right operand respectively. You then
can call the `substitute` function to get its function arguments in an
expression format and proceed to do what you want. Here is an example to
show the idea.
*Code:*
`%myOperator%` <- function(x, y) {
x = substitute(x)
y = substitute(y)
return(list(x, y))
}
myIf(i == 1, arg1) %myOperator% {
doSomeThing
}
*Results:*
[[1]]
myIf(i == 1, arg1)
[[2]]
{
doSomeThing
}
I hope that helps.
Best,
Jiefei
On Sun, May 26, 2019 at 4:45 AM Alexandre Courtiol <
alexandre.court...@gmail.com> wrote:
Hi all,
Could anyone refer to me to a good source to learn how to program a simple
control-flow construct* in R, or provide me with a simple example?
Control-flow constructs are programmed as primitives, but I would like to
be able to do that (if possible) in pure R.
The general context is that those functions are a mystery to me. The
motivating example is that I would like to create a function that behave
similarly to base::`if` with an extra argument to the function (e.g. to
include an error rate on the condition).
Many thanks,
Alex
* control-flow constructs are functions such as if, for, while... that
allow for call of the form fn(x) expr to work (see ?Control).
--
Alexandre Courtiol
https://urldefense.proofpoint.com/v2/url?u=http-3A__sites.google.com_site_alexandrecourtiol_home=DwICAg=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=neKFCw86thQe2E2-61NAgpDMw4cC7oD_tUTTzraOkQM=S7ovivpWoG4APCzcwxdRGsn_Rr4FYxNrxmayP5prMhQ=LOGFMQPijyvAAyk5wcsWQkM6HjyrNqd9bJTkHhi_4YA=
*"Science is the belief in the ignorance of experts"*, R. Feynman
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel=DwICAg=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=neKFCw86thQe2E2-61NAgpDMw4cC7oD_tUTTzraOkQM=S7ovivpWoG4APCzcwxdRGsn_Rr4FYxNrxmayP5prMhQ=sAVfqnOeGqyqLFdvykN3nfGgEwxOCo7oq3slMwEWKi8=
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] most robust way to call R API functions from a secondary thread
Hi Andreas, note that with the introduction of ALTREP, as far as I understand, calls as "simple" as DATAPTR can execute arbitrary code (R or native). Even without ALTREP, if you execute user-provided R code via Rf_eval and such on some custom thread, you may end up executing native code of some package, which may assume it is executed only from the R main thread. Could you (1) decompose your problem in a way that in some initial phase you pull all the necessary data from R, then start the parallel computation, and then again in the R main thread "submit" the results back to the R world? If you wanted something really robust, you can (2) "send" the requests for R API usage to the R main thread and pause the worker thread until it receives the results back. This looks similar to what the "later" package does. Maybe you can even use that package for your purposes? Do you want to parallelize your code to achieve better performance? Even with your proposed solution, you need synchronization and chances are that excessive synchronization will severely affect the expected performance benefits of parallelization. If you do not need to synchronize that much, then the question is if you can do with (1) or (2). Best regards, Stepan On 19/05/2019 11:31, Andreas Kersting wrote: Hi, As the subject suggests, I am looking for the most robust way to call an (arbitrary) function from the R API from another but the main POSIX thread in a package's code. I know that, "[c]alling any of the R API from threaded code is ‘for experts only’ and strongly discouraged. Many functions in the R API modify internal R data structures and might corrupt these data structures if called simultaneously from multiple threads. Most R API functions can signal errors, which must only happen on the R main thread." (https://urldefense.proofpoint.com/v2/url?u=https-3A__cran.r-2Dproject.org_doc_manuals_r-2Drelease_R-2Dexts.html-23OpenMP-2Dsupport=DwIFaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=neKFCw86thQe2E2-61NAgpDMw4cC7oD_tUTTzraOkQM=d1r2raD4w0FF7spOVuz2IVEo0P_II3ZtSbw0TU2NmaE=JaadZR_m-QiJ3BQzzQ_fJPYt034tM5Ts6vKhdi6f__A=) Let me start with my understanding of the related issues and possible solutions: 1) R API functions are generally not thread-safe and hence one must ensure, e.g. by using mutexes, that no two threads use the R API simultaneously 2) R uses longjmps on error and interrupts as well as for condition handling and it is undefined behaviour to do a longjmp from one thread to another; interrupts can be suspended before creating the threads by setting R_interrupts_suspended = TRUE; by wrapping the calls to functions from the R API with R_ToplevelExec(), longjmps across thread boundaries can be avoided; the only reason for R_ToplevelExec() itself to fail with an R-style error (longjmp) is a pointer protection stack overflow 3) R_CheckStack() might be executed (indirectly), which will (probably) signal a stack overflow because it only works correctly when called form the main thread (see https://urldefense.proofpoint.com/v2/url?u=https-3A__cran.r-2Dproject.org_doc_manuals_r-2Drelease_R-2Dexts.html-23Threading-2Dissues=DwIFaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=neKFCw86thQe2E2-61NAgpDMw4cC7oD_tUTTzraOkQM=d1r2raD4w0FF7spOVuz2IVEo0P_II3ZtSbw0TU2NmaE=J_TMw2gu43dxB_EX2vHbtF4Zr4bIAFR8RSFzvbRV6jE=); in particular, any function that does allocations, e.g. via allocVector3() might end up calling it via GC -> finalizer -> ... -> eval; the only way around this problem which I could find is to adjust R_CStackLimit, which is outside of the official API; it can be set to -1 to disable the check or be changed to a value appropriate for the current thread 4) R sets signal handlers for several signals and some of them make use of the R API; hence, issues 1) - 3) apply; signal masks can be used to block delivery of signals to secondary threads in general and to the main thread while other threads are using the R API I basically have the following questions: a) Is my understanding of the issues accurate? b) Are there more things to consider when calling the R API from secondary threads? c) Are the solutions proposed appropriate? Are there scenarios in which they will fail to solve the issue? Or might they even cause new problems? d) Are there alternative/better solutions? Any feedback on this is highly appreciated. Below you can find a template which, combines the proposed solutions (and skips all non-illustrative checks of return values). Additionally, R_CheckUserInterrupt() is used in combination with R_UnwindProtect() to regularly check for interrupts from the main thread, while still being able to cleanly cancel the threads before fun_running_in_main_thread() is left via a longjmp. This is e.g. required if the secondary threads use memory which was allocated in fun_running_in_main_thread() using e.g. R_alloc(). Bes
[Rd] RIOT 2019
I hope you don’t mind us using this mailing list for a small advertisement, but we think it is most relevant for this group: We'd like to invite you to RIOT 2019 - the 4rd workshop on R Implementation, Optimization and Tooling [1]. It will take place co-located with, and during, useR! 2019 in Toulouse on July 11th. RIOT is an excellent venue for deep technical discussions about R implementations, tools, optimizations and extension, and will be very interesting for anyone interested in what’s under the hood of R implementations. Regards, Stepan Sindelar, Lukas Stadler (Oracle), Jan Vitek (Northeastern), Alexander Bertram (BeDataDriven) [1] http://riotworkshop.github.io/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Bioc-devel] rhdf5 NA's in int32
Hello, I believe I noticed two bugs in rhdf5. I'm using R 3.3.3 on Windows64, 2.18.0 linking to C-library HDF5 1.8.7 1. problem: h5read(..., bit64conversion='double') converts int32 columns to double as well. I did not expect that the option would affect 64bit integer types only. Is this intentional? Is it documented? 2. problem: AFAIK -2^63 of type int64 is not a number, it encodes NA. I would expect that even with bit64conversion='double' or 'int', this special value silently becomes NA. (Or am I wrong? Is this convention generally accepted?) The warnings "integer precision lost" or "integer overflow" (respectively), that are triggered by values that are in abs >= 2^53 or >= 2^31 (resp.), should not be triggered by this special value. Thank you all for the package, Stepan Kasal ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] adding a coathor to the tracker
Hi, How can I give to someone else in addition to myself the rights to edit my package and upload their changes to the tracker? Thanks, Stepan = Stepan Grinek, PhD. stepan.grinek@<https://mail.ladydavis.ca/owa/redir.aspx?C=qquqSpiJ50SABBw090180slY-Rvz79AIdIr9h8bhRnAHABmkmAPuVI5ircr4JEf7atwZfoW0jg4.=mailto%3acelia.greenwood%40mcgill.ca>ladydavis.ca Lady Davis Institute for Medical Research, Jewish General Hospital. Mailing address: H414, Clinical Epidemiology, Jewish General Hospital 3755 C�te Ste. Catherine, Montr�al, QC H3T 1E2 Tel 514 340 8222 x8380. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
