Re: [Rd] A trap for young players with the lapply() function.
On 29/03/17 20:32, Enrico Schumann wrote:
(inline)
On Tue, 28 Mar 2017, Rolf Turner writes:
On 28/03/17 04:21, Barry Rowlingson wrote:
On Mon, Mar 27, 2017 at 1:17 AM, Rolf Turner wrote:
Is there any way to trap/detect the use of an optional argument called
"X" and thereby issue a more perspicuous error message?
This would be helpful to those users who, like myself, are bears of very
little brain.
Failing that (it does look impossible)
You can get the names of named arguments:
> z = function(x,X){cos(x*X)}
> names(formals(z))
[1] "x" "X"
That doesn't seem to help. I tried putting a browser inside lapply()
and looked at formals(FUN). This gave NULL, because FUN has already
been taken to be the list argument "x" to which lapply() is being
applied.
You can get the call, without the arguments being
matched, with `sys.call`. In your call of lapply,
saying `sys.call()` before anything else would give
you
lapply(y, function(x, X) {
cos(x * X)
}, X = 2 * pi)
which would allow you to get at the argument names of
your function.
if ("X" %in% names(sys.call()[[3L]][[2L]]))
warnings("found 'X'")
But more would be needed: you need to figure out which
argument you actually meant to be FUN. (In the above,
I simply assumed it was the second passed argument.)
That is, you would need to figure out which passed
argument is a function, which is not safe either,
since ... may also contain functions.
This idea appears to work for the particular example that I used, but it
is not clear to me how to make it work in general. E.g. if we define
foo <- function(x,X){X*x}
and then do
lapply(xxx,foo,X=2*pI)
we find that sys.call[[3]] is of length 1 and consists only of the
*name* "foo". One can then inspect
formals(get(as.character(sys.call[[3]])))
and find "X" therein, on the basis of which to trigger a warning.
However I don't know how to approach distinguish the two cases, or how
to discern if there may be other cases lurking in the bushes.
So the problem still looks insoluble to me.
cheers,
Rolf
--
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Transferring ownership of R-managed buffer
Tim, What you're describing is a special case of the ALTREP framework/API that Luke Tierney, Tomas Kalibera, and I are working on putting into R. See my initial proposal to the DSC here: https://www.r-project.org/dsc/2016/slides/customvectors.html and the subsequent branch here: https://svn.r-project.org/R/branches/ALTREP/ where Luke and I have merged the ideas from that proposal with work he had been pursuing independently. The framework is implemented and passes basic testing (make check all minus the internet checks, from what I recall). The concept of having a vector which is a "window" into another vector without duplication ( which I suspect is at least related to your use-case, though I could be wrong) is a special case of one of alt-representations I have implemented there. Unfortunately we were not able to get it ready in time for this upcoming release (it is a rather sizable change to the internals), but we hope to merge it into R-devel not too long after (there isn't a specific timeline currently AFAIK). I know that the release after next is quite a long wait but unfortunately that is how it played out. So anyway, in the future this will be easy to do, and what you are trying to do may even happen automatically. More comments inline. On Wed, Mar 29, 2017 at 11:36 AM, Tim Keitt wrote: > http://www.keittlab.org/ > > On Wed, Mar 29, 2017 at 1:04 PM, Hervé Pagès wrote: > > > Hi Tim, > > > > On 03/29/2017 08:24 AM, Tim Keitt wrote: > > > >> I have a use case where I would like to create an SEXP around an > existing > >> buffer that is managed by R, thus avoiding a copy operation. > >> > > > > What to you mean exactly by "an existing buffer managed by R"? > > > > Not allocated from the general heap, but rather allocated using R's > allocator and therefore returnable (hypothesis!) to the R interpreter as a > vector (not external pointer) without a copy operation. > > > > > If I have > >> something like: > >> > >> void *p = (void*) RAW(PROTECT(Rf_allocVector(RAWSXP, n))); > >> ... additional maniupulation ... > >> SEXP x = somefunc(SXPTYPE, n, p); // > >> > >> Is there a "placement" constructor available? > >> > > > > What's a "placement" constructor? > > > > Function that constructs an object using previously allocated memory (for > example provided by a 3rd party library). In some cases, responsibility for > further management of the buffer is transferred to the new object. > I'm not sure what you mean by this last part, but the rest is one of the primary use cases of the ALTREP stuff. I don't think you can do what you're describing now. See my comments below. > > > > > (I have arranged for the > >> corresponding UNPROTECT.) I've looked at and experimented with > R_allocator > >> and allocVector3, but can't quite get it right. I know this is odd, but > it > >> makes sense for my use case. > >> > > > > Not sure I follow what you are trying to do. Note that an SEXP is a > > pointer to a C struct called SEXPREC. I think that trying to point an > > SEXPREC struct to data pointed to by an existing SEXPREC struct is very > > likely to lead to a disaster. > > > > Possibly (although that is not what I am proposing). I'm still trying to > grok where the state resides (all in the SEXPREC or between that and the > allocator). > > > > > > Note that if the existing buffer managed by R is an SEXP (e.g. b) and > > your code has access to this SEXP then you don't need to create another > > SEXP around its data. Since SEXPs are pointers doing > > > > SEXP x = b; > > > > is fine and doesn't generate any copy. > > > > The buffer in this case might be resized and reallocated and the original > SEXP will have gone out of scope. > I'm a little unclear what you're trying to do here. Traditional atomic vectors are a SEXP header plus a C array contiguous in memory, where the SEXP header knows the length of the array. Note that resizing the array is AFAIK not supported (it's not part of the blessed C api) and the internal macros that code uses to do this now will have to go away when the ALTREP branch is merged. That is in fact one of the major reasons we couldn't get it prepared in time for this release. > > > > > > And if your code only has access to a "naked" pointer to the buffer > > managed by R (e.g. to RAW(b) is the buffer is actually in an SEXP) > > then why would you need to wrap it inside an SEXP? > > > > I'm working with 3rd party structures that can work with a pointer and > buffer length, but cannot track the original SEXP. I just need to construct > an SEXP (SEXPREC) and point it at the buffer, set the vector length and > data type. I was checking to see if there is an API for that (and that wont > interfere with Rcpp declarations as it pulls in parts of Rinternals.h, but > as far as I can tell, not the parts that I need to work with SEXPRECs at a > low-level). > Atomic vector SEXPRECs are (and must be) contiguous in memory (header plus payload), so I don't think what you're wanting i
Re: [Rd] Transferring ownership of R-managed buffer
http://www.keittlab.org/ On Wed, Mar 29, 2017 at 1:04 PM, Hervé Pagès wrote: > Hi Tim, > > On 03/29/2017 08:24 AM, Tim Keitt wrote: > >> I have a use case where I would like to create an SEXP around an existing >> buffer that is managed by R, thus avoiding a copy operation. >> > > What to you mean exactly by "an existing buffer managed by R"? > Not allocated from the general heap, but rather allocated using R's allocator and therefore returnable (hypothesis!) to the R interpreter as a vector (not external pointer) without a copy operation. > > If I have >> something like: >> >> void *p = (void*) RAW(PROTECT(Rf_allocVector(RAWSXP, n))); >> ... additional maniupulation ... >> SEXP x = somefunc(SXPTYPE, n, p); // >> >> Is there a "placement" constructor available? >> > > What's a "placement" constructor? > Function that constructs an object using previously allocated memory (for example provided by a 3rd party library). In some cases, responsibility for further management of the buffer is transferred to the new object. > > (I have arranged for the >> corresponding UNPROTECT.) I've looked at and experimented with R_allocator >> and allocVector3, but can't quite get it right. I know this is odd, but it >> makes sense for my use case. >> > > Not sure I follow what you are trying to do. Note that an SEXP is a > pointer to a C struct called SEXPREC. I think that trying to point an > SEXPREC struct to data pointed to by an existing SEXPREC struct is very > likely to lead to a disaster. > Possibly (although that is not what I am proposing). I'm still trying to grok where the state resides (all in the SEXPREC or between that and the allocator). > > Note that if the existing buffer managed by R is an SEXP (e.g. b) and > your code has access to this SEXP then you don't need to create another > SEXP around its data. Since SEXPs are pointers doing > > SEXP x = b; > > is fine and doesn't generate any copy. > The buffer in this case might be resized and reallocated and the original SEXP will have gone out of scope. > > And if your code only has access to a "naked" pointer to the buffer > managed by R (e.g. to RAW(b) is the buffer is actually in an SEXP) > then why would you need to wrap it inside an SEXP? > I'm working with 3rd party structures that can work with a pointer and buffer length, but cannot track the original SEXP. I just need to construct an SEXP (SEXPREC) and point it at the buffer, set the vector length and data type. I was checking to see if there is an API for that (and that wont interfere with Rcpp declarations as it pulls in parts of Rinternals.h, but as far as I can tell, not the parts that I need to work with SEXPRECs at a low-level). THK > H. > > >> Thanks for any pointers. >> >> THK >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.keit >> tlab.org_&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd >> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5dLU_mdGwmpIk_iUqE0dPNjqw >> y8wiRy6hS_lWF9k&s=h04DJujKDfqzbLz4FmP3_fZ5bYS3t7UEjSwpLrW5mL0&e= >> >> [[alternative HTML version deleted]] >> >> __ >> R-devel@r-project.org mailing list >> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et >> hz.ch_mailman_listinfo_r-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84V >> tBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5d >> LU_mdGwmpIk_iUqE0dPNjqwy8wiRy6hS_lWF9k&s=WpLxtWWwjoZ7TjW6oW- >> vxE6s3LY5kZCG1H5h0xb0Bbs&e= >> >> > -- > Hervé Pagès > > Program in Computational Biology > Division of Public Health Sciences > Fred Hutchinson Cancer Research Center > 1100 Fairview Ave. N, M1-B514 > P.O. Box 19024 > Seattle, WA 98109-1024 > > E-mail: hpa...@fredhutch.org > Phone: (206) 667-5791 > Fax:(206) 667-1319 > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Transferring ownership of R-managed buffer
On 03/29/2017 11:04 AM, Hervé Pagès wrote: Hi Tim, On 03/29/2017 08:24 AM, Tim Keitt wrote: I have a use case where I would like to create an SEXP around an existing buffer that is managed by R, thus avoiding a copy operation. What to you mean exactly by "an existing buffer managed by R"? If I have something like: void *p = (void*) RAW(PROTECT(Rf_allocVector(RAWSXP, n))); ... additional maniupulation ... SEXP x = somefunc(SXPTYPE, n, p); // Is there a "placement" constructor available? What's a "placement" constructor? (I have arranged for the corresponding UNPROTECT.) I've looked at and experimented with R_allocator and allocVector3, but can't quite get it right. I know this is odd, but it makes sense for my use case. Not sure I follow what you are trying to do. Note that an SEXP is a pointer to a C struct called SEXPREC. I think that trying to point an SEXPREC struct to data pointed to by an existing SEXPREC struct is very likely to lead to a disaster. Note that if the existing buffer managed by R is an SEXP (e.g. b) and your code has access to this SEXP then you don't need to create another SEXP around its data. Since SEXPs are pointers doing SEXP x = b; is fine and doesn't generate any copy. And if your code only has access to a "naked" pointer to the buffer managed by R (e.g. to RAW(b) is the buffer is actually in an SEXP) ^^ if sorry H. then why would you need to wrap it inside an SEXP? H. Thanks for any pointers. THK https://urldefense.proofpoint.com/v2/url?u=http-3A__www.keittlab.org_&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5dLU_mdGwmpIk_iUqE0dPNjqwy8wiRy6hS_lWF9k&s=h04DJujKDfqzbLz4FmP3_fZ5bYS3t7UEjSwpLrW5mL0&e= [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5dLU_mdGwmpIk_iUqE0dPNjqwy8wiRy6hS_lWF9k&s=WpLxtWWwjoZ7TjW6oW-vxE6s3LY5kZCG1H5h0xb0Bbs&e= -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax:(206) 667-1319 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Transferring ownership of R-managed buffer
Hi Tim, On 03/29/2017 08:24 AM, Tim Keitt wrote: I have a use case where I would like to create an SEXP around an existing buffer that is managed by R, thus avoiding a copy operation. What to you mean exactly by "an existing buffer managed by R"? If I have something like: void *p = (void*) RAW(PROTECT(Rf_allocVector(RAWSXP, n))); ... additional maniupulation ... SEXP x = somefunc(SXPTYPE, n, p); // Is there a "placement" constructor available? What's a "placement" constructor? (I have arranged for the corresponding UNPROTECT.) I've looked at and experimented with R_allocator and allocVector3, but can't quite get it right. I know this is odd, but it makes sense for my use case. Not sure I follow what you are trying to do. Note that an SEXP is a pointer to a C struct called SEXPREC. I think that trying to point an SEXPREC struct to data pointed to by an existing SEXPREC struct is very likely to lead to a disaster. Note that if the existing buffer managed by R is an SEXP (e.g. b) and your code has access to this SEXP then you don't need to create another SEXP around its data. Since SEXPs are pointers doing SEXP x = b; is fine and doesn't generate any copy. And if your code only has access to a "naked" pointer to the buffer managed by R (e.g. to RAW(b) is the buffer is actually in an SEXP) then why would you need to wrap it inside an SEXP? H. Thanks for any pointers. THK https://urldefense.proofpoint.com/v2/url?u=http-3A__www.keittlab.org_&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5dLU_mdGwmpIk_iUqE0dPNjqwy8wiRy6hS_lWF9k&s=h04DJujKDfqzbLz4FmP3_fZ5bYS3t7UEjSwpLrW5mL0&e= [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Ceu5dLU_mdGwmpIk_iUqE0dPNjqwy8wiRy6hS_lWF9k&s=WpLxtWWwjoZ7TjW6oW-vxE6s3LY5kZCG1H5h0xb0Bbs&e= -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax:(206) 667-1319 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Transferring ownership of R-managed buffer
I have a use case where I would like to create an SEXP around an existing buffer that is managed by R, thus avoiding a copy operation. If I have something like: void *p = (void*) RAW(PROTECT(Rf_allocVector(RAWSXP, n))); ... additional maniupulation ... SEXP x = somefunc(SXPTYPE, n, p); // Is there a "placement" constructor available? (I have arranged for the corresponding UNPROTECT.) I've looked at and experimented with R_allocator and allocVector3, but can't quite get it right. I know this is odd, but it makes sense for my use case. Thanks for any pointers. THK http://www.keittlab.org/ [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] A trap for young players with the lapply() function.
(inline)
On Tue, 28 Mar 2017, Rolf Turner writes:
> On 28/03/17 04:21, Barry Rowlingson wrote:
>> On Mon, Mar 27, 2017 at 1:17 AM, Rolf Turner wrote:
>>>
>>> Is there any way to trap/detect the use of an optional argument called
>>> "X" and thereby issue a more perspicuous error message?
>>>
>>> This would be helpful to those users who, like myself, are bears of very
>>> little brain.
>>>
>>> Failing that (it does look impossible)
>>
>> You can get the names of named arguments:
>>
>> > z = function(x,X){cos(x*X)}
>> > names(formals(z))
>> [1] "x" "X"
>
> That doesn't seem to help. I tried putting a browser inside lapply()
> and looked at formals(FUN). This gave NULL, because FUN has already
> been taken to be the list argument "x" to which lapply() is being
> applied.
You can get the call, without the arguments being
matched, with `sys.call`. In your call of lapply,
saying `sys.call()` before anything else would give
you
lapply(y, function(x, X) {
cos(x * X)
}, X = 2 * pi)
which would allow you to get at the argument names of
your function.
if ("X" %in% names(sys.call()[[3L]][[2L]]))
warnings("found 'X'")
But more would be needed: you need to figure out which
argument you actually meant to be FUN. (In the above,
I simply assumed it was the second passed argument.)
That is, you would need to figure out which passed
argument is a function, which is not safe either,
since ... may also contain functions.
>>> might it not be a good idea to
>>> add a warning to the help for lapply(), to the effect that if FUN has an
>>> optional argument named "X" then passing this argument via "..." will
>>> cause this argument to be taken as the first argument to lapply() and
>>> thereby induce an error?
>>
>> Another idea might be to use purrr:map instead, which is quite happy
>> with X in your function:
>>
>> > xxx <- purrr::map(y,function(x,X){cos(x*X)},X=2*pi)
>> > xxx
>> [[1]]
>> [1] 0.08419541
>>
>> [[2]]
>> [1] 0.6346404
>>
>> [[3]]
>> [1] 0.9800506
>>
>> [[4]]
>> [1] 0.8686734
>>
>> [[5]]
>> [1] -0.9220073
>>
>> But don't feed `.x` to your puing cats, or fails silently:
>>
>> > xxx <- purrr::map(y,function(x,.x){cos(x*.x)},.x=2*pi)
>> > xxx
>> [[1]]
>> NULL
>>
>> But who would have a function with `.x` as an argument?
>
> Indeed. It struck me that a possible workaround would be to change
> the name of the first argument of lapply() from "X" to ".X". No-one
> would have a function with an argument names ".X" --- at least I
> wouldn't, so this would solve the problem for me.
>
> It seems to me that this change could be made without breaking anything.
> But perhaps I am engaging in my usual PollyAnna-ish optimism! :-)
>
> cheers,
>
> Rolf
--
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] `[` not recognized as a primitive in certain cases.
> Joris Meys
> on Tue, 28 Mar 2017 15:19:14 +0200 writes:
> Thank you gents, I overlooked the subtle differences.
> On Tue, Mar 28, 2017 at 2:49 PM, Lukas Stadler
> wrote:
>> “typeof” is your friend here:
>>
>> > typeof(`[`)
>> [1] "special"
>> > typeof(mc[[1]])
>> [1] "symbol"
>> > typeof(mc2[[1]])
>> [1] "special"
>>
>> so mc[[1]] is a symbol, and thus not a primitive.
or str() which should be better known to Joe Average useR
> mc <- call("[",iris,2,"Species")
> str(mc[[1]])
symbol [
> str(`[`)
.Primitive("[")
>
>> - Lukas
>>
>> > On 28 Mar 2017, at 14:46, Michael Lawrence
>> wrote:
>> >
>> > There is a difference between the symbol and the function (primitive
>> > or closure) to which it is bound.
>> >
>> > This:
>> > mc2 <- as.call(list(`[`,iris,2,"Species"))
>> >
>> > Evaluates `[` to its value, in this case the primitive object, and the
>> > primitive itself is incorporated into the returned call.
>> >
>> > If you were to do this:
>> > mc2 <- as.call(list(quote(`[`),iris,2,"Species"))
>> >
>> > The `[` would _not_ be evaluated, quote() would return the symbol, and
>> > the symbol would end up in the call.
>> >
>> > The two forms have virtually identical behavior as long as the call
>> > ends up getting evaluated in the same environment.
>> >
>> > On Tue, Mar 28, 2017 at 3:03 AM, Joris Meys
wrote:
>> >> Dear,
>> >>
>> >> I have noticed this problem while looking at the following question on
>> >> Stackoverflow :
>> >>
>> >> http://stackoverflow.com/questions/42894213/s4-class-
>> subset-inheritance-with-additional-arguments
>> >>
>> >> While going through callNextMethod, I've noticed the following odd
>> >> behaviour:
>> >>
>> >> mc <- call("[",iris,2,"Species")
>> >>
>> >> mc[[1]]
>> >> ## `[`
>> >>
>> >> is.primitive(`[`)
>> >> ## [1] TRUE
>> >>
>> >> is.primitive(mc[[1]])
>> >> ## [1] FALSE
>> >> # Expected to be TRUE
>> >>
>> >> mc2 <- as.call(list(`[`,iris,2,"Species"))
>> >>
>> >> is.primitive(mc2[[1]])
>> >> ## [1] TRUE
>> >>
>> >> So depending on how I construct the call (using call() or as.call() ),
>> the
>> >> function `[` is or is not recognized as a primitive by is.primitive()
>> >>
>> >> The behaviour is counterintuitive and -unless I miss something obvious
>> >> here- likely to be a bug imho. I immediately admit that my C chops
>> aren't
>> >> sufficient to come up with a patch.
>> >>
>> >> Cheers
>> >> Joris
>> >>
>> >> --
>> >> Joris Meys
>> >> Statistical consultant
>> >>
>> >> Ghent University
>> >> Faculty of Bioscience Engineering
>> >> Department of Mathematical Modelling, Statistics and Bio-Informatics
>> >>
>> >> tel : +32 (0)9 264 61 79
>> >> joris.m...@ugent.be
>> >> ---
>> >> Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php
>> >>
>> >>[[alternative HTML version deleted]]
>> >>
>> >> __
>> >> R-devel@r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>> >
>> > __
>> > R-devel@r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
> --
> Joris Meys
> Statistical consultant
> Ghent University
> Faculty of Bioscience Engineering
> Department of Mathematical Modelling, Statistics and Bio-Informatics
> tel : +32 (0)9 264 61 79
> joris.m...@ugent.be
> ---
> Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php
> [[alternative HTML version deleted]]
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
