Re: [Rd] How can C++ read the R object written into socket with saveRDS or save

[Rong lI Li]

Hi, all,

Many thanks for your suggestions/infos!

For my previous question: "how to get the size of R object, so that the C++
process can know exact bytes to read, and receive the R objectvia socket
connection", for now, I used the following way to achieve this. It worked
for me. Pls feel free to let me know if there are anything I missed.

I sent the length of serialized raw vector into socket connection first,
and then send the raw vector into connection:
> a <- list(1,2,3)
> a_serial <- serialize(a, NULL)
> a_length <- length(a_serial)
> a_length
[1] 70
> writeBin(as.integer(a_length), connection, endian="big")
> serialize(a, connection)

In C++ process, I receive one int variable first to get the length, and
then read  bytes from the connection.

It worked for me now.

BTW, if there is anything I against the posting rule, pls feel free to
correct me.

=

Rong "Jessica", Li
Platform Symphony TET, CSTL, IBM Systems &Technology Group, Development
Email:rong...@cn.ibm.com
[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Peter Meilstrup]

On Thu, Jun 27, 2013 at 8:49 PM, Ben Bolker  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
>   The lmer() function in the lme4 package has some code of the form
>
> mc <- match.call()
> mc[[1]] <- as.name("lFormula")
> lmod <- eval(mc, parent.frame(1L))
>
>   this is a fairly common idiom in R, found e.g. in lm(), used when
> one wants to pass all of the arguments of a function to a different
> function (in the case of lm() it's model.frame()).
>

I think the use of stack inspection ( usually parent.frame(), but with
match.call() often playing an accessory role) should be more widely
considered Evil, but I could get very off topic on that.

Passing along all the current function's arguments to another function is
implemented fairly well in S3 method dispatch. For example:

###

aMethod <- function(...) UseMethod("aMethod")

aMethod.default <- function(object, continue) {
  if (continue) {
.Class <- "alternate"
NextMethod()
  }
  c("default", object)
}

aMethod.alternate <- function(object, continue) {
  c("alternate", object)
}

x <- "this is the wrong x"

local({
  x <- "this is the right x"
  aMethod(x, TRUE)
})

###

Peter

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Duncan Murdoch]

On 13-06-27 11:49 PM, Ben Bolker wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


   The lmer() function in the lme4 package has some code of the form

mc <- match.call()
mc[[1]] <- as.name("lFormula")
lmod <- eval(mc, parent.frame(1L))

   this is a fairly common idiom in R, found e.g. in lm(), used when
one wants to pass all of the arguments of a function to a different
function (in the case of lm() it's model.frame()).

   This idiom fails ("error in eval(...) : could not find function
'lFormula'") when lme4 is not explicitly loaded, as will happen if one
uses lme4::lmer(...) to call the function without loading the package,
or when lmer() is called from within a function that Imports: but does
not Depend: on lme4.

   A simpler version suggested by Martin Maechler shows that this is a
generic issue (except that it's unlikely to happen with base R
functions because the relevant packages _will_ be loaded unless one
bends over backwards):

R_DEFAULT_PACKAGES=NULL R --vanilla

stats::lm(y~x,data=data.frame(1:5,1:5))

Error in eval(expr, envir, enclos) :
   could not find function "model.frame"

   I wonder if anyone has encountered this and/or can think of a fix
(something with setting the enclosing environment appropriately?)  The
only workarounds we can think of at present are (1) telling
dependent-package maintainers that they have to use Depends: instead
of Imports: (which seems to subvert the whole dependency-minimization
goal of Imports:) or (2) littering our code with lme4:: to make sure
that the function gets looked for in the right place (ugh).

   See also https://github.com/lme4/lme4/issues/50 .

   Any ideas appreciated.


I believe

mc[[1]] <- lFormula

will mostly solve the problem.  Since your code is evaluating lFormula, 
it will be found.


This will likely lead to ugly error messages if errors occur during 
lFormula (because the call will no longer look like lFormula(...), it 
will look like (function(...) { ... })( ... ).  You can mitigate this by 
using call. = FALSE in your error messages.


If you're saving mc in any of the results of the function, you'll also 
see the ugly display.


The other choice is to use the explicit ::, i.e.

mc[[1]] <- quote(lmer::lFormula)

This would be the solution I'd prefer, but you seem to have some 
prejudice against :: :-).


Duncan Murdoch

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] 'modifyList' drops (not adds) NULL components

[Raubertas, Richard]

Dear list,

Utils::modifyList() drops NULL components in its second argument, instead of 
adding them to the first argument.  Compare:
> modifyList(x=list(A=1), val=list(B=2, C=3))
$A
[1] 1

$B
[1] 2

$C
[1] 3

> modifyList(x=list(A=1), val=list(B=NULL, C=3))
$A
[1] 1

$C
[1] 3

To me this seems inconsistent with the documentation ("Elements in 'val' which 
are missing from 'x' are added to 'x'."), and also with how I'd want the 
function to behave.

> sessionInfo()
R version 3.0.1 Patched (2013-06-16 r62969)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C  
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base 


Richard Raubertas
Merck & Co.


Notice:  This e-mail message, together with any attachme...{{dropped:11}}

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] 'modifyList' drops (not adds) NULL components

[Deepayan Sarkar]

On Thu, Jun 27, 2013 at 10:59 PM, Raubertas, Richard
 wrote:
> Dear list,
>
> Utils::modifyList() drops NULL components in its second argument, instead of 
> adding them to the first argument.  Compare:
>> modifyList(x=list(A=1), val=list(B=2, C=3))
> $A
> [1] 1
>
> $B
> [1] 2
>
> $C
> [1] 3
>
>> modifyList(x=list(A=1), val=list(B=NULL, C=3))
> $A
> [1] 1
>
> $C
> [1] 3
>
> To me this seems inconsistent with the documentation ("Elements in 'val' 
> which are missing from 'x' are added to 'x'."),

Agreed.

> and also with how I'd want the function to behave.

Note that you still get

> foo <- modifyList(x=list(A=1), val=list(B=NULL, C=3))
> foo$B
NULL

So is there a specific reason you want the NULL elements to be
explicitly listed?

-Deepayan

>
>> sessionInfo()
> R version 3.0.1 Patched (2013-06-16 r62969)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
> locale:
> [1] LC_COLLATE=English_United States.1252
> [2] LC_CTYPE=English_United States.1252
> [3] LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
>
> Richard Raubertas
> Merck & Co.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] 'modifyList' drops (not adds) NULL components

[Raubertas, Richard]

See response inline below.

> -Original Message-
> From: Deepayan Sarkar [mailto:deepayan.sar...@gmail.com]
> Sent: Friday, June 28, 2013 7:17 AM
> To: Raubertas, Richard
> Cc: R-devel@r-project.org
> Subject: Re: [Rd] 'modifyList' drops (not adds) NULL components
> 
> On Thu, Jun 27, 2013 at 10:59 PM, Raubertas, Richard
>  wrote:
> > Dear list,
> >
> > Utils::modifyList() drops NULL components in its second argument, instead
> of adding them to the first argument.  Compare:
> >> modifyList(x=list(A=1), val=list(B=2, C=3))
> > $A
> > [1] 1
> >
> > $B
> > [1] 2
> >
> > $C
> > [1] 3
> >
> >> modifyList(x=list(A=1), val=list(B=NULL, C=3))
> > $A
> > [1] 1
> >
> > $C
> > [1] 3
> >
> > To me this seems inconsistent with the documentation ("Elements in 'val'
> which are missing from 'x' are added to 'x'."),
> 
> Agreed.
> 
> > and also with how I'd want the function to behave.
> 
> Note that you still get
> 
> > foo <- modifyList(x=list(A=1), val=list(B=NULL, C=3))
> > foo$B
> NULL
> 
> So is there a specific reason you want the NULL elements to be
> explicitly listed?
> 
> -Deepayan

Yes, definitely.  Although 'foo$B' gives the same result, 'length(foo)' or 
'names(foo)' do not.  Basically, the object 'foo', considered as a whole, is 
very different in the two cases, and this was a problem in the code I was 
writing.

Richard
> 
> >
> >> sessionInfo()
> > R version 3.0.1 Patched (2013-06-16 r62969)
> > Platform: x86_64-w64-mingw32/x64 (64-bit)
> >
> > locale:
> > [1] LC_COLLATE=English_United States.1252
> > [2] LC_CTYPE=English_United States.1252
> > [3] LC_MONETARY=English_United States.1252
> > [4] LC_NUMERIC=C
> > [5] LC_TIME=English_United States.1252
> >
> > attached base packages:
> > [1] stats graphics  grDevices utils datasets  methods   base
> >
> >
> > Richard Raubertas
> > Merck & Co.
Notice:  This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates Direct contact information
for affiliates is available at 
http://www.merck.com/contact/contacts.html) that may be confidential,
proprietary copyrighted and/or legally privileged. It is intended solely
for the use of the individual or entity named on this message. If you are
not the intended recipient, and have received this message in error,
please notify us immediately by reply e-mail and then delete it from 
your system.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Ben Bolker]

Duncan Murdoch  gmail.com> writes:

>

[snip]
> 
> The other choice is to use the explicit ::, i.e.
> 
> mc[[1]] <- quote(lmer::lFormula)
> 
> This would be the solution I'd prefer, but you seem to have some 
> prejudice against :: .
> 

  I just thought that the whole point of using the machinery of
namespaces, environments, etc., was that we shouldn't need to
tell R explicitly what environment we were working in ...

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Duncan Murdoch]

On 28/06/2013 12:42 PM, Ben Bolker wrote:

Duncan Murdoch  gmail.com> writes:

>

[snip]
>
> The other choice is to use the explicit ::, i.e.
>
> mc[[1]] <- quote(lmer::lFormula)
>
> This would be the solution I'd prefer, but you seem to have some
> prejudice against :: .
>

   I just thought that the whole point of using the machinery of
namespaces, environments, etc., was that we shouldn't need to
tell R explicitly what environment we were working in ...


And usually it works, just not always when you do surgery on 
expressions.  The snipped version worked, for instance.


Duncan Murdoch

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[peter dalgaard]

On Jun 28, 2013, at 18:42 , Ben Bolker wrote:

> Duncan Murdoch  gmail.com> writes:
> 
>> 
> 
> [snip]
>> 
>> The other choice is to use the explicit ::, i.e.
>> 
>> mc[[1]] <- quote(lmer::lFormula)
>> 
>> This would be the solution I'd prefer, but you seem to have some 
>> prejudice against :: .
>> 
> 
>  I just thought that the whole point of using the machinery of
> namespaces, environments, etc., was that we shouldn't need to
> tell R explicitly what environment we were working in ...

Not really. The point is to avoid name clashes. In a sense, the fundamental 
mode of operation is to use qualified names, but there are convenience features 
that let you use them unqualified by attaching or importing (parts of) the 
namespace. But, if, as in the present case, you can't assume that the lmer 
namespace is imported/attached by the caller, you likely shouldn't force it to 
be so for an eval.parent()-type call. So the qualified name is a natural 
solution in that case. 

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Read a text file into R with .Call()

[Hervé Pagès]

Hi Ge,

On 06/28/2013 04:45 AM, Ge Tan wrote:

Hi  Hervé,

Thank you so much!!! I tweaked it a little and it works exactly in the 
Biostrings way which I want to learn!
You are my idol!!

One minor question I want to ask is when I change the
alloc_XRawList("BStringSet", "BString", ans_width));
to DNAStringSet.
It reprots an error,

myAxt

   A DNAStringSet instance of length 315050
  width seq
Error in .Call2("new_CHARACTER_from_XString", x, xs_dec_lkup(x), PACKAGE = 
"Biostrings") :
   key 65 (char 'A') not in lookup table


Glad it works for you. I'll answer your new question on the Bioc-devel
mailing list. See you there.

H.



Do you have any idea?

Thanks again!
Ge

-- Original --
From:  "Herv  Pag s";
Date:  Fri, Jun 28, 2013 05:44 AM
To:  "Ge Tan"<184523...@qq.com>;
Cc:  "r-devel";
Subject:  Re: [Rd] Read a text file into R with .Call()



Hi Ge,

Here is one way to do this with the Biostrings C API. It does
2 passes on the file. There is also a 1-pass way but not necessarily
worth it because not as memory efficient.

The code below is untested (not even guaranteed to compile)!

SEXP read_text_file_in_BStringSet(FILE *FN)
{
  SEXP ans, width;
  IntAE width_buf;
  char *foo;
  cachedXVectorList cached_ans;
  cachedCharSeq cached_ans_elt;
  int i;

  /* 1st pass: compute the 'width' vector. */
  width_buf = new_IntAE(0, 0, 0);
  while (foo = readLine(FN)) {
  IntAE_insert_at(&width_buf, IntAE_get_nelt(width_buf),
strlen(foo));
  }
  PROTECT(width = new_INTEGER_from_IntAE(&width_buf));

  /* Allocate 'ans'. */
  PROTECT(ans = alloc_XRawList("BStringSet", "BString", ans_width));

  /* 2nd pass: Fill 'ans' with data. */
  cached_ans = cache_XVectorList(ans);
  rewind(FN);
  i = 0;
  while (foo = readLine(FN)) {
  cached_ans_elt = get_cachedXRawList_elt(&cached_ans, i);
  memcpy((char *) cached_ans_elt->seq, foo, INTEGER(width)[i] *
sizeof(char));
  i++;
  }

  UNPROTECT(2);
  return ans;
}

The returned object is a BStringSet object.

Note that I kept your

  foo = readLine(FN)

approach for reading the file line by line. A more efficient way would
be to use something like

  n = getLineLength(FN)

for the 1st pass (no need to load the line of text into the foo
buffer), and something like

  readLine(FN, (char *) cached_ans_elt->seq)

for the 2nd pass so the data is loaded directly to where it needs to
go (i.e. without going first thru the foo buffer, hence avoiding the
memcpy()).

Cheers,
H.

On 06/27/2013 12:37 PM, Ge Tan wrote:

Hi Simons,

Thanks for your reply.
1 is just an example I wrote. In fact, there can be millions of strings 
(all of them are different and each has thousands of characters) I want to read 
from the file. So if I use mkChar it will store the same amount of the copies 
in the global cache.
The problem is when I get the returned qNames in R, and then rm(qNames) and do 
the gc().
gc() shows a normal amout of memory it uses. But from the top command, this R 
session can still use several GB. The rm() and gc() does not take effect on the 
memory release. (I suspect the release of the global cache is not done, even 
now there is not objects pointing to them.)
I am sure there is no other memory leak problem. Once I run the mkChar, the 
memory issue emerges.

So I am comfused how to read lines from text files and make it into R character 
vectors to pass back to R. We cannot store each of them into the global cache 
nor is not necessary as they are not duplicated.
Regarding the raw vector method, I am not quite clear how to manipulate it. 
Could you give some more detailed examples?

I attached more complete code I wrote. BTW, I am using R version 2.15.2.

Thanks!
Ge

PROTECT(qNames = NEW_CHARACTER(nrAxts));
PROTECT(qStart = NEW_INTEGER(nrAxts));
PROTECT(qEnd = NEW_INTEGER(nrAxts));
PROTECT(qStrand = NEW_CHARACTER(nrAxts));
PROTECT(qSym = NEW_CHARACTER(nrAxts));
PROTECT(tNames = NEW_CHARACTER(nrAxts));
PROTECT(tStart = NEW_INTEGER(nrAxts));
PROTECT(tEnd = NEW_INTEGER(nrAxts));
PROTECT(tStrand = NEW_CHARACTER(nrAxts));
PROTECT(tSym = NEW_CHARACTER(nrAxts));
PROTECT(score = NEW_INTEGER(nrAxts));
PROTECT(symCount = NEW_INTEGER(nrAxts));
PROTECT(returnList = NEW_LIST(12));
int *p_qStart, *p_qEnd, *p_tStart, *p_tEnd, *p_score, *p_symCount;
p_qStart = INTEGER_POINTER(qStart);
p_qEnd = INTEGER_POINTER(qEnd);
p_tStart = INTEGER_POINTER(tStart);
p_tEnd = INTEGER_POINTER(tEnd);
p_score = INTEGER_POINTER(score);
p_symCount = INTEGER_POINTER(symCount);
int j = 0;
i = 0;
for(j = 0; j < nrAxtFiles; j++){
  char *filepath_elt = (char *) R_alloc(strlen(CHAR(STRING_ELT(filepath, 
j))), sizeof(char));
  strcpy(filepath_elt, CHAR(STRING_ELT(filepath, j)));
  lf = lineFileOpen(filepath_elt, TRUE);
  while((axt = axtRead(lf)) != N

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Ben Bolker]

On 13-06-28 01:56 PM, Duncan Murdoch wrote:
> On 28/06/2013 12:42 PM, Ben Bolker wrote:
>> Duncan Murdoch  gmail.com> writes:
>>
>> >
>>
>> [snip]
>> >
>> > The other choice is to use the explicit ::, i.e.
>> >
>> > mc[[1]] <- quote(lmer::lFormula)
>> >
>> > This would be the solution I'd prefer, but you seem to have some
>> > prejudice against :: .
>> >
>>
>>I just thought that the whole point of using the machinery of
>> namespaces, environments, etc., was that we shouldn't need to
>> tell R explicitly what environment we were working in ...
> 
> And usually it works, just not always when you do surgery on
> expressions.  The snipped version worked, for instance.
> 
> Duncan Murdoch

  Fair enough.  I guess the problem is that I often look in core R
functions to figure out the right idioms to use, and it wasn't obvious
that this was a dangerous one ... it's fairly common and I don't see
warnings in the source code about it ...

  Ben

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Peter Meilstrup]

Well, once you begin messing with eval.parent(), you _are_ telling R what
environment to work in, and things have the potential to go wrong at that
point because you're telling R to work in environment you don't know
anything about. Having to be explicit with a :: is just compensating for an
earlier thing you "shouldn't need to" be doing with environments (in my
opinion.)


On Fri, Jun 28, 2013 at 9:42 AM, Ben Bolker  wrote:

> Duncan Murdoch  gmail.com> writes:
>
> >
>
> [snip]
> >
> > The other choice is to use the explicit ::, i.e.
> >
> > mc[[1]] <- quote(lmer::lFormula)
> >
> > This would be the solution I'd prefer, but you seem to have some
> > prejudice against :: .
> >
>
>   I just thought that the whole point of using the machinery of
> namespaces, environments, etc., was that we shouldn't need to
> tell R explicitly what environment we were working in ...
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Ben Bolker]

On 13-06-28 03:57 PM, Peter Meilstrup wrote:
> Well, once you begin messing with eval.parent(), you _are_ telling R
> what environment to work in, and things have the potential to go wrong
> at that point because you're telling R to work in environment you don't
> know anything about. Having to be explicit with a :: is just
> compensating for an earlier thing you "shouldn't need to" be doing with
> environments (in my opinion.)

  What would your preferred idiom be for this task (calling another
function with all of the arguments of the previous call, without
uglifying the call in the process)?  I read your e-mail about making the
comparison with S3 method dispatch, but to be honest I don't understand
it very well -- don't know if those particular techniques are applicable
here ... ?

  Ben

> 
> 
> On Fri, Jun 28, 2013 at 9:42 AM, Ben Bolker  > wrote:
> 
> Duncan Murdoch  gmail.com >
> writes:
> 
> >
> 
> [snip]
> >
> > The other choice is to use the explicit ::, i.e.
> >
> > mc[[1]] <- quote(lmer::lFormula)
> >
> > This would be the solution I'd prefer, but you seem to have some
> > prejudice against :: .
> >
> 
>   I just thought that the whole point of using the machinery of
> namespaces, environments, etc., was that we shouldn't need to
> tell R explicitly what environment we were working in ...
> 
> __
> R-devel@r-project.org  mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem with eval(..., parent.frame(1L)) when package is not loaded

[Peter Meilstrup]

On Fri, Jun 28, 2013 at 3:07 PM, Ben Bolker  wrote:

> On 13-06-28 03:57 PM, Peter Meilstrup wrote:
> > Well, once you begin messing with eval.parent(), you _are_ telling R
> > what environment to work in, and things have the potential to go wrong
> > at that point because you're telling R to work in environment you don't
> > know anything about. Having to be explicit with a :: is just
> > compensating for an earlier thing you "shouldn't need to" be doing with
> > environments (in my opinion.)
>
>   What would your preferred idiom be for this task (calling another
> function with all of the arguments of the previous call, without
> uglifying the call in the process)?  I read your e-mail about making the
> comparison with S3 method dispatch, but to be honest I don't understand
> it very well -- don't know if those particular techniques are applicable
> here ... ?
>
>   Ben
>

In the case of lmer you could do this:

delegate <- structure(NULL, class="delegate")

#' @S3method glmer delegate
glmer.delegate <- glmer

lmer <- function (formula, data, family = NULL, blahblahblah, ...) {
  if (!is.null(family)) {
UseMethod("glmer", delegate)
  }
  #...
}

###

Peter

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel