[Rd] minor windows dialogue box problem-"File/Change dir..." (PR#8430)

[andrew]

Full_Name: Andrew Corson
Version: 2.2.0
OS: win XP
Submission from: (NULL) (222.152.185.145)


If you are not using a mouse, you use the tab key to move between fields in a
dialogue box. If you go File/Change dir... and then press tab to move to cursor
from the path text box to the Browse button it appears that the focus moves
correctly, but when you press Enter to activate the browse button the resulting
action is the same as if you had used the mouse and clicked on OK instead.

If you enter some non-existant path in the path box and use the keys to select
the Browse button, you get the error "Unable to set '...' as working directory"

The problem was present in R 2.0.1, I have just upgraded to R 2.2.0 and it still
exists.

> version
 _  
platform i386-pc-mingw32
arch i386   
os   mingw32
system   i386, mingw32  
status  
major2  
minor2.0
year 2005   
month10 
day  06 
svn rev  35749  
language R

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] tree/graph data structure APIs?

[Andrew Piskorski]

What is the best code available for simple general purpose
manipulation of tree (and/or directed graph) data structures in R?

Looking through CRAN, I see a bunch of packages for stastical
regression trees, but nothing that seems to provide an API for
manipulating tree data structures.  What I'm looking for is something
along the lines of an R version of the Tcllib ::struct::tree API
available for Tcl:

  http://wiki.tcl.tk/210
  http://tcllib.sourceforge.net/doc/struct_tree.html
  http://tcllib.sourceforge.net/doc/graph.html

Thanks!

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] access to R parse tree for Lisp-style macros?

[Andrew Piskorski]

R folks, I'm curious about possible support for Lisp-style macros in
R.  I'm aware of the "defmacro" support for S-Plus and R discussed
here:

  http://www.biostat.wustl.edu/archives/html/s-news/2002-10/msg00064.html 

but that's really just a syntactic short-cut to the run-time use of
substitute() and eval(), which you could manually put into a function
yourself if you cared too.  (AKA, not at all equivalent to Lisp
macros.)  The mlocal() function in mvbutils also has seemingly similar
macro-using-eval properties:

  http://cran.r-project.org/src/contrib/Descriptions/mvbutils.html 
  http://www.maths.lth.se/help/R/.R/library/mvbutils/html/mlocal.html 

I could of course pre-process R source code, either using a custom
script or something like M5:

  http://www.soe.ucsc.edu/~brucem/samples.html
  
http://groups.google.com/group/comp.compilers/browse_thread/thread/8ece2f34620f7957/000475ab31140327

But that's not what I'm asking about here.  As I understand it,
Lisp-style macros manipulate the already-parsed syntax tree.  This
seems very uncommon in non-Lisp languages and environments, but some -
like Python - do have such support.  (I don't use Python, but I'm told
that its standard parser APIs are as powerful as Lisp macros, although
clunkier to use.)

Is implementing Lisp-style macros feasible in R?  Has anyone
investigated this or tried to do it?

What internal representation does R use for its parse tree, and how
could I go about manipulating it in some fashion, either at package
build time or at run time, in order to support true Lisp-style macros?

Whenever I try something like this in R:

  > dput(parse(text="1+2"))
  expression(1 + 2)

what I see looks exactly like R code - that '1 + 2' expression doesn't
look very "parsed" to me.  Is that really it, or is there some sort of
Scheme-like parse tree hiding underneath?  I see that the interactive
Read-Eval-Print loop basically calls R_Parse1() in "src/main/gram.c",
but from there I'm pretty much lost.

Also, what happens at package build time?  I know that R CMD INSTALL
generates binary *.rdb and *.rdx files for my package, but what do
those do exactly, and how do they relate to the REPL and R_Parse1()?

Finally, are there any docs describing the design and implementation
of the R internals?  Should I be looking anywhere other than the R
developer page here?:

  http://developer.r-project.org/

Thanks!

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Brainstorm: Alpha and Beta testing of R versions

[Andrew Robinson]

Hi Martin,

On Fri, Nov 04, 2005 at 09:58:47AM +0100, Martin Maechler wrote:
> [Mainly for R-foundation members; but kept in public for general
>  brainstorming...]

I'll take up the invitation to brainstorm.

As a user of R for a number of years, I'd really like to perform some
useful service.  I use a relatively obscure platform (FreeBSD) and I
can compile code.  I'd like to think that I'm in the target market for
beta testing :).  But, I'm timid.  I do not feel, in general, that R
core welcomes bug reports.

I think that there are several things that could be tried to encourage
more, and more useful, bug reports.

1) Put the following text on the *front page* of the tracking system, so
   that it is seen before the reader clicks on "New Bug Report":

"Before submitting a bug report, please read Chapter `R Bugs' of `The
R FAQ'. It describes what a bug is and how to report a bug.

If you are not sure whether you have observed a bug or not, it is a
good idea to ask on the mailing list R-Help by sending an e-mail to
r-help@stat.math.ethz.ch rather than submitting a bug report."

(BTW is this true also for alpha/beta testing?)

2) Try to use the structure of the reporting page to prompt good
   reporting.  On the report page, summarize the key points of
   identifying and reporting a bug in a checklist format.  Maybe even
   insist that the boxes be checked before allowing submission.
   Include seperate text boxes for description and sample code, to
   suggest that sample code is valued.

3) On either or both pages (and in FAQ), explain that thoughtful bug
   reports are valued and appreciated.  Further, explain that bug
   reports that do not follow the protocol are less valuable, and take
   more time.

4) Add checkboxes to the report page for alpha/beta.  (I suggest this
   for the purposes of marketing, not organization.)

5) On the report page, include hyperlinks to archived bug reports that
   were good.  Do likewise with some artificial bug reports that are
   bad.

6) Add an intermediate, draft step for bug submission, to allow
   checking.  If possible, include as part of this step an automated
   pattern matching call that identifies similarly texted bug reports,
   provides links to the reports, and invites a last-minute cross-check.

7) Keep a list of people who report useful bugs in alpha/beta phase on
   the website.  Many academics could point to it as evidence of
   community service.

> In order to discourage an increased number of non-bug reports we
> may have to also open a "hall of shame" though...

8) I'm sure that you're being ironic!  But I will take the point
   seriously, for what it's worth.  I think that humiliating
   submitters who haven't followed the protocol is deleterious.  It
   seems like almost every month we see someone get slapped on the
   wrist for not doing something the right way.  Of course, it's
   frustrating that people aren't following the posting guide.  But,
   why is that?  Where is the breakdown?  It might be interesting to
   try some follow-up (an exit interview!). If someone has failed to
   follow the protocol, perhaps we should try to find out why it was
   confusing, or if they just ignored it.

   The R-core is surrounded by, and serves, a community that comprises
   people who are not sufficiently good at what R-core does to be
   invited in to R-core. But, we're clearly interested in what R-core
   produces.  Please don't assume that bug submissions that do not
   follow the R protocol are the consequence of deliberate
   malfeasance.  

   To paraphrase Ian Fleming: Once is happenstance.  Twice is
   incompetence.  The third time, Mr. Bond, is enemy action. So, ...

9) Publicly thank bug reporters whether their reports are useful or
   not.  I just googled 'R-devel thank' and you figure prominently,
   Martin :).

Cheers

Andrew
-- 
Andrew Robinson
Senior Lecturer in Statistics   Tel: +61-3-8344-9763
Department of Mathematics and StatisticsFax: +61-3-8344-4599
University of Melbourne, VIC 3010 Australia
Email: [EMAIL PROTECTED]Website: http://www.ms.unimelb.edu.au

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Brainstorm: Alpha and Beta testing of R versions

[Andrew Robinson]

Martin, you're very welcome.  

> A general point about your suggestions:  You seem to assume that
> bug reports are typically entered via the R-bugs web interface

Yes, that was the premiss of my suggestions.  Perhaps to supplement
these ideas, bug.report() could be rewritten to prompt useful
information, much as prompt() does. And simple e-mail to
[EMAIL PROTECTED] could be filtered to only allow entries from R
core, and/or a list of registered beta testers.

> actually we have been thinking of moving to bugzilla -- if only
> Peter Dalgaard could find a smart enough person (even to be paid)
> who'd port all the old bug reports into the new format..

I see that this would be useful.  What are the challenges? If funding
is available, then I wonder if any of the linked organizations might
be suitable?

http://www.bugzilla.org/support/consulting.html

> As you've remarked below, I've expressed gratitude more than once
> for helpful bug reports.

Absolutely, and the effect is appreciated.  Sadly, the average user
struggles to distinguish between a helpful and an unhelpful bug report
before sending it.

> indeed I was, partly.  The point was just that if the bug
> reporting will be something like a challenge with prizes, we had
> to discourage too many entries {which would be made just to try
> to win (a|the) prize}.

Yes, I see that.  I really don't think that we need prizes; I really
do think that we need to create an environment that actively mitigates
against the kinds of error that you/we find so frustrating. A sort of
TQM strategy.

Cheers

Andrew
-- 
Andrew Robinson
Senior Lecturer in Statistics   Tel: +61-3-8344-9763
Department of Mathematics and StatisticsFax: +61-3-8344-4599
University of Melbourne, VIC 3010 Australia
Email: [EMAIL PROTECTED]Website: http://www.ms.unimelb.edu.au

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R thread safe

[Andrew Piskorski]

On Mon, Nov 07, 2005 at 07:57:28PM +0100, [EMAIL PROTECTED] wrote:

> I would like to accelerate my R computation by using parallel OpenMP
> compilers (e.g from Pathscale) on a 2-processor AMD server and I
> would like to know whether R is a tread safe library. The main

R is not thread safe, but others will have to tell you just how and
why not and what needs to be done to fix that (I don't know).

Does OpenMP require thread support, or can it alternately use multiple
processes plus System V shared memory?  Perhaps the latter would still
be an option even in R's currently non-thread-safe state.

Your approach is interesting.  Why do you want to do shared memory
parallel programming with OpenMP, rather than say message passing via
MPI or PVM?  Do you have a particularly fine-grained problem for which
message passing would be unsuitable?  Or are you just hoping to
automatically take advantage of both CPUs of your dual CPU workstation
without having to write any extra message passing code?

I haven't heard of anyone else doing shared memory programming with R.
But if instead you are interested in message passing, there are lots
of different tools and projects in R for that:

  http://cran.us.r-project.org/src/contrib/Descriptions/Rmpi.html
  http://cran.us.r-project.org/src/contrib/Descriptions/rpvm.html
  http://cran.us.r-project.org/src/contrib/Descriptions/papply.html
  http://cran.us.r-project.org/src/contrib/Descriptions/snow.html
  http://www.aspect-sdm.org/Parallel-R/
  http://cran.us.r-project.org/src/contrib/Descriptions/RScaLAPACK.html
  http://cran.us.r-project.org/src/contrib/Descriptions/taskPR.html
  http://cran.us.r-project.org/src/contrib/Descriptions/biopara.html
  http://www.omegahat.org/download/R/packages/CORBA.tar.gz

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] t() dropping NULL dimnames {was "all.equal() for mismatching names..."}

[Andrew Piskorski]

On Fri, Dec 02, 2005 at 05:56:31PM +0100, Martin Maechler wrote:

> BeT> x <- c(a = 1, b = 2)
> BeT> is.vector(x)
> BeT> as.vector(x)
> BeT> all.equal(x, as.vector(x)) ## FALSE

> BeT> However, in all versions of R in which I executed this example, the
> BeT> all.equal command returned TRUE which suggest that either the comment

My PR#8191 patch to all.equal() does fix that, e.g.:

  > x <- c(a = 1, b = 2) 
  > is.vector(x) 
  [1] TRUE 
  > all.equal(x, as.vector(x)) 
  [1] "names"  "for Target but not for Current" 
  > x 
  a b  
  1 2  
  > as.vector(x) 
  [1] 1 2 

> MM> We recently had the following posting on R-devel
> MM> https://stat.ethz.ch/pipermail/r-devel/2005-October/034962.html
> MM> (Subject: [Rd] all.equal() improvements (PR#8191))

> I'm testing the first part of Andy's proposition
> {the 2nd part was about making the result strings more informative for
>  the case where all.equal() does *not* return TRUE}.

Excellent, thank you for digging into this, Martin!

> t() drops dimnames when they are list(NULL,NULL) 
> and has been doing so at least since R version 1.0.0 :
> 
>  x <- cbind(1:2, 2:1); dimnames(x) <- list(NULL, NULL) 
>  identical(x, t(x))  ## -> FALSE !
>  str(t(x)) # "no dimnames" (i.e. dimnames(x) === NULL)
> 
> Now I'm looking into changing that one

Interesting.  FYI, my PR#8192 "subscripting sometimes loses names"
hack does NOT fix or change that, I get the same result as you do
above - t(x) is losing dimnames in that case.

  http://bugs.r-project.org/cgi-bin/R/wishlist?id=8192

S-Plus 6.2.1 (or at least my somewhat patched version of it) does not
seem to have that bug:

  > x <- cbind(1:2, 2:1); dimnames(x) <- list(NULL, NULL)  
  > identical(x, t(x)) 
  [1] T 

  > dimnames(x) 
  [[1]]: 
  character(0) 
  [[2]]: 
  character(0) 

  > dimnames(t(x)) 
  [[1]]: 
  character(0) 
  [[2]]: 
  character(0) 

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] extension to missing()? {was "hist() ... helpful warning? (PR#8376)"}

[Andrew Clausen]

Hi Martin,

On Mon, Dec 12, 2005 at 12:20:06PM +0100, Martin Maechler wrote:
> AndrewC> I didn't occur to me to check the plot.histogram()
> AndrewC> help page.  
> 
> [ even though it's prominently mentioned on  help(hist)  ?? ]

Yes.  I expected plot.histogram() was something that no-one ever
calls directly (and I still expect that!), and I expected hist()
to pass everything on to plot.histogram().

I guess I think the most elegant design would be to remove all
plotting functionality from hist(), and put all arguments into
plot.histogram().  Or make histogram objects store everything.
But, I expect you can't changed this now, for compatability / user 
familiarity reasons...

> AndrewC> Perhaps it might be helpful to document in the
> AndrewC> hist() help page which attributes are stored in the
> AndrewC> hist() object.  
> you mean the 'histogram' object.

Yes.

> Yes, that might be helpful; diffs against
>   https://svn.R-project.org/R/trunk/src/library/graphics/man/hist.Rd
> are welcome.

I added it to my R-TODO.  (I doubt I will be able to get to this
for about a year... I am at the oppressive beginning of an obnoxious
degree program!)

> and of course
> is.miss <- lapply(formals(), function(n) missing(n))
> ``works'' but trivially {why ?} and hence not usefully.

R's introspection capabilities are a little mysterious to me.

The missing() docs mention that it should be improved in future.
That might be a better long-term solution?

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R on Zaurus (was: Wish list)

[Andrew Robinson]

On Sun, Jan 01, 2006 at 11:08:42AM -0800, M. Edward (Ed) Borasky wrote:
> 
> Hasn't someone ported R to the Sharp Zaurus, 

Yes, Simon Pickering did this.  I ran his version of R on a Zaurus two
years ago.  His site implies that development is still underway.  

http://people.bath.ac.uk/enpsgp/Zaurus/index.html

> I'm not sure how well a number crunching application like R would
> run on the Zaurus processor, though -- IIRC the floating point is
> emulated in software.

It was slow :).  Also I never got it to work with X, but in fairness I
didn't really try hard.  Simon's site implies that improvements are
being implemented.

Cheers

Andrew
-- 
Andrew Robinson
Senior Lecturer in Statistics   Tel: +61-3-8344-9763
Department of Mathematics and StatisticsFax: +61-3-8344-4599
University of Melbourne, VIC 3010 Australia
Email: [EMAIL PROTECTED]Website: http://www.ms.unimelb.edu.au

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Using STL containers in R/C++

[Andrew Finley]

Hi All,
I am in the process of writing an R extension in c++ and am using several
STL containers  (e.g., vector, map, multimap).  I make sure to clear all these containers at the end of the
.Call.   Everything compiles and runs just fine, but I'm a bit worried
since I haven't found any other packages that use STL.

So, my question: is it OK to use the STL containers in my code?  Will the
use of this library somehow limit portability of this package? Or cause
memory management problems?

Thanks-
Andy


Andrew Finley, Research Fellow
Department of Forest Resources
College of Natural Resources
University of Minnesota
305 Green Hall
1530 Cleveland Avenue N.
St. Paul, MN 55108

Ph 612-624-1714 office
http://blue.fr.umn.edu/home
www.cnr.umn.edu/FR/people/facstaff/finley/index.html

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] strange lme errors (PR#8477)

[Andrew Robinson]

I have seen this error before. I think that you need to upgrade gcc.
See

http://tolstoy.newcastle.edu.au/R/help/05/08/10820.html

I hope that this helps.

Andrew

On Thu, Jan 12, 2006 at 11:50:56AM +0100, [EMAIL PROTECTED] wrote:
> Full_Name: Wilhelm Bernhard Kloke
> Version: R-2.2.1
> OS: FreeBSD-5.3-i386
> Submission from: (NULL) (195.253.22.63)
> 
> 
> Since 2.2.0 I am getting strange lme errors like
> 
> > lme(ampl ~ gapf * bl,hframe2,~1|VP)
> Fehler in lme.formula(ampl ~ gapf * bl, hframe2, ~1 | VP) : 
> See PORT documentation.  Code (27)
> 
> I have no clue how to understand this. The specification worked well until
> 2.1.0.
> I tried a second dataset with the same error message.
> 
> At least the said "PORT documentation" should be included in the R 
> distribution,
> just in case.
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Andrew Robinson  
Department of Mathematics and StatisticsTel: +61-3-8344-9763
University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599
Email: [EMAIL PROTECTED] http://www.ms.unimelb.edu.au

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] natural sorting

[Andrew Piskorski]

On Wed, Jan 11, 2006 at 05:45:10PM -0500, Gabor Grothendieck wrote:
> It would be nifty to incorporate this into R or into an R package:
> 
> http://sourcefrog.net/projects/natsort/

Btw, I haven't looked at the implementation, but Tcl also contains
equivalent functionality, they call it dictionary sort:

  http://tcl.activestate.com/man/tcl8.4/TclCmd/lsort.htm

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Rprintf loop status does not print under windows

[Andrew Finley]

Hello,
I am writing a c/c++ extension package that does some mcmc sampling, and
periodically writes the sampling status to the terminal via Rprintf.  So in
my sampling loop I have:

if(status == 100){
  Rprintf("%i...", s);
  status = 0;
}
status++;

Under linux/unix this works fine, but under windows the status is not
printed.  Am I missing something?

Thanks-
Andy

-- 
Andrew Finley, Research Fellow
Department of Forest Resources
College of Natural Resources
University of Minnesota
305 Green Hall
1530 Cleveland Avenue N.
St. Paul, MN 55108

Ph 612-624-1714 office
http://blue.fr.umn.edu/home

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [tryExcept] New try Function

[Andrew Redd]

I have two related packages that are already submitted to CRAN but are
awaiting approval.  They are part of the R Documentation Task Force efforts
to improve documentation.  The exact function you are referring to I have
called `catch_condition()` and included it in my `testextra` package.  You
might also want to check out the `pkgcond` package which facilitates
creating informative conditions (errors, warnings, and messages).  These
conditions are automatically classed to tell you where the error is coming
from, the package and function, including class for reference methods.

https://github.com/RDocTaskForce/testextra
https://github.com/RDocTaskForce/pkgcond

On Fri, Nov 23, 2018 at 7:48 AM Ernest Benedito 
wrote:

> Hi Emil,
>
> First, thanks for the response. As you mentioned, a lot of times tryCatch
> does the work, as you can return a value. However, sometimes it is useful
> to assign several variables when an error occurs. You could do it with <<-,
> but I prefer to reduce it's usage unless completely necessary.
>
> I guess that the attachment was missed in the moderation. Here it is the
> function:
>
> tryExcept <- function (expr,
>except = {})
> {
>   doTryExcept <- function(expr, parentenv) {
> .Internal(.addCondHands("error", list(NULL), parentenv,
> environment(), FALSE))
> expr
>   }
>   parentenv <- parent.frame()
>   doTryExcept(return(expr), parentenv)
>   invisible(except)
> }
>
> As you can see, the tryExcept function uses a simplified version of the
> tryCatch architecture, but it allows you to pass by a second expression
> that is evaluated in case an error occurs during the evaluation of the
> first expression. It could even work as an infix operator:
>
> `%except%` <- tryExcept
>
> # dummy example
> {foo <- "foo"} %except% {foo <- "foo bar"}
> print(foo) # "foo"
>
> { foo <- "foo"
>   stop()
> } %except% {
>   foo <- "foo bar"
> }
> print(foo) # "foo bar"
>
> It's main downside is that you are not able to handle the error occured,
> although there is the possibility to add a 'silent' parameter such as in
> 'try' in order to print the error if desired. All in all, this would be a
> function for simple error handling, but I think it would be practical, and
> you can always move to tryCatch if you need a more complex error handling.
>
> I will be looking forward to hearing your insights.
>
> Best,
> Ernest Benedito
>
> Missatge de Emil Bode  del dia dv., 23 de nov.
> 2018
> a les 13:17:
>
> > Hi Ernest,
> >
> > To start: I don't see an attachment, I think they're not (always) allowed
> > on this mailing-list. If you want to send something, text is your safest
> > bet.
> > But regarding the issue of tryCatch: I think you're not fully using what
> > it already can do. In almost all circumstances I've encountered the
> > following works fine:
> > res <- tryCatch(expr, error = function(cond) {
> >   # a bunch of code
> >   # Some value to be stored in res
> > })
> > The only difference is that now "#abunchofcode" is run from inside a
> > function, which means you're working in a different environment, and if
> you
> > want to assign values to other variables you need to use <<- or assign.
> > For a modified function, I think it would be nice if there's a way to
> > supply an expression instead of a function, so that evaluation (and
> > assignment!) takes place in the same environment as the main code in the
> > tryCatch (in expr). Is that what you made?
> > And with the current tryCatch, you could use something like this:
> > res <- tryCatch(expr, error=function(e) evalq({
> >   # a bunch of code
> >   # Some value for res
> > }, envir=parent.frame(4))) # The 4 is because some internal functions are
> > involved, parent.frame(4) is the same environment as used by expr
> >
> > Although this is cumbersome, and it gets even more cumbersome if you want
> > to access the error-object in #abunchofcode, or use #abunchofcode to
> return
> > to a higher level, so I get it you're looking for a more elegant
> solution.
> >
> > Best regards,
> > Emil Bode
> >
> > On 23/11/2018, 08:49, "R-devel on behalf of Ernest Benedito" <
> > r-devel-boun...@r-project.org on behalf of ebenedi...@gmail.com> wrote:
> >
> > Hi everyone,
> >
> > When dealing with errors, sometimes I want to run a bunch of code
> when
> > an error occurs.
> > For now I usually use a structure such as:
> >
> > res <- tryCatch(expr, error = function(cond) cond) # or try(expr)
> >
> > if (inherits(res, “error”)) # or inherits(res, “try-error”)
> >   # a bunch of code
> >
> > I though it would be useful to have a function that does this
> > naturally, so I came up with the attached function.
> >
> > I would be glad to hear your insights and if you think it would make
> > sense to add this function to R.
> >
> > Best regards,
> > Ernest
> > __
> > R-devel@r-project.org mailing list
> > htt

Re: [Rd] default for 'signif.stars'

[Andrew Robinson]

Hi Martin,

I take your point - but I'd argue that significance stars are a clumsy
solution to the very real problem that you outline, and their inclusion as
a default sends a signal about their appropriateness that I would prefer R
not to endorse.

My preference (to the extent that it matters) would be to see the
significance stars be an option but not a default one, and the addition of
different functionality to handle the many-predictor problem, perhaps a new
summary that more efficiently provides more useful information.

If we were to invent lm() now, how would we solve the problem of big P?  I
don't think we would use stars.

Cheers,

Andrew




On Thu, 28 Mar 2019 at 20:19, Martin Maechler 
wrote:

> >>>>> Lenth, Russell V
> >>>>> on Wed, 27 Mar 2019 00:06:08 + writes:
>
> > Dear R-Devel, As I am sure many of you know, a special
> > issue of The American Statistician just came out, and its
> > theme is the [mis]use of P values and the many common ways
> > in which they are abused. The lead editorial in that issue
> > mentions the 2014 ASA guidelines on P values, and goes one
> > step further, by now recommending that the words
> > "statistically significant" and related simplistic
> > interpretations no longer be used. There is much
> > discussion of the problems with drawing "bright lines"
> > concerning P values.
>
> > This is the position of a US society, but my sense is that
> > the statistical community worldwide is pretty much on the
> > same page.
>
> > Meanwhile, functions such as 'print.summary.lm' and
> > 'print.anova' have an argument 'signif.stars' that really
> > does involve drawing bright lines when it is set to
> > TRUE. And the default setting for the "show.signif.stars"
> > option is TRUE. Isn't it time to at least make
> > "show.signif.stars" default to FALSE? And, indeed, to
> > consider deprecating those 'signif.stars' options
> > altogether?
>
> Dear Russ,
> Abs has already given good reasons why this article may well be
> considered problematic.
>
> However, I think you and (many but not all) others who've raised
> this issue before you, slightly miss the following point.
>
> If p-values are misleading they should not be shown (and hence
> the signif.stars neither.
> That has been the approach adopted e.g., in the lme4 package
> *AND* has been an approach originally used in S and I think
> parts of R as well, in more places than now, notably, e.g., for
> print( summary() ).
>
> Fact is that users will write wrappers and their own packages
> just to get to p values, even in very doubtful cases...
> But anyway that (p values or not) is a different discussion
> which has some value.
>
> You however focus on the "significance stars".  I've argued for
> years why they are useful, as they are just a simple
> visualization of p values, and saving a lot of human time when
> there are many (fixed) effects looked at simultaneously.
> Why should users have to visually scan 20 or 50 numbers?  In
> modern Data analysis they should never have to but rather look
> at a visualization of those numbers. ... and that's what
> significance stars are, not more, nor less.
>
> Martin
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Porting R example datasets to GNU Octave

[Andrew Janke]

Hi, R Developers,

I'm interested in porting the R example datasets package to GNU Octave
and Matlab. Would you have objections to my doing so?

This would involve transforming the example data and metadata into a
format that Octave understands, and porting all of the datasets' Example
code pieces to Octave M-code. (This would require no work on your part;
it'd be my project.)

I think this would be a benefit to the scientific programming community.
In addition to helping Octave users, having code for identical example
data sets in both languages would serve as a Rosetta Stone for not only
users moving from R to Octave, but for users coming from Octave or
Matlab to R.

Since R's datasets package is GPL, I think I'd be within my rights to
just do this. But I wanted to ask first, to make sure I didn't ruffle
any feathers. I would include documentation indicating that R is the
original source (well, intermediate source) for these datasets, and have
links pointing back to R's documentation.

Cheers,
Andrew Janke

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Incorrect date range in austres example dataset?

[Andrew Janke]

Hi, R developers,

It seems there might be an issue with the "austres" example dataset in
the "datasets" package. The description in austres.Rd says it's
"measured quarterly from March 1971 to March 1994". But there are only
89 observations in the data as defined in the source code. By my count,
that only brings you up to about March 1993. Is there an issue with the
data transcription, or the Description?

I'm looking at the source code from the R 3.6.0 source distribution.

Cheers,
Andrew Janke

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Porting R example datasets to GNU Octave

[Andrew Janke]

On 5/6/19 7:47 AM, Dirk Eddelbuettel wrote:
>
> On 5 May 2019 at 10:47, Andrew Janke wrote:
> | I'm interested in porting the R example datasets package to GNU Octave
> | and Matlab. Would you have objections to my doing so?
>
> You don't even have to ask...
>
> [...]
>
> | Since R's datasets package is GPL, I think I'd be within my rights to
> | just do this. But I wanted to ask first, to make sure I didn't ruffle
> | any feathers. I would include documentation indicating that R is the
> | original source (well, intermediate source) for these datasets, and have
> | links pointing back to R's documentation.
>
> That is the right way to do that. Respect both copyright (citing and
> referencing source) and licensing (by picking a license compatible
> with GPL 2
> or later; many of us just prefer to stick to GPL which Octave uses too).
>
> Dirk, in no way speaking for R Core but just handing out his $0.02

Thanks Dirk!

I've ported many of the datasets over to Octave code in my "Tablicious"
project.

To preserve credit, I've:
- propagated the "Copyright (C) 1995-2007 R Core Team" copyright
statement to the copyright headers in all the individual files for the
Octave code, since GNU Octave does per-file copyright headers
- added a "This is based on the  dataset from R's datasets
package" comment to each individual dataset source file
- added a section to the user manual mentioning that they came from R:
https://apjanke.github.io/octave-tablicious/doc/tablicious.html#Data-Sets-from-R

My project is GPL3+, so it's license compatible.

If you're curious to see how it turned out, the source is at
https://github.com/apjanke/octave-tablicious under
https://github.com/apjanke/octave-tablicious/tree/master/inst/%2Boctave/%2Binternal/%2Bdatasets
and
https://github.com/apjanke/octave-tablicious/tree/master/inst/%2Boctave/%2Bexamples/%2Binternal/%2Bdatasets.

R seems to have an edge on Octave in terms of plotting capabilities, so
many of the code examples in my port are just "TODO: Port this plot type
to Octave".

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Adding Tcl source to an R package

[Andrew Piskorski]

On Tue, Aug 10, 2010 at 04:13:18PM -0400, Adrian Waddell wrote:

> >> My Question: Where can I put my procedures in a *.tcl file within my
> >> package structure and how do I make my R package to load the Tcl
> >> procedures? And will these functions be within a Tcl namespace?

Whether you use Tcl namespaces or not is up to you and your Tcl code,
AFAICT R's Tcl interface has nothing to do with it.

> I have to load the tcl file within my R Code at some point with
> 
> tcl("source", tclfile)

"have to" is much too strong, there are certainly other ways.  E.g.,
help(".Tcl") specifically talks about addTclPath() and tclRequire(),
which are R interfaces to Tcl's "package require" and its "auto_path"
global variable:

  http://tcl.activestate.com/man/tcl8.4/TclCmd/package.htm
  http://tcl.activestate.com/man/tcl8.4/TclCmd/library.htm#M24

Remember that those are just helper functions to make combining R and
Tcl more convenient.  .Tcl() lets you execute any Tcl code you want,
so you could accomplish all the Tcl-side manipulation you need using
just it.

Some of the other R helper functions are doing more than just adding a
little syntactic sugar though.  For example, tclVar() and tclObj() are
very handy:

  input <- tclVar()
  # This nicely sets the Tcl list structure from the R vector: 
  tclObj(input) <- as.tclObj(my.vector ,drop=FALSE) 

Btw, it's easier to write Tcl code using variable names you know at
the time you write the code rather than the run-time determined
variable names that tclVar() gives you.  However, the implementation
of tclVar() is nicely simple, and if you look you'll see how to make
your own version that uses whatever Tcl variable name you want.

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] must .Call C functions return SEXP?

[Andrew Piskorski]

For using R's .Call interface to C functions, all the examples I've
seen have the C function return type SEXP.  Why?  What does R actually
do with this return type?  What happens if I *don't* return a SEXP?

Reason I ask, is I've written some R code which allocates two long
lists, and then calls a C function with .Call.  My C code writes to
those two pre-allocated lists, thus, I thought I should NOT need to
return any SEXP from my C function.  However, if I do that, it
segfaults somewhere in "src/main/memory.c".

If I instead return the SEXP for one of my two result lists it appears
to work fine...  But of course I don't understand what's going on here
and so don't trust it.  The list SEXP I am returning was allocated by
R, not by my C code, so why does it make any difference whether my C
function returns it or not?

>From "src/main/names.c" I see that .Call is implemented by do_dotcall
in "src/main/dotcode.c".  I don't necessarily understand what that
really does, but AFAICT if my C function returns nothing, the
do_dotcall's retval should simply remain set to R_NilValue, which
should be fine.

I must be misunderstanding something here, but I don't know what, and
would definitely appreciate any help.  Thanks!


My R code looks like this:

  result.1 <- vector("list" ,1e6)
  result.2 <- vector("list" ,1e6)
  .Call("my_C_function", result.1, result.2, other.input)

My C code looks like this:

  SEXP result_v; 
  result_v = Rf_allocVector(REALSXP, 5); 
  SET_VECTOR_ELT(result_list_1, k1, result_v); 
  REAL(result_v)[0] = some_number; 
  REAL(result_v)[1] = another_number; 
  /* Also do the same sort of thing for result_list_2. */
  return(result_list_1);  /* Appears to work ok. */
  /* return; */  /* Segfaults. */

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] must .Call C functions return SEXP?

[Andrew Piskorski]

On Thu, Oct 28, 2010 at 12:15:56AM -0400, Simon Urbanek wrote:

> > Reason I ask, is I've written some R code which allocates two long
> > lists, and then calls a C function with .Call.  My C code writes to
> > those two pre-allocated lists,

> That's bad! All arguments are essentially read-only so you should
> never write into them! 

I don't see how.  (So, what am I missing?)  The R docs themselves
state that the main point of using .Call rather than .C is that .Call
does not do any extra copying and gives one direct access to the R
objects.  (This is indeed very useful, e.g. to reorder a large matrix
in seconds rather than hours.)

I could allocate the two lists in my C code, but so far it was more
convenient to so in R.  What possible difference in behavior can there
be between the two approaches?

> R has pass-by-value(!) semantics, so semantically you code has
> nothing to do with the result.1 and result.2 variables since only
> their *values* are guaranteed to be passed (possibly a copy).

Clearly C code called from .Call must be allowed to construct R
objects, as that's how much of R itself is implemented, and further
down, it's what you recommend I should do instead.

But why does it follow that C code must never modify an object
initially allocated by R code?  Are you saying there is some special
magic difference in the state of an object allocated by R's C code
vs. one allocated by R code?  If so, what is it?

What is the potential problem here, that the garbage collector will
suddenly run while my C code is in the middle of writing to an R list?
Yes, if the gc is going to move the object elsewhere, that would be
very bad.  But it looks to me like that cannot happen, because lots of
the R implementation itself would fail badly if it did.

E.g.:  The PROTECT call is used to increment reference counts, but I
see no guarantees that it is atomic with the operations that allocate
objects.  I see no mutexes or other barriers in C code to prevent the
gc from running, thus implying that it *can't* run until the C
function completes.

And R is single threaded, of course.  But what about signal handlers,
could they ever invoke R's gc?

Also, I was initially surprised not to find any matrix C APIs, but
grepping for examples (sorry, I don't remember exactly which
functions) showed me that the apparently accepted way to do matrix
operations from C is to simply assume R's column-first dense matrix
order, and access the 2D matrix as a flat 1D vector.  (Which is easy.)

> The fact that internally R attempts to avoid copying for performance
> reasons is the only reason why your code may have appeared to work,
> but it's invalid!

I will probably change my code to allocate a new list from the C code
and return that, as you recommend.  My main reason for doing the
allocation in R was just that it was simpler, especially given the
very limited documentation of R's C API.

But, I didn't see anything in the "Writing R Extensions" doc saying
that what my code is doing is "invalid", and more importantly, I don't
see why it would or should be invalid...

I'd still like to better understand why you think doing the initial
allocation of an object in R rather than C code is such a problem.  So
far, I don't see any way that the R interpreter could ever tell the
difference.

Wait, or is the only objection here that I'm using C in a way that
makes pass-by-reference semantics visible to my R code?  Which will
work completely correctly, but is not the The Proper R Way?

I don't actually need pass-by-reference behavior here at all, but I
can imagine cases where I might want it, so I'd like to understand
your objections better.  Is using C to implement pass-by-reference
actually Broken, or merely Ugly?  From my reasons above, I think it
will always work correctly and thus is not Broken.  But of course
given R's devotion to pass-by-value, it could be considered
unacceptably Ugly.

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] must .Call C functions return SEXP?

[Andrew Piskorski]

On Thu, Oct 28, 2010 at 10:59:26AM -0400, Simon Urbanek wrote:

> Because R assumes that you don't mess with the arguments, it also
> optimizes b to point to the same object as a which you then modify.

> I hope it sheds some light on it.

Indeed!  Thank you Simon, that was a thorough and illuminating answer.

(Now I have better understanding of how the machine works, and what
the dangerous sharp bits are.  :)

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] BLAS benchmarks on R 2.12.0

[Andrew Piskorski]

On Sun, Oct 31, 2010 at 12:41:24PM -0400, Michael Spiegel wrote:

> 1) Compile the reference BLAS implementation with unsafe optimizations
> and include it as a part of the OpenMx library.

If BLAS speed is important to you, why are you even trying to use the
slow reference BLAS library at all, rather than one of the faster
optimized BLAS libraries (Atlas, Goto, AMD, Intel, etc.)?

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Segmentation Fault when using CUDA

[Andrew Redd]

I'm developing packages with CUDA, and I'm running into a problem with
memory allocation.  Since CUDA involves memory on the GPU device it
requires that the program handle memory allocation and deallocation
separately from R.

The simplest case of allocating a char and then deallocating causes a
segmentation fault when R closes.  Not on garbage collection but only
on exit.  Is there anything in the R internals that explain why this
is happening?

The relevant C++ code is
---
void gpualloctest(){
   char * _a;
   cudaMalloc(&_a,sizeof(char));
   cudaFree(_a);
   _a=NULL;
}
--- from gpu.cu
gputest<-function(){
   cat("testing allocation on gpu\n")
   Module('test','gputest')$gpualloctest()
   cat("Test successful\n")
   cat("Collecting Garbage\n")
   gc()
   cat("done.\n")
}
--- from gputest.R

As you can see Rcpp is used to interface with the C++ code, but the
Rcpp is not the problem, but happened to be convenient to use to get
the configure scripts right for the CUDA cu files.

The results from running "valgrind Rscript testgpu.R" are linked to in
testgpu.log but do not show anything interesting.

Can anyone give any insight?

Thanks,
Andrew Redd

Links to files:
http://andrewredd.us/gputest/testgpu.R
http://andrewredd.us/gputest/testgpu.Rout
http://andrewredd.us/gputest/gputest_1.0.tar.gz
http://andrewredd.us/gputest/testgpu.log

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] unloading compiled code.

[Andrew Redd]

I have a package that I'm developing that I need to unload the
library.  Long story short I figured out that the leaving the compiled
code loaded lead to a segmentation fault, but unloading the code will
fix it.  I've read the documentation and it appears that there are
several ways to do this?  What is the popper accepted current standard
for unloading compiled code?

The options as I understand them are:
1. dyn.unload
2. library.dynam.unload
used with either
A. .Last.lib
B. .onUnload

If it makes a difference my package does use a NAMESPACE so the
package is loaded through useDynLib.

Thanks,
Andrew Redd

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unloading compiled code.

[Andrew Redd]

Perhaps you could help me make some sense of this.  Here is a printout
of my sessions.
---
toys$R -q
> library(test2)
> gpualloctest()
testing allocation on gpu
C Finished
Collecting Garbage
done.
> q()
Save workspace image? [y/n/c]: n

 *** caught segfault ***
address 0x7f12ec1add50, cause 'memory not mapped'

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 1
aborting ...
Segmentation fault
toys$R -q
> library(test2)
> gpualloctest()
testing allocation on gpu
C Finished
Collecting Garbage
done.
> library.dynam.unload('test2',system.file(package='test2'))
> q()
Save workspace image? [y/n/c]: n
toys$
---

I have a in the test2/R/zzz.R file
---
.onUnload <- function(libpath)
library.dynam.unload("test2", libpath)
---

so the code should be unloaded.  But it appears that it is not from
errors when I explicitly unload the test2.so it does not through a
segfault.  Why would this be happening?  and how do I circumvent it.

thanks,
Andrew


On Fri, Nov 12, 2010 at 3:32 PM, Prof Brian Ripley
 wrote:
> On Fri, 12 Nov 2010, Andrew Redd wrote:
>
>> I have a package that I'm developing that I need to unload the
>> library.  Long story short I figured out that the leaving the compiled
>> code loaded lead to a segmentation fault, but unloading the code will
>> fix it.  I've read the documentation and it appears that there are
>> several ways to do this?  What is the popper accepted current standard
>> for unloading compiled code?
>
> Depends how you loaded it: you basically reverse the process.
>
>> The options as I understand them are:
>> 1. dyn.unload
>> 2. library.dynam.unload
>> used with either
>> A. .Last.lib
>> B. .onUnload
>>
>> If it makes a difference my package does use a NAMESPACE so the
>> package is loaded through useDynLib.
>
> So you need an .onUnload action calling library.dynam.unload.
>
> Slightly longer version: you need the DLL loaded whilst the namepace is in
> use, so it has to be in .onUnload, and useDynLib calls library.dynam so you
> need library.dynam.unload to do the housekeeping around dyn.unload which
> matches what library.dynam does around dyn.load.
>
> There are quite a lot of examples to look at, including in R itself. MASS is
> one example I've just checked.
>
> Having said all that, my experience is that unloading the DLL often does not
> help if you need to load it again (and that is why e.g. tcltk does not
> unload its DLL).
>
>> Thanks,
>> Andrew Redd
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> --
> Brian D. Ripley,                  rip...@stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unloading compiled code.

[Andrew Redd]

Are packages unloaded on quit so that the .Last.lib or .onUnload are
called for packages?

-Andrew

On Fri, Nov 12, 2010 at 3:52 PM, Andrew Redd  wrote:
> Perhaps you could help me make some sense of this.  Here is a printout
> of my sessions.
> ---
> toys$R -q
>> library(test2)
>> gpualloctest()
> testing allocation on gpu
> C Finished
> Collecting Garbage
> done.
>> q()
> Save workspace image? [y/n/c]: n
>
>  *** caught segfault ***
> address 0x7f12ec1add50, cause 'memory not mapped'
>
> Possible actions:
> 1: abort (with core dump, if enabled)
> 2: normal R exit
> 3: exit R without saving workspace
> 4: exit R saving workspace
> Selection: 1
> aborting ...
> Segmentation fault
> toys$R -q
>> library(test2)
>> gpualloctest()
> testing allocation on gpu
> C Finished
> Collecting Garbage
> done.
>> library.dynam.unload('test2',system.file(package='test2'))
>> q()
> Save workspace image? [y/n/c]: n
> toys$
> ---
>
> I have a in the test2/R/zzz.R file
> ---
> .onUnload <- function(libpath)
>    library.dynam.unload("test2", libpath)
> ---
>
> so the code should be unloaded.  But it appears that it is not from
> errors when I explicitly unload the test2.so it does not through a
> segfault.  Why would this be happening?  and how do I circumvent it.
>
> thanks,
> Andrew
>
>
> On Fri, Nov 12, 2010 at 3:32 PM, Prof Brian Ripley
>  wrote:
>> On Fri, 12 Nov 2010, Andrew Redd wrote:
>>
>>> I have a package that I'm developing that I need to unload the
>>> library.  Long story short I figured out that the leaving the compiled
>>> code loaded lead to a segmentation fault, but unloading the code will
>>> fix it.  I've read the documentation and it appears that there are
>>> several ways to do this?  What is the popper accepted current standard
>>> for unloading compiled code?
>>
>> Depends how you loaded it: you basically reverse the process.
>>
>>> The options as I understand them are:
>>> 1. dyn.unload
>>> 2. library.dynam.unload
>>> used with either
>>> A. .Last.lib
>>> B. .onUnload
>>>
>>> If it makes a difference my package does use a NAMESPACE so the
>>> package is loaded through useDynLib.
>>
>> So you need an .onUnload action calling library.dynam.unload.
>>
>> Slightly longer version: you need the DLL loaded whilst the namepace is in
>> use, so it has to be in .onUnload, and useDynLib calls library.dynam so you
>> need library.dynam.unload to do the housekeeping around dyn.unload which
>> matches what library.dynam does around dyn.load.
>>
>> There are quite a lot of examples to look at, including in R itself. MASS is
>> one example I've just checked.
>>
>> Having said all that, my experience is that unloading the DLL often does not
>> help if you need to load it again (and that is why e.g. tcltk does not
>> unload its DLL).
>>
>>> Thanks,
>>> Andrew Redd
>>>
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>> --
>> Brian D. Ripley,                  rip...@stats.ox.ac.uk
>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford,             Tel:  +44 1865 272861 (self)
>> 1 South Parks Road,                     +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>>
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unloading compiled code.

[Andrew Redd]

Just found in the documentation for getHook that packages are not
unloaded on quit.  How should I force a package to unload on quit?

-Andrew

On Tue, Nov 16, 2010 at 10:25 AM, Andrew Redd  wrote:
> Are packages unloaded on quit so that the .Last.lib or .onUnload are
> called for packages?
>
> -Andrew
>
> On Fri, Nov 12, 2010 at 3:52 PM, Andrew Redd  wrote:
>> Perhaps you could help me make some sense of this.  Here is a printout
>> of my sessions.
>> ---
>> toys$R -q
>>> library(test2)
>>> gpualloctest()
>> testing allocation on gpu
>> C Finished
>> Collecting Garbage
>> done.
>>> q()
>> Save workspace image? [y/n/c]: n
>>
>>  *** caught segfault ***
>> address 0x7f12ec1add50, cause 'memory not mapped'
>>
>> Possible actions:
>> 1: abort (with core dump, if enabled)
>> 2: normal R exit
>> 3: exit R without saving workspace
>> 4: exit R saving workspace
>> Selection: 1
>> aborting ...
>> Segmentation fault
>> toys$R -q
>>> library(test2)
>>> gpualloctest()
>> testing allocation on gpu
>> C Finished
>> Collecting Garbage
>> done.
>>> library.dynam.unload('test2',system.file(package='test2'))
>>> q()
>> Save workspace image? [y/n/c]: n
>> toys$
>> ---
>>
>> I have a in the test2/R/zzz.R file
>> ---
>> .onUnload <- function(libpath)
>>    library.dynam.unload("test2", libpath)
>> ---
>>
>> so the code should be unloaded.  But it appears that it is not from
>> errors when I explicitly unload the test2.so it does not through a
>> segfault.  Why would this be happening?  and how do I circumvent it.
>>
>> thanks,
>> Andrew
>>
>>
>> On Fri, Nov 12, 2010 at 3:32 PM, Prof Brian Ripley
>>  wrote:
>>> On Fri, 12 Nov 2010, Andrew Redd wrote:
>>>
>>>> I have a package that I'm developing that I need to unload the
>>>> library.  Long story short I figured out that the leaving the compiled
>>>> code loaded lead to a segmentation fault, but unloading the code will
>>>> fix it.  I've read the documentation and it appears that there are
>>>> several ways to do this?  What is the popper accepted current standard
>>>> for unloading compiled code?
>>>
>>> Depends how you loaded it: you basically reverse the process.
>>>
>>>> The options as I understand them are:
>>>> 1. dyn.unload
>>>> 2. library.dynam.unload
>>>> used with either
>>>> A. .Last.lib
>>>> B. .onUnload
>>>>
>>>> If it makes a difference my package does use a NAMESPACE so the
>>>> package is loaded through useDynLib.
>>>
>>> So you need an .onUnload action calling library.dynam.unload.
>>>
>>> Slightly longer version: you need the DLL loaded whilst the namepace is in
>>> use, so it has to be in .onUnload, and useDynLib calls library.dynam so you
>>> need library.dynam.unload to do the housekeeping around dyn.unload which
>>> matches what library.dynam does around dyn.load.
>>>
>>> There are quite a lot of examples to look at, including in R itself. MASS is
>>> one example I've just checked.
>>>
>>> Having said all that, my experience is that unloading the DLL often does not
>>> help if you need to load it again (and that is why e.g. tcltk does not
>>> unload its DLL).
>>>
>>>> Thanks,
>>>> Andrew Redd
>>>>
>>>> __
>>>> R-devel@r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>
>>> --
>>> Brian D. Ripley,                  rip...@stats.ox.ac.uk
>>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>>> University of Oxford,             Tel:  +44 1865 272861 (self)
>>> 1 South Parks Road,                     +44 1865 272866 (PA)
>>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>>>
>>
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unloading compiled code.

[Andrew Redd]

so should I use reg.finalizer or overwrite .Last()?  If I use
reg.finalizer, what should be the environment that I specify?  The
straight forward solution would be to have a hook .onExit that a
package could specify to make sure  that the code was unloaded before
the program terminates, that way I don't overwrite .Last if if has
another purpose.

-Andrew

On Tue, Nov 16, 2010 at 11:27 AM, Charles C. Berry  wrote:
> On Tue, 16 Nov 2010, Andrew Redd wrote:
>
>> Just found in the documentation for getHook that packages are not
>> unloaded on quit.  How should I force a package to unload on quit?
>
> See
>
>        ?q
>
>
> HTH,
>
> Chuck
>
>>
>> -Andrew
>>
>> On Tue, Nov 16, 2010 at 10:25 AM, Andrew Redd  wrote:
>>>
>>> Are packages unloaded on quit so that the .Last.lib or .onUnload are
>>> called for packages?
>>>
>>> -Andrew
>>>
>>> On Fri, Nov 12, 2010 at 3:52 PM, Andrew Redd  wrote:
>>>>
>>>> Perhaps you could help me make some sense of this.  Here is a printout
>>>> of my sessions.
>>>> ---
>>>> toys$R -q
>>>>>
>>>>> library(test2)
>>>>> gpualloctest()
>>>>
>>>> testing allocation on gpu
>>>> C Finished
>>>> Collecting Garbage
>>>> done.
>>>>>
>>>>> q()
>>>>
>>>> Save workspace image? [y/n/c]: n
>>>>
>>>>  *** caught segfault ***
>>>> address 0x7f12ec1add50, cause 'memory not mapped'
>>>>
>>>> Possible actions:
>>>> 1: abort (with core dump, if enabled)
>>>> 2: normal R exit
>>>> 3: exit R without saving workspace
>>>> 4: exit R saving workspace
>>>> Selection: 1
>>>> aborting ...
>>>> Segmentation fault
>>>> toys$R -q
>>>>>
>>>>> library(test2)
>>>>> gpualloctest()
>>>>
>>>> testing allocation on gpu
>>>> C Finished
>>>> Collecting Garbage
>>>> done.
>>>>>
>>>>> library.dynam.unload('test2',system.file(package='test2'))
>>>>> q()
>>>>
>>>> Save workspace image? [y/n/c]: n
>>>> toys$
>>>> ---
>>>>
>>>> I have a in the test2/R/zzz.R file
>>>> ---
>>>> .onUnload <- function(libpath)
>>>>    library.dynam.unload("test2", libpath)
>>>> ---
>>>>
>>>> so the code should be unloaded.  But it appears that it is not from
>>>> errors when I explicitly unload the test2.so it does not through a
>>>> segfault.  Why would this be happening?  and how do I circumvent it.
>>>>
>>>> thanks,
>>>> Andrew
>>>>
>>>>
>>>> On Fri, Nov 12, 2010 at 3:32 PM, Prof Brian Ripley
>>>>  wrote:
>>>>>
>>>>> On Fri, 12 Nov 2010, Andrew Redd wrote:
>>>>>
>>>>>> I have a package that I'm developing that I need to unload the
>>>>>> library.  Long story short I figured out that the leaving the compiled
>>>>>> code loaded lead to a segmentation fault, but unloading the code will
>>>>>> fix it.  I've read the documentation and it appears that there are
>>>>>> several ways to do this?  What is the popper accepted current standard
>>>>>> for unloading compiled code?
>>>>>
>>>>> Depends how you loaded it: you basically reverse the process.
>>>>>
>>>>>> The options as I understand them are:
>>>>>> 1. dyn.unload
>>>>>> 2. library.dynam.unload
>>>>>> used with either
>>>>>> A. .Last.lib
>>>>>> B. .onUnload
>>>>>>
>>>>>> If it makes a difference my package does use a NAMESPACE so the
>>>>>> package is loaded through useDynLib.
>>>>>
>>>>> So you need an .onUnload action calling library.dynam.unload.
>>>>>
>>>>> Slightly longer version: you need the DLL loaded whilst the namepace is
>>>>> in
>>>>> use, so it has to be in .onUnload, and useDynLib calls library.dynam so
>>>>> you
>>>>> need library.dynam.unload to do the housekeeping around dyn.unload
>>>>> which
>>>>> matches what library.dynam does around dyn.load.
>>>>>
>>>>> There are quite a lot of examples to look at, including in R itself.
>>>>> MASS is
>>>>> one example I've just checked.
>>>>>
>>>>> Having said all that, my experience is that unloading the DLL often
>>>>> does not
>>>>> help if you need to load it again (and that is why e.g. tcltk does not
>>>>> unload its DLL).
>>>>>
>>>>>> Thanks,
>>>>>> Andrew Redd
>>>>>>
>>>>>> __
>>>>>> R-devel@r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>>>
>>>>>
>>>>> --
>>>>> Brian D. Ripley,                  rip...@stats.ox.ac.uk
>>>>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>>>>> University of Oxford,             Tel:  +44 1865 272861 (self)
>>>>> 1 South Parks Road,                     +44 1865 272866 (PA)
>>>>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>>>>>
>>>>
>>>
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> Charles C. Berry                            Dept of Family/Preventive
> Medicine
> cbe...@tajo.ucsd.edu                        UC San Diego
> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901
>
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unloading compiled code.

[Andrew Redd]

I thought I would update with what I did.  Not liking overwriting the
.Last function in case another package I decided to see if the
reg.finalizer would work.  This is what my .onLoad function look like.
---
.onLoad<-function(libname, pkgname){
e<-emptyenv()
library.dynam('gpuBayes',package=pkgname)
reg.finalizer(e,function(...){unloadNamespace(pkgname)},T)
}
.onUnload<-function(libpath){
library.dynam.unload('gpuBayes',libpath)
}
---
 This takes attaches a reg.finalizer to the empty environment with
onexit=T.  Since the empty environment will never be collected this
will only run on exit.

-Andrew

On Tue, Nov 16, 2010 at 6:34 PM, Charles C. Berry  wrote:
> On Tue, 16 Nov 2010, Andrew Redd wrote:
>
>> so should I use reg.finalizer or overwrite .Last()?
>
>> .Last
>
> Error: object '.Last' not found
>>
>
> You create your own .Last - there is nothing to overwrite.
>
> Chuck
>
>
>  If I use
>>
>> reg.finalizer, what should be the environment that I specify?  The
>> straight forward solution would be to have a hook .onExit that a
>> package could specify to make sure  that the code was unloaded before
>> the program terminates, that way I don't overwrite .Last if if has
>> another purpose.
>>
>> -Andrew
>>
>> On Tue, Nov 16, 2010 at 11:27 AM, Charles C. Berry 
>> wrote:
>>>
>>> On Tue, 16 Nov 2010, Andrew Redd wrote:
>>>
>>>> Just found in the documentation for getHook that packages are not
>>>> unloaded on quit.  How should I force a package to unload on quit?
>>>
>>> See
>>>
>>>        ?q
>>>
>>>
>>> HTH,
>>>
>>> Chuck
>>>
>>>>
>>>> -Andrew
>>>>
>>>> On Tue, Nov 16, 2010 at 10:25 AM, Andrew Redd  wrote:
>>>>>
>>>>> Are packages unloaded on quit so that the .Last.lib or .onUnload are
>>>>> called for packages?
>>>>>
>>>>> -Andrew
>>>>>
>>>>> On Fri, Nov 12, 2010 at 3:52 PM, Andrew Redd  wrote:
>>>>>>
>>>>>> Perhaps you could help me make some sense of this.  Here is a printout
>>>>>> of my sessions.
>>>>>> ---
>>>>>> toys$R -q
>>>>>>>
>>>>>>> library(test2)
>>>>>>> gpualloctest()
>>>>>>
>>>>>> testing allocation on gpu
>>>>>> C Finished
>>>>>> Collecting Garbage
>>>>>> done.
>>>>>>>
>>>>>>> q()
>>>>>>
>>>>>> Save workspace image? [y/n/c]: n
>>>>>>
>>>>>>  *** caught segfault ***
>>>>>> address 0x7f12ec1add50, cause 'memory not mapped'
>>>>>>
>>>>>> Possible actions:
>>>>>> 1: abort (with core dump, if enabled)
>>>>>> 2: normal R exit
>>>>>> 3: exit R without saving workspace
>>>>>> 4: exit R saving workspace
>>>>>> Selection: 1
>>>>>> aborting ...
>>>>>> Segmentation fault
>>>>>> toys$R -q
>>>>>>>
>>>>>>> library(test2)
>>>>>>> gpualloctest()
>>>>>>
>>>>>> testing allocation on gpu
>>>>>> C Finished
>>>>>> Collecting Garbage
>>>>>> done.
>>>>>>>
>>>>>>> library.dynam.unload('test2',system.file(package='test2'))
>>>>>>> q()
>>>>>>
>>>>>> Save workspace image? [y/n/c]: n
>>>>>> toys$
>>>>>> ---
>>>>>>
>>>>>> I have a in the test2/R/zzz.R file
>>>>>> ---
>>>>>> .onUnload <- function(libpath)
>>>>>>    library.dynam.unload("test2", libpath)
>>>>>> ---
>>>>>>
>>>>>> so the code should be unloaded.  But it appears that it is not from
>>>>>> errors when I explicitly unload the test2.so it does not through a
>>>>>> segfault.  Why would this be happening?  and how do I circumvent it.
>>>>>>
>>>>>> thanks,
>>>>>> Andrew
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 12, 2010 at 3:32 PM, Prof Brian 

[Rd] datalist and data objects in R Package building

[andrew stewart]

Hello all,

I have,say 4 R objects...  bar1, bar2, bar3, bar4.. that I'd like to include
in an R package "foobar".

The desired functionality would be:

> library(foobar)
> data(foo)
> ls()
[1] "bar1" "bar2" "bar3" "bar4"

I've tried the following two approaches:

1) I created the file 'datalist' under pre-build directory 'foobar/data/'
with the following contents:
foo: bar1 bar2 bar3 bar4

After package build and install, "data(foo)" reports that data set 'foo' not
found (bar1, bar2, etc are all available individually, and are listed under
data() as "bar1 (foo)".

2) I created an image via save.image resulting in foo.rda (containing bar1,
bar2, etc).

data(foo) now loads bar1 - bar4, but 'foo' doesn't appear in the list of
available datasets displayed when trying to tab complete within data().


So my question is, what's the correct approach for what I'm trying to do
here?  Any advice welcome and appreciated.

Thanks,
Andrew

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] datalist and data objects in R Package building

[andrew stewart]

Thank you for your response.

Yes, using a call to data() after 1) building and 2) installing my own
package is exactly what I'm trying to accomplish.  I am building a package.
 I would like to load the data objects that are part of the custom package
that I have created and installed on my machine.  Apologies if I wasn't
clear about that part.


2011/3/28 Uwe Ligges 

>
>
> On 24.03.2011 16:51, andrew stewart wrote:
>
>> Hello all,
>>
>> I have,say 4 R objects...  bar1, bar2, bar3, bar4.. that I'd like to
>> include
>> in an R package "foobar".
>>
>> The desired functionality would be:
>>
>>  library(foobar)
>>> data(foo)
>>> ls()
>>>
>> [1] "bar1" "bar2" "bar3" "bar4"
>>
>> I've tried the following two approaches:
>>
>> 1) I created the file 'datalist' under pre-build directory 'foobar/data/'
>> with the following contents:
>> foo: bar1 bar2 bar3 bar4
>>
>> After package build and install, "data(foo)" reports that data set 'foo'
>> not
>> found (bar1, bar2, etc are all available individually, and are listed
>> under
>> data() as "bar1 (foo)".
>>
>
>
> If you want just one object "foo", then prpare a list
>
> foo <- list(bar1,...)
>
> that contains the 4 objects bar1, ... .
> You can load that objects and access the list components afterwards.
>
> I think you misunderstood the data concept: You can save objects and load
> them if the package is installed. That's it.
>
> Best,
> Uwe Ligges
>
>
>
>  2) I created an image via save.image resulting in foo.rda (containing
>> bar1,
>> bar2, etc).
>>
>> data(foo) now loads bar1 - bar4, but 'foo' doesn't appear in the list of
>> available datasets displayed when trying to tab complete within data().
>>
>>
>> So my question is, what's the correct approach for what I'm trying to do
>> here?  Any advice welcome and appreciated.
>>
>> Thanks,
>> Andrew
>>
>>[[alternative HTML version deleted]]
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Interrupting C++ code execution

[Andrew Runnalls]

Peter,

On 25/04/11 10:22, schattenpfla...@arcor.de wrote:

1. Calling R_CheckUserInterrupt() interrupts immediately, so I have no
possibility to exit my code gracefully. In particular, I suppose that
objects created on the heap (e.g., STL containers) are not destructed
properly.


Sorry not to have seen this thread sooner.

You may like to give CXXR a try 
(http://www.cs.kent.ac.uk/projects/cxxr/).  In CXXR the R interpreter is 
written in C++, and a user interrupt is handled by throwing a C++ 
exception, so the stack is unwound in an orderly fashion, destructors 
are invoked, etc.


However, it's fair to say that in using CXXR with a multi-threaded 
program you'll be on the bleeding edge...


Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] authorship and citation

[Andrew Robinson]

As a fourth option, I wonder if the first author could fork the package?

Presumably, appropriately cited, a fork is permitted by the license under
which it was released. Then the original package, by both authors, still
exists (and a final version could point to the new one) and the new
package, citing the previous version appropriately, is by a single author.

The page of CRAN's policies doesn't seem to touch on forking, presumably
because it's out of scope.

Best wishes,

Andrew



On Tue, Oct 6, 2015 at 8:22 AM, Uwe Ligges 
wrote:

> Simply advice:
>
> The former co-author contributed, so he is still author and probably
> copyright holder and has to be listed among the authors, otherwise it would
> be a CRAN policy violation since even if he does not develop further on, he
> developed parts of the so far existing package (if you talk about a CRAN
> package).
>
> I am not a lawyer, hence I cannot speak for copyright/license stuff in
> general, hence my comments only about CRAN policies.
>
>
> Best,
> Uwe Ligges
>
>
>
> On 05.10.2015 23:02, Adrian Dușa wrote:
>
>> Dear R developers,
>>
>> This is a rather peculiar question, but nevertheless I would still need an
>> answer for.
>> It is about an R package which I created (namely QCA), and from versions
>> 1.0-0 to 1.1-4 I had a co-author.
>> The co-author recently withdrawn from the package development, but still
>> requires to be left in the authors list and be cited for the package in
>> the
>> CITATION file.
>>
>> Obviously, one could not require citations for further developments, but
>> don't know how exactly to proceed (I would like to be fair and comply to
>> rules).
>>
>> I have three options:
>>
>> 1. Since the co-author withdrawn from the package development, erase his
>> name from the list of authors (but duly recognising his past contribution
>> in the package description file)
>>
>> 2. Preserve his name in the list of authors (with the comment "up to
>> version 1.1-4"), but erasing his name from the citation file
>>
>> 3. Keep his name both in the authors list and in the citation file
>> indefinitely, even though he doesn't do any development work anymore (I
>> have been threatened with a legal process for plagiarism if I did
>> otherwise).
>>
>> My gut feeling is, since his name is related to the previous versions,
>> anyone using those versions would cite him as well, but otherwise I don't
>> feel comfortable citing my former co-author for the current work he hasn't
>> contributed to.
>>
>> At this point, I would really use an advice, as on the other hand I
>> wouldn't want to break any regulation I might not be aware of.
>>
>> Best wishes,
>> Adrian
>>
>>
>>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



-- 
Andrew Robinson
Deputy Director, CEBRA, School of Biosciences
Reader & Associate Professor in Applied Statistics  Tel: (+61) 0403 138 955
School of Mathematics and StatisticsFax: +61-3-8344
4599
University of Melbourne, VIC 3010 Australia
Email: a.robin...@ms.unimelb.edu.au
Website: http://www.ms.unimelb.edu.au/~andrewpr

MSME: http://www.crcpress.com/product/isbn/9781439858028
FAwR: http://www.ms.unimelb.edu.au/~andrewpr/FAwR/
SPuR: http://www.ms.unimelb.edu.au/spuRs/

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Custom C finalizers for .Call

[Andrew Piskorski]

On Tue, Nov 24, 2015 at 12:10:12AM +0100, Jeroen Ooms wrote:

> Currently it is all to easy for package authors to introduce a memory
> leak or stack imbalance by calling Rf_error() or
> R_CheckUserInterrupt() in a way that skips over the usual cleanup
> steps.

I have a more modest request:  Please improve the documentation of
exactly what Rf_error() does and how it should be used!  E.g., does
calling it terminate execution of the C function from which it is
called, or not?  The docs below don't actually say:

  https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Error-handling
  https://cran.r-project.org/doc/manuals/r-devel/R-ints.html#Warnings-and-errors

The docs imply that these are "C-level equivalents" to the R stop()
function, which of course DOES terminate execution.  I believe
Rf_error() does also, but I really wish it was explicitly stated one
way or the other.

Grepping the R source, I never did find where Rf_error() is actually
implemented.  What am I missing?

In src/include/Rinternals.h, the implementation of error_return()
first calls Rf_error(msg) and then does a separate "return R_NilValue"
in the next statement.  So you might think that Rf_error() must NOT
terminate execution, otherwise why the extra return statement?  But it
also has a comment:

  /* return(.) NOT reached : for -Wall */

Meaning that that return statement is just to silence compiler
warnings, and is not supposed to be reached because Rf_error()
terminates execution.  But this is a ridiculously roundabout way to
infer what the behavior of Rf_error() is supposed to be...

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] failed to assign RegisteredNativeSymbol for splitString

[Andrew Piskorski]

I saw a warning from R that I don't fully understand.  Here's one way
to reproduce it:

  $ /usr/local/pkg/R-3.2-branch-20160718/bin/R --version | head -n 3 
  R version 3.2.5 Patched (2016-05-05 r70929) -- "Very, Very Secure Dishes" 
  Copyright (C) 2016 The R Foundation for Statistical Computing 
  Platform: x86_64-pc-linux-gnu/x86_64 (64-bit) 
   
  $ /usr/local/pkg/R-3.2-branch-20160718/bin/R --vanilla --no-restore --no-save 
--silent 
  > splitString <- function(...) { print("Test, do nothing") } 
  > invisible(tools::toTitleCase) 
  Warning message: 
  failed to assign RegisteredNativeSymbol for splitString to splitString since 
splitString is already defined in the 'tools' namespace  

Another way to trigger that warning is by loading the knitr package, e.g.:

  > require("knitr") 
  Loading required package: knitr 
  Warning: failed to assign RegisteredNativeSymbol for splitString to 
splitString since splitString is already defined in the 'tools' namespace 

The warning only happens the FIRST time I run any code that triggers it.
To get it to happen again, I need to restart R.

R 3.1.0 and all earlier versions do not throw that warning, because
they do not have any splitString C function (see below) at all.  R
3.2.5 does throw the warning, and I believe 3.3 and all later versions
of R do also (but I cannot currently test that on this machine).

In my case, normally I start R without "--vanilla", and load various
custom libraries of my own, one of which contained an R function
"splitString".  That gave the exact same symptoms as the simpler way
of reproducing the warning above.  In practice, I solved the problem
by renaming my "splitString" function to something else.  But I still
wonder what exactly was going on with that warning.

I noticed that the toTitleCase() R code calls .Call() with a bare
splitString identifier, no quotes around it:

  $ grep -n splitString R-3-[234]*/src/library/tools/R/utils.R
  R-3-2-branch/src/library/tools/R/utils.R:1988:xx <- 
.Call(splitString, x, ' -/"()')
  R-3-3-branch/src/library/tools/R/utils.R:2074:xx <- 
.Call(splitString, x, ' -/"()\n')
  R-3-4-trunk/src/library/tools/R/utils.R:2074:xx <- .Call(splitString, 
x, ' -/"()\n')

  $ find R-3-4-trunk -name .svn -prune -o -type f -print0 | xargs -0 grep -n 
splitString
  R-3-4-trunk/src/library/tools/R/utils.R:2074:xx <- .Call(splitString, 
x, ' -/"()\n')
  R-3-4-trunk/src/library/tools/src/text.c:264:SEXP splitString(SEXP string, 
SEXP delims)
  R-3-4-trunk/src/library/tools/src/tools.h:45:SEXP splitString(SEXP string, 
SEXP delims);
  R-3-4-trunk/src/library/tools/src/init.c:53:CALLDEF(splitString, 2),

Doing that is perfectly legal according to help(".Call"), and
interestingly, it apparently does NOT matter whether that code puts
quotes around the splitString or not - I tried it, and it made no
difference.

Is it generally the case the users MUST NOT define R functions with
the same names as "registered" C functions?  Will something break if
we do?

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] What happened to Ross Ihaka's proposal for a Common Lisp based R successor?

[Andrew Judson]

I read this paper
 and
haven't been able to find out what happened - I have seen some sporadic
mention in message groups but nothing definitive. Does anyone know?

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Announcing the R Documentation Task Force

[Andrew Redd]

cross-posting announcement to R-Announce, R-devel and R-package-devel.

The R Consortium recently announced 
(https://www.r-consortium.org/news/blogs/2016/08/r-consortium-funds-three-projects-july)
 
support of the R Documentation Task Force.  The task force aims to 
design and implement the next generation documentation system for R.  We 
aim to take the best from the many attempts to improve documentation and 
unify them into a system that will be more standard, flexible and 
powerful.  We have the support and participation of R Core Developers.  
The full proposal is available on the announcement website given above.

If you have expertise in documentation systems or documentation of 
objects or interest in building a documentation system we invite you to 
participate.  We will be considering documentation in all common forms 
such as function and class documentation, but also areas such as data 
frames, non-standard objects, packages, and how vignettes fit into the 
documentation framework.  If you have opinions or concerns that you 
would like to make sure are addressed please join.

To join send an email to Andrew dot Redd at hsc dot utah dot edu, 
expressing your interests, skills or expertise as it relates to R 
documentation.  Also email if you have ideas or concerns but do not wish 
to play and active role.

Thank you,
Andrew Redd
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Announcing the R Documentation Task Force

[Andrew Redd]

To clarify, the developers, Duncan Murdoch, Michael Lawrence, and Martin
Maechler, who are R Core Developers are participating in the project, but
this should not be construed to be an endorsement from the R Core
Developers as a whole, and the work of the R Documentation Task Force is
not guaranteed to be included in R Core.  My apologies to any who may have
misinterpreted the intentions of the announcement.

Sincerely,
Andrew Redd

On Fri, Sep 9, 2016 at 8:46 PM Andrew Redd  wrote:

> cross-posting announcement to R-Announce, R-devel and R-package-devel.
>
> The R Consortium recently announced
> (
> https://www.r-consortium.org/news/blogs/2016/08/r-consortium-funds-three-projects-july
> )
> support of the R Documentation Task Force.  The task force aims to
> design and implement the next generation documentation system for R.  We
> aim to take the best from the many attempts to improve documentation and
> unify them into a system that will be more standard, flexible and
> powerful.  We have the support and participation of R Core Developers.
> The full proposal is available on the announcement website given above.
>
> If you have expertise in documentation systems or documentation of
> objects or interest in building a documentation system we invite you to
> participate.  We will be considering documentation in all common forms
> such as function and class documentation, but also areas such as data
> frames, non-standard objects, packages, and how vignettes fit into the
> documentation framework.  If you have opinions or concerns that you
> would like to make sure are addressed please join.
>
> To join send an email to Andrew dot Redd at hsc dot utah dot edu,
> expressing your interests, skills or expertise as it relates to R
> documentation.  Also email if you have ideas or concerns but do not wish
> to play and active role.
>
> Thank you,
> Andrew Redd
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] formal process for orphaning a package

[Andrew Redd]

The README states clearly that a package is orphaned under any of three
conditions:

   1. The Maintainer requests is.
   2. The maintainer email bounces
   3. The maintainer is unresponsive to requests regarding the package from
   CRAN maintainers

But I think that it is a good idea to include those conditions in the
manuals.

On Wed, Sep 21, 2016 at 9:30 AM Max Kuhn  wrote:

> The CRAN policy page
> (https://cran.r-project.org/web/packages/policies.html) implies that
> there is a formal procedure for orphaning a package but none is
> mentioned in the Extensions manual
> (https://cran.r-project.org/doc/manuals/r-devel/R-exts.html).
>
> This page (https://cran.r-project.org/src/contrib/Orphaned/README)
> implies that one would simply resubmit the package to CRAN with the
> text "ORPHANED" in the `Maintainer` field.
>
> Is this the case?
>
> If this is not documented somewhere, can it be added to the Extensions
> manual?
>
> Thanks,
>
> Max
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Rprofile.site function or variable definitions break with R 4.1

[Andrew Piskorski]

With R 4.1, it seems you can no longer do much in your "Rprofile.site"
file.  Attempting to define any functions or set any variables there
gives errors like these:

  Error: cannot add binding of 'my_function_name' to the base environment
  Error: cannot add binding of 'my_variable_name' to the base environment

Presumably that's because of this change in R 4.1.0:

  https://cran.r-project.org/doc/manuals/r-patched/NEWS.html
  CHANGES IN R 4.1.0
  The base environment and its namespace are now locked (so one can no
  longer add bindings to these or remove from these).

Ok, but what's the recommended way to actually USE Rprofile.site now?
Should I move all my local configuration into a special package, and
do nothing in Rprofile.site except require() that package?

Thanks for your help and advice!

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R 4.1.x make check fails, stats-Ex.R, step factor reduced below minFactor

[Andrew Piskorski]

I recently built R 4.1.1 (Patched) from source, as I have many older
versions over the years.  This version, on Ubuntu 18.04.4 LTS:

  R 4.1.1 (Patched), 2021-09-21, svn.rev 80946, x86_64-pc-linux-gnu

Surprisingly, "make check" fails, which I don't recall seeing before.
The error is in from stats-Ex.R, which unfortunately terminates all
further testing!  This particular error, "step factor ... reduced
below 'minFactor'" does not seem very serious, but I can't figure out
why it's happening.

I installed with "make install install-tests" as usual, which seemed
to work fine.  Running the same tests after install, I'm able to get
more coverage by using errorsAreFatal=FALSE.  However, it seems the
rest of the 'stats' tests after the bad one still do not run.

I'm confused about the intent of this particular test.  The comment
above it seems to says that it's SUPPOSED to throw this error, yet
getting the error still terminates further testing, which seems
strange.  What's supposed to happen here?

Any ideas on why this error might be occurring, and how I should debug
it?  What's the right way for me to disable this one failing test, so
the ones after it can run?

Thanks for your help!


## "make check" output:
make[1]: Entering directory 
'/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests'
make[2]: Entering directory 
'/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests'
make[3]: Entering directory 
'/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests/Examples'
Testing examples for package 'base'
Testing examples for package 'tools'
  comparing 'tools-Ex.Rout' to 'tools-Ex.Rout.save' ... OK
Testing examples for package 'utils'
Testing examples for package 'grDevices'
  comparing 'grDevices-Ex.Rout' to 'grDevices-Ex.Rout.save' ... OK
Testing examples for package 'graphics'
  comparing 'graphics-Ex.Rout' to 'graphics-Ex.Rout.save' ... OK
Testing examples for package 'stats'
Error: testing 'stats' failed
Execution halted
Makefile:37: recipe for target 'test-Examples-Base' failed
make[3]: *** [test-Examples-Base] Error 1
make[3]: Leaving directory 
'/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests/Examples'
../../tests/Makefile.common:198: recipe for target 'test-Examples' failed
make[2]: *** [test-Examples] Error 2
make[2]: Leaving directory '/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests'
../../tests/Makefile.common:184: recipe for target 'test-all-basics' failed
make[1]: *** [test-all-basics] Error 1
make[1]: Leaving directory '/home/nobackup/co/R/R-4-1-branch/Build-x86_64/tests'
Makefile:305: recipe for target 'check-all' failed
make: *** [check-all] Error 2


## From file:  tests/Examples/stats-Ex.Rout.fail

> ## Here, requiring close convergence, you need to use more accurate numerical
> ##  differentiation; this gives Error: "step factor .. reduced below 
> 'minFactor' .."
> options(digits = 10) # more accuracy for 'trace'
> ## IGNORE_RDIFF_BEGIN
> try(nlm1 <- update(nlmod, control = list(tol = 1e-7))) # where central diff. 
> work here:
Warning in nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  :
  No starting values specified for some parameters.
Initializing 'Const', 'A', 'B' to '1.'.
Consider specifying 'start' or using a selfStart model
>(nlm2 <- update(nlmod, control = list(tol = 8e-8, nDcentral=TRUE), 
> trace=TRUE))
Warning in nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  :
  No starting values specified for some parameters.
Initializing 'Const', 'A', 'B' to '1.'.
Consider specifying 'start' or using a selfStart model
1017460.306(4.15e+02): par = (1 1 1)
758164.7503(2.34e+02): par = (13.42031396 1.961485 0.05947543745)
269506.3538(3.23e+02): par = (51.75719816 -13.09155957 0.8428607709)
68969.21893(1.03e+02): par = (76.0006985 -1.935226745 1.0190858)
633.3672230(1.29e+00): par = (100.3761515 8.624648402 5.104490259)
151.4400218(9.39e+00): par = (100.6344391 4.913490985 0.2849209569)
53.08739850(7.24e+00): par = (100.6830407 6.899303317 0.4637755074)
1.344478640(5.97e-01): par = (100.0368306 9.897714142 0.5169294939)
0.9908415909   (1.55e-02): par = (100.0300625 9.9144191 0.5023516843)
0.9906046057   (1.84e-05): par = (100.0288724 9.916224018 0.5025207336)
0.9906046054   (9.95e-08): par = (100.028875 9.916228366 0.50252165)
0.9906046054   (9.93e-08): par = (100.028875 9.916228366 0.50252165)
Error in nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  : 
  step factor 0.000488281 reduced be

Re: [Rd] R 4.1.x make check fails, stats-Ex.R, step factor reduced below minFactor

[Andrew Piskorski]

tform: x86_64-pc-linux-gnu/x86_64 (64-bit)

## The two examples below show that you can fit a model to
## artificial data with noise but not to artificial data
## without noise.
> x <- 1:10
> y <- 2*x + 3# perfect fit
## terminates in an error, because convergence cannot be confirmed:
> try(nls(y ~ a + b*x, start = list(a = 0.12345, b = 0.54321)))
> Error in nls(y ~ a + b * x, start = list(a = 0.12345, b = 0.54321)) : 
  number of iterations exceeded maximum of 50
## adjusting the convergence test by adding 'scaleOffset' to its denominator 
RSS:
> nls(y ~ a + b*x, start = list(a = 0.12345, b = 0.54321),
control = list(scaleOffset = 1, printEval=TRUE))
> +   It.   1, fac=   1, eval (no.,total): ( 1,  1): new dev = 
> 1.05935e-12
Nonlinear regression model
  model: y ~ a + b * x
   data: parent.frame()
a b 
3 2 
 residual sum-of-squares: 1.059e-12

Number of iterations to convergence: 1 
Achieved convergence tolerance: 3.639e-07
> ## Alternatively jittering the "too exact" values, slightly:
set.seed(27)
> yeps <- y + rnorm(length(y), sd = 0.01) # added noise
> nls(yeps ~ a + b*x, start = list(a = 0.12345, b = 0.54321))
Nonlinear regression model
  model: yeps ~ a + b * x
   data: parent.frame()
a b 
3.001 2.000 
 residual sum-of-squares: 0.001346

Number of iterations to convergence: 2 
Achieved convergence tolerance: 8.658e-09
> 
> ## the nls() internal cheap guess for starting values can be sufficient:
> x <- -(1:100)/10
> y <- 100 + 10 * exp(x / 2) + rnorm(x)/10
> nlmod <- nls(y ~  Const + A * exp(B * x))
Warning message:
In nls(y ~ Const + A * exp(B * x)) :
  No starting values specified for some parameters.
Initializing 'Const', 'A', 'B' to '1.'.
Consider specifying 'start' or using a selfStart model
> options(digits = 10) # more accuracy for 'trace'
> try(nlm1 <- update(nlmod, control = list(tol = 1e-7))) # where central diff. 
> work here:
Warning message:
In nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  :
  No starting values specified for some parameters.
Initializing 'Const', 'A', 'B' to '1.'.
Consider specifying 'start' or using a selfStart model
>(nlm2 <- update(nlmod, control = list(tol = 8e-8, nDcentral=TRUE), 
> trace=TRUE))
1017460.306(4.15e+02): par = (1 1 1)
758164.7503(2.34e+02): par = (13.42031396 1.961485 0.05947543745)
269506.3538(3.23e+02): par = (51.75719816 -13.09155957 0.8428607709)
68969.21893(1.03e+02): par = (76.0006985 -1.935226745 1.0190858)
633.3672230(1.29e+00): par = (100.3761515 8.624648402 5.104490259)
151.4400218(9.39e+00): par = (100.6344391 4.913490985 0.2849209569)
53.08739850(7.24e+00): par = (100.6830407 6.899303317 0.4637755074)
1.344478640(5.97e-01): par = (100.0368306 9.897714142 0.5169294939)
0.9908415909   (1.55e-02): par = (100.0300625 9.9144191 0.5023516843)
0.9906046057   (1.84e-05): par = (100.0288724 9.916224018 0.5025207336)
0.9906046054   (9.95e-08): par = (100.028875 9.916228366 0.50252165)
0.9906046054   (9.93e-08): par = (100.028875 9.916228366 0.50252165)
Error in nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  : 
  step factor 0.000488281 reduced below 'minFactor' of 0.000976562
In addition: Warning message:
In nls(formula = y ~ Const + A * exp(B * x), algorithm = "default",  :
  No starting values specified for some parameters.
Initializing 'Const', 'A', 'B' to '1.'.
Consider specifying 'start' or using a selfStart model


-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Blue Sky Statistics - GPL Violation?

[Andrew Simons]

Hi,

There are (sometimes creative) ways that one can combine proprietary code
with GPL code, without running afoul of the GPL, I assume this is the case
with h2o -- but I have difficulty seeing that this would be the case with
blue sky. For example:

https://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.en.html#MereAggregation

Kind regards

On Tue, 28 Jun 2022 at 11:44, Tom Woolman  wrote:

> Hi. I am not a lawyer and I didn't stay at a Holiday Inn last night, but
> I don't think this is a GPL violation. The closest analogy for non-GPL
> release packages that comes to mind is with h2o. h2o is proprietary (for
> profit) that is also simultaneously released on CRAN; their business
> model is consulting services for selling support and app development for
> h2o-enabled models, but they allow free use of the core algorithm
> without paying a license fee.
>
> I'm a huge fan of h2o and have used their GBM and MLP algorithms to
> build models for my two most recent published papers (in press). I
> suspect that this type of hybrid freeware/retained rights framework
> might be the future for some of the larger, more scalable and robust
> library package releases. It seems to also mitigate somewhat the issue
> of abandonware packages, because people are throwing some bucks into the
> tip jar via enterprise support license subscriptions.
>
> Which reminds me that RStudio also does something similar :)
>
>
>
>
> On 2022-06-27 20:06, Andrew Simons wrote:
> > Hello,
> >
> > I notice that v10 of BlueSky Statistics is quite tightly integrated
> > with R,
> > but is not released under a GPL compatible license.
> >
> > Do people think this represents a violation of the GPL?
> >
> > Kind regards,
> >
> > Andrew
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] as.character.Date() strips names in R 4.3.2 beta, bug?

[Andrew Piskorski]

In previous versions of R, as.character.Date() retained any names on
its input vector.  In R 4.3.2 beta, it removes names.  Is this change
intentional, or a bug?  (For what it's worth, I greatly dislike this
change, and hope it gets changed back.)


$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
$ R --vanilla
R version 4.2.1 Patched (2022-07-09 r82577) -- "Funny-Looking Kid"
> v2 <- structure(as.Date(c('2021-10-06','2021-10-08')) ,names=c('a','b'))
> v2
   ab
"2021-10-06" "2021-10-08"
> class(v2)
[1] "Date"
> as.character(v2)
   ab
"2021-10-06" "2021-10-08"
> as.character.Date(v2)
   ab
"2021-10-06" "2021-10-08"


$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
$ R --vanilla
R version 4.3.2 beta (2023-10-22 r85392) -- "Eye Holes"
> v2 <- structure(as.Date(c('2021-10-06','2021-10-08')) ,names=c('a','b'))
> v2
   ab
"2021-10-06" "2021-10-08"
> class(v2)
[1] "Date"
> as.character(v2)
[1] "2021-10-06" "2021-10-08"
> as.character.Date(v2)
[1] "2021-10-06" "2021-10-08"

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] as.character.Date() strips names in R 4.3.2 beta, bug?

[Andrew Piskorski]

On Tue, Oct 24, 2023 at 10:53:10AM +0200, Martin Maechler wrote:

> Yes, this change has been *very* intentional:  as.character() for
> these objects since 4.3.0 (April 2023) finally behaves as other
> as.character() methods and *does* drop attributes.
[...]

Thank you for the detailed explanation and pointers to the old info
I'd missed, Martin.  The as.character.Date tip in particular was very
helpful in adapting my code to work with this change.

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Undesirable behaviour of base::factor

[Andrew Gustar]

This thread on stackoverflow illustrates the problem... 
https://stackoverflow.com/questions/78523612/r-factor-from-numeric-vector-drops-every-100-000th-element-from-its-levels

The issue is that factor(), applied to numeric values, uses as.character(), 
which converts numbers to character strings according to the value of scipen. 
The stackoverflow thread illustrates a case where this causes some factor 
levels to become NA. There is also an inconsistency between the treatment of 
numeric and integer values.

On the face of it, using format(..., scientific = FALSE) instead of 
as.character() would solve the problem, but this probably needs careful 
thinking through in case of other side effects!


[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] how to interpose my own "[" function?

[Andrew Piskorski]

I want to create my own "[" function (for use on vectors, matrices,
arrays, etc.), which calls the stock R "[", does some additional work,
and then finally returns the modified result.

But, how do I properly call the stock R "[" function?  It takes a
varying number of positional arguments, and its R-level closure is
just:  .Primitive("[")  It's implemented by do_subset_dflt in
src/main/subset.c.

The only thing I've come up with so far is using eval after crudely
re-constructing the original incoming call (example below), which is
both very slow and gets some of the semantics wrong.

Is there some good way for my function to just say, "take ALL my
incoming arguments, whatever they might be, and pass them as-is to
.Primitive('['), then return control to me here"?  Or said another
way, what I want is to hook the end of the R-level "[" function and do
some extra work there before returning to the user.

Basically, where should I look to figure out how to do this?  Is it
even feasible at all when the call I want to intercept is implemented
in C and is Primitive like "[" is?  Is there some other technique I
should be using instead to accomplish this sort of thing?

Thanks for your help!  Example of awful eval-based code follows:


my.subset <- function(x ,i ,j ,... ,drop=TRUE) { 
   brace.fcn <- get("[",pos="package:base") 
   code <- 'brace.fcn(x,' 
   if (!missing(i))code <- paste(code ,'i' ,sep="") 
   # This fails to distinguish between the mat[1:21] and mat[1:21,] cases: 
   if (length(dim(x)) > 1 && (missing(i) || length(dim(i)) <= 1)) 
  code <- paste(code ,',' ,sep="") 
   if (!missing(j))code <- paste(code ,'j' ,sep="") 
   if (!missing(...))  code <- paste(code ,',...' ,sep="") 
   if (!missing(drop)) code <- paste(code ,',drop=drop' ,sep="") 
   code <- paste(code ,')' ,sep="") 
   result <- eval(parse(text=code)) 
   # FINALLY we have the stock result, now modify it some more... 
   result 
} 

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Why did R 3.0's resolveNativeRoutine remove full-search ability?

[Andrew Piskorski]

In versions of R prior to 3.0, by default .C and .Call would find the
requested C function regardless of which shared library it was located
in.  You could use the PACKAGE argument to restrict the search to a
specific library, but doing so was not necessary for it to work.

R 3.0 introduced a significant change to that behavior; from the NEWS
file:

  CHANGES IN R 3.0.0: 
  PERFORMANCE IMPROVEMENTS: 
* A foreign function call (.C() etc) in a package without a PACKAGE 
  argument will only look in the first DLL specified in the 
  NAMESPACE file of the package rather than searching all loaded 
  DLLs.  A few packages needed PACKAGE arguments added. 

That is not merely a performance improvement, it is a significant
change in functionality.  Now, when R code in my package foo tries to
call C code located in bar.so, it fails with a "not resolved from
current namespace (foo)" error.  It works if I change all my uses of
.C and .Call to pass a PACKAGE="bar" argument.  Ok, I can make that
change in my code, no big deal.

What surprises me though, is that there appears to be no way to invoke
the old (and very conventional Unix-style), "I don't want to specify
where the function is located, just keep searching until you find it"
behavior.  Is there really no way to do that, and if so, why not?

Comparing the R sources on the 3.1 vs. 2.15 branches, it looks as if
this is due to some simple changes to resolveNativeRoutine in
"src/main/dotcode.c".  Specifically, the newer code adds this:

   errorcall(call, "\"%s\" not resolved from current namespace (%s)",
 buf, ns);

And removes these lines:

   /* need to continue if the namespace search failed */
   *fun = R_FindSymbol(buf, dll.DLLname, symbol);
   if (*fun) return args;

Is that extra call to R_FindSymbol really all that's necessary to
invoke the old "keep searching" behavior?  Would it be a good idea to
provide an optional way of finding a native routine regardless of
where it's located, perhaps via an optional PACKAGE=NA argument to .C,
.Call, etc.?

And now I see that help(".Call") says:

   'PACKAGE = ""' used to be accepted (but was undocumented): it is 
now an error. 

I assume passing PACKAGE="" used to invoke the same "keep searching"
behavior as not passing any PACKAGE argument at all.  So apparently
the removal of functionality was intentional.  I'd like to better
understand why.  Why should that be an error?  Or said another way,
why has traditional Unix-style symbol resolution been banned from use
with .C and .Call ?

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] read.table() code fails outside of the utils package

[Andrew Piskorski]

One of the great things about R is how readable and re-usable much of
its own implementation is.  If an R function doesn't do quite what you
want but is close, it is usually very easy to read its code and start
adapting that as the base for a modified version.

In the 2.x versions of R, that was the case with read.table().  It was
easy to experiment with its source code, as it all worked just fine
when run at the top level or from inside any other package.

In R 3.1.0, that is no longer true.  The read.table() source ONLY works
when run from inside the "utils" package.  The (only) culprit is this:

  .External(C_readtablehead, file, 1L, comment.char, blank.lines.skip, quote, 
sep, skipNul)

Older versions of read.table() instead did this, which ran fine from
any package; this entry point no longer exists:

  .Internal(readTableHead(file, nlines, comment.char, blank.lines.skip, quote, 
sep)) 

The C implementation of readTableHead is in utils.so, but the symbol
is marked as local.  I tried adding "attribute_visible" to its
function definition in "src/library/utils/src/io.c" and recompiling,
which DID make the symbol globally visible.  With that change, my own
C code works just fine when calling readTableHead.  But interestingly,
R code using .External() like this still fails:

   .External("readtablehead", ..., PACKAGE="utils") 
   Error: "readtablehead" not available for .External() for package "utils" 

Why is that?  Apparently the C symbol being visible isn't enough, but
what else is needed for .External() to work?
(Clearly there's something here about how R C programming works that I
don't understand.)

Finally, since it is generally useful to be able to experiment with
and re-use parts of the stock read.table() implementation, I suggest:

1. R add "attribute_visible" or otherwise make readtablehead callable
   from user C code.
2. R make readtablehead callable from user R code via .External().

What do you think?  Note that I'm not asking that the current
interface or behavior of readtablehead necessarily be SUPPORTED in any
way, just that it be callable for experimental purposes, much as the
old .Internal(readTableHead()) was in earlier versions of R.

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] read.table() code fails outside of the utils package

[Andrew Piskorski]

On Mon, Apr 21, 2014 at 12:43:55PM -0400, Simon Urbanek wrote:

> And that's how it should be - there is not reason why any other code should 
> link to it. Why don't you just use
> 
> .External(utils:::C_readtablehead, ...)

Ah, that works fine, and is nice and simple.  So problem solved, thank
you!

I do still wonder though, with the C symbol made visible in utils.so,
how come this still failed?:

   .External("readtablehead", ..., PACKAGE="utils") 
   Error: "readtablehead" not available for .External() for package "utils" 

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] how to get old type.convert() numeric behavior?

[Andrew Piskorski]

Regarding this change:

> CHANGES IN R 3.1.0: 
>   NEW FEATURES: 
> * type.convert() (and hence by default read.table()) returns a 
>   character vector or factor when representing a numeric input as a 
>   double would lose accuracy.  Similarly for complex inputs. 
>  
>   If a file contains numeric data with unrepresentable numbers of 
>   decimal places that are intended to be read as numeric, specify 
>   colClasses in read.table() to be "numeric". 

How do I get the old behavior where type.convert() automatically
converts to numeric if suitable, regardless of whether or not the
string has more than 17 digits of accuracy?

Sure, I could first pass every single column of data through a kludgy
checking function like my.can.be.numeric() below, and then set
colClasses to "numeric" or not based on that, but is there a better
way?


my.can.be.numeric <- function(xx) { 
   old.warn <- options(warn = -1) 
   on.exit(options(old.warn)) 
   (!is.na(as.numeric(xx))) 
} 

Example of the changed behavior in R 3.1.0 vs. earlier versions, both
with options("digits"=10) set:
  
# R version 3.1.0 Patched (2014-04-15 r65398) -- "Spring Dance" 
# Platform: x86_64-unknown-linux-gnu/x86_64 (64-bit) 
> type.convert(paste("0.", paste(rep(0:9,3)[seq_len(17)],collapse=""), sep=""), 
> as.is=TRUE) 
[1] 0.01234568 
> type.convert(paste("0.", paste(rep(0:9,3)[seq_len(18)],collapse=""), sep=""), 
> as.is=TRUE) 
[1] "0.012345678901234567" 

# R version 3.0.2 Patched (2013-10-23 r64103) -- "Frisbee Sailing" 
# Platform: x86_64-unknown-linux-gnu/x86_64 (64-bit) 
> type.convert(paste("0.", paste(rep(0:9,3)[seq_len(17)],collapse=""), sep=""), 
> as.is=TRUE) 
[1] 0.01234568 
> type.convert(paste("0.", paste(rep(0:9,3)[seq_len(18)],collapse=""), sep=""), 
> as.is=TRUE) 
[1] 0.01234568 

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] read.table() code fails outside of the utils package

[Andrew Piskorski]

On Mon, Apr 21, 2014 at 06:44:05PM +0100, Prof Brian Ripley wrote:
> On 21/04/2014 18:08, Andrew Piskorski wrote:

> >> .External(utils:::C_readtablehead, ...)
> >
> > Ah, that works fine, and is nice and simple.  So problem solved, thank
> > you!
> >
> > I do still wonder though, with the C symbol made visible in utils.so,
> 
> That isn't true on platforms which support hiding entry points.  Try
> 
> % nm -g library/utils/libs/utils.so | grep readtablehead
> 
> on Linux.

What isn't true?  That .External(utils:::C_readtablehead, ...)
should work for me under stock R 3.0.1?  My Linux (Ubuntu 12.04.3 LTS)
definitely does seem to support hiding entry points.

I have two separate installs, both built from source.  The first one
is stock, and the lower-case "t" here indicates that the readtablehead
symbol is local:

  andy@odo:~$ nm 
/usr/local/pkg/R-3.1-branch-20140416/lib/R/library/utils/libs/x86_64/utils.so | 
grep readtablehead
  59a0 t readtablehead

This second install is where I added "attribute_visible" to
readtablehead and recompiled, the upper-case "T" means it is a global
symbol:

  andy@odo:~$ nm 
/usr/local/pkg/R-3.1-branch-20140418/lib/R/library/utils/libs/x86_64/utils.so | 
grep readtablehead
  59c0 T readtablehead

Fortunately, calling .External(utils:::C_readtablehead, ...) works
with either of those installs.  But writing my won C code that calls
readtablehead only works in the second one where the symbol is global.

> > how come this still failed?:
> >
> > .External("readtablehead", ..., PACKAGE="utils")
> > Error: "readtablehead" not available for .External() for package "utils"
> 
> Rather, you need to tell us why that should have worked 

That exact style of call DOES work for every cross-package use of .C
and .Call I've tried in my own code.  But I have never "registered"
any of my C code with R.  Obviously there is something different about
readtablehead and/or the utils package (probably the latter), and I am
trying to understand what it is.

> Maybe you failed to read in the code
> 
> R_init_utils(DllInfo *dll)
> {
>  R_registerRoutines(dll, NULL, CallEntries, NULL, ExtEntries);
>  R_useDynamicSymbols(dll, FALSE);
>  R_forceSymbols(dll, TRUE);
> }

I believe you are implying that using R_registerRoutines in a package
changes the behavior of .Call() and .External() such that ONLY
registered functions will be found, even if I invoke .External() with
a string function name like "readtablehead" rather than the registered
value C_readtablehead.  While if I do not register any C functions at
all in a package, then using a string name like "readtablehead" will
work as long as that C function symbol is visible.

> > .External("readtablehead", ..., PACKAGE="utils")
> > Error: "readtablehead" not available for .External() for package "utils"

> See 'Writing R Extensions'.

I already had, many times.  If this question is answered there, it was
not apparent to me.

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] palette() can hang and fail due to X11

[Andrew Piskorski]

For many years, when my R process starts up I've been automatically
setting my preferred default plot colors, basically like so:

  my.colors <-
 c("black" ,"red" ,"gold" ,"sky blue" ,"green" ,"blue" ,"orange"
   ,"grey" ,"hot pink" ,"brown" ,"sea green" ,"cyan" ,"purple" ,"tomato1")
  require("grDevices")
  palette(my.colors)

That seemed to work reliably in all 2.x versions of R, regardless of
whether my R was interactive or not, or if my Linux, ssh, and screen
environment had X-Windows properly set up or not.  It Just Worked.

However, now in R 3.1.0 Patched (2014-04-15 r65398, on Linux),
depending on whether I have a good X-Windows connection or not it can
fail like so:

  Error in .External2(C_X11, d$display, d$width, d$height, d$pointsize,  :  
unable to start device X11 

Simply wrapping the palette() call in try() of course helps keep that
error from breaking the rest of my R start up.  However, occasionally
the call to palette() will hang for perhaps a minute, unexpectedly
locking up my R process until it finishes whatever it was doing.

But, all I want to do here is set my default colors to the length 14
vector above, which seems like something that SHOULD be simple...  Is
there some way for me to reliably do that WITHOUT invoking all this
extra X11 device machinery?

The relevant C code appears to be "palette" in
"src/library/grDevices/src/colors.c" and "do_dotcallgr" for
.Call.graphics in "src/main/dotcode.c", but I don't understand what
part is triggering the additional complex behavior, nor how I should
avoid it.

Any advice on how I should handle this robustly?  (Thanks!)

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] palette() can hang and fail due to X11

[Andrew Piskorski]

The fundamental problem here seems to be a change (probably a bug) in
the behavior of palette().  In R 3.1.0, calling palette() opens a new
X window (on Linux)!  That seems like a bug, as I can't think of any
good reason for it to open a window, and it never did in any of the
2.x versions of R I've ever used.

I am using:

  R 3.1.0 (Patched), 2014-04-15, svn.rev 65398, x86_64-unknown-linux-gnu  
  Ubuntu 12.04.3 LTS

Is there something else I should check to help track down the bug?

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] palette() can hang and fail due to X11

[Andrew Piskorski]

On Thu, Apr 24, 2014 at 09:22:40PM -0400, Simon Urbanek wrote:

> The bottom line is that you probably don't want to set the palette
> if you don't have a device that could be used.

Ok.  In my testing so far, it seems all I need is a simple little
function that does:

  pdf(); nn <- dev.cur(); rr <- palette(my.colors); dev.off(nn); rr 

I could have it check dev.list() and skip creating the sacrificial pdf
device if there already is a real graphics device up and running, but
in my testing so far that doesn't seem to be necessary, creating the
new pdf device gives the right behavior no matter what.

So if anybody else runs into this, that seems to be the practical
solution, it's pretty easy.  Pedantically though, ideally this hack
wouldn't be necessary.  You said:

> palette is a property recorded in the graphics device*

Perhaps its implementation currently is, I don't know.  (But what
about the special "null" graphics device that is always open,
shouldn't that be a good place to record the session-wide palette?)
But logically, the color palette is not and cannot be solely a
property of a graphics device.  Note that help("palette") says:

   There is only one palette setting for all devices in a R session. 
   If the palette is changed, the new palette applies to all 
   subsequent plotting. 

So the current color choices are logically a property of the R
session, independent of any specific graphics devices.

Clearly, palette() was always intended to do two things.  One, it
changes the user's session-wide default colors.  Two, it tells the
currently open graphics device (if any), to use those new colors.

A new (and no doubt useful) feature in R 3.0 is that each graphics
device remembers its history of palette changes.  Apparently as a side
effect of that change, palette() now INSISTS that a real non-null
graphics device be open, and opens one if there isn't.  This is a new
third thing palette() does that it never did before.

Glomming together all three of these essentially independent jobs into
the single palette() command seems a bit unfortunate.  Is there was
some way for me to do ONLY the first of palette()'s jobs, set my
session-wide default colors and that's it?  It looks like there is no
such entry point in the code, but the little hack with the sacrificial
pdf device above is a way to approximate one.

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Please make Pre-3.1 read.csv (type.convert) behavior available

[Andrew Piskorski]

On Fri, Apr 25, 2014 at 09:23:23PM -0700, Tom Kraljevic wrote:

> This, needless to say, is disruptive for us.  (Actually, it was downright 
> shocking.)

It WAS somewhat shocking.  I trust the R core team to get things
right, and (AFAICT) they nearly always do.  This was an exception, and
shocking mostly in that it was so obviously wrong to completely
discard all possibility of backwards compatibility.

The old type.convert() functionality worked fine and was very useful,
so the *obviously* right thing to do would be to at least retain the
old behavior as a (non-default) option.

Reproducing the old behavior in user R code is not simple.  For
anybody else stuck with this, you can do it (probably inefficiently)
with the two functions below.  Create your own version of read.table()
that calls the dtk.type.convert() below instead of the stock
type.convert().  It's not pretty, but that will do it.


dtk.type.convert <- function(xx ,... ,ignore.signif.p=TRUE) { 
   # Add backwards compatibility to R 3.1's "new feature": 
   if(ignore.signif.p && all(dtk.can.be.numeric(xx ,ignore.na.p=TRUE))) { 
  if(all(is.na(xx))) type.convert(xx ,...) 
  else methods::as(xx ,"numeric")  
   } else type.convert(xx ,...) 
} 

dtk.can.be.numeric <- function(xx ,ignore.na.p=TRUE) { 
   # Test whether a value can be converted to numeric without becoming NA. 
   # AKA, can this value be usefully represented as numeric? 
   # Optionally ignore NAs already present in the incoming data. 

   old.warn <- options(warn = -1) ; on.exit(options(old.warn)) 
   aa <- !is.na(as.numeric(xx)) 
   if(ignore.na.p) (is.na(xx) | aa) else aa 
}

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Write unix format files on windows and vice versa

[Andrew Redd]

I go back and forth between windows and linux, and find myself running
into problem with line endings.  Is there a way to control the line
ending conversion when writing files, such as write and cat?  More
explicitly I want to be able to write files with LF line endings
rather than CRLF line ending on windows; and CRLF line endings instead
of LF on linux, and I want to be able to control when the conversion
is made and/or choose the line endings that I want.

As far as I can tell the conversion is not optional and buried deep in
compiled code. Is there a possible work around?

Thanks,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Quiz: How to get a "named column" from a data frame

[Andrew Piskorski]

On Sat, Aug 18, 2012 at 02:13:20PM -0400, Christian Brechb?hler wrote:
> On 8/18/12, Martin Maechler  wrote:
> > On Sat, Aug 18, 2012 at 5:14 PM, Christian Brechb?hler  wrote:
> >> On Sat, Aug 18, 2012 at 11:03 AM, Martin Maechler
> >>  wrote:
> 
> >>> Consider this toy example, where the dataframe already has only
> >>> one column :
> >>>
> >>> > nv <- c(a=1, d=17, e=101); nv
> >>>   a   d   e
> >>>   1  17 101
> >>>
> >>> > df <- as.data.frame(cbind(VAR = nv)); df
> >>>   VAR
> >>> a   1
> >>> d  17
> >>> e 101
> >>>
> >>> Now how, can I get 'nv' back from 'df' ?   I.e., how to get

> >>> identical(nv, df[,1])
> >> [1] TRUE
> 
> > But it is not a solution in a current version of R!
> > though it's still interesting that   df[,1]  worked in some incantation of
> > R.
> 
> My mistake!  We disliked some quirks of indexing, so we've long had
> our own patch for "[.data.frame" in place, which I used inadvertently.

As I understand it, when when doing 'df[,1]' on a data frame, Bell
Labs S and all versions of S-Plus prior to 3.4 always retained the
data frame's row names as the names on the result vector.  'df[,1]'
gave you a named vector identical to your 'nv' above.  Then in 1996
with S-Plus 3.4, Insightful broke that behavior, after which 'df[,1]'
returned a vector without any names.  I believe R copied that
late-1990s S-Plus behavior, but I don't know why exactly.

When subscripting objects, R sometimes retains the object's dimnames
as names in the result, and sometimes not, which I find frustrating.
Personally, I think it would make much more sense if subscripting
ALWAYS retained any names it could, and worked as similarly as
possible across data frames, matrices, arrays, vectors, etc.  After
all, explicitly dropping names afterwards is trivial, while adding
them back on is not.

Back on 2005-10-19 with R 2.2.0, I gave a simple test of 15 cases; 4
of them dropped names during subscripting, the other 11 preseved them.
That's towards the end of the discussion here:

  https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=8192

Contrary to the initial tone of my old 2005 "bug" report, current R
subscripting behavior is of course NOT a bug, as AFAIK it's working as
the R Core Team intended.  However, I definitely consider the current
behavior a design infelicity.

Just now on stock R 2.15.1 (with --vanilla), I ran an updated version
of those same simple tests.  Of 22 subscripting test cases, 7 lose
names and 15 preserve them.  (If anyone's interested in the specific
tests, I can send them, or try to append them to that old 8192 feature
request.)

For what it's worth, at work, for years we ran various versions of
pre-namespace R using some ugly patches of "[" and "[.data.frame" to
force name retention during subscripting.  Since we were not using
namespaces at all, those "keep names" subscripting hacks were
affecting ALL R code we ran, not just our own custom code which needed
and expected the names to be retained.  Yet perhaps surprisingly, I
don't think I ever ran into a single case where the forced retention
of names broke any code.  We of course ran only a tiny sample of the
huge amount of code on CRAN, but that experience suggests that most R
code which expects un-named objects doesn't mind at all if names are
present.

If anyone would genuinely like to add an option for name-preserving
subscripting to R, I'm willing to work on it, so please do let me know
your thoughts.  So far though, I've never dug into the guts of the
.Primitive("[") and "[.data.frame" functions to see how/why they
sometimes keep and sometime discard names during subscripting.

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] why does do_stop call R_GetTraceback ?

[Andrew Piskorski]

I'm using:
  R 2.15.1 (beta), 2012-06-11, svn.rev 59557, x86_64-unknown-linux-gnu
And I normally use:
  options("error"=recover)

I recently ran into a problem where when my code called stop(),
recover() was incredibly slow (hours) because it was trying to deparse
an extremely large call.  I worked around that by instead using my own
recover function which avoids deparsing anything large.  (Perhaps it
would be better to instead make the deparsing much faster, but I don't
know how to do that.)

However, now I find that when I'm DONE debugging and exit my recover
function, R still sits there using 100% cpu for another 41 seconds
before finishing and returning me to the top-level interactive prompt.
I wondered why...  Rprof() seems to say that all the time is being
taken in stop().

AFAICT, in this case stop() is basically just doing this:

  .Internal(stop(as.logical(TRUE), .makeMessage(..., domain=NULL)))

where the "..." is just the single string I passed to stop(), e.g.,
stop("This code is broken.").

So next, after exiting (my hacked version of) recover(), and while
stop() was still trying to finish up, I attached gdb to my R process.
I've appended the relevant part of the (long) backtrace below.

As you can see, R seems to have been busy calling R_GetTraceback() and
a bunch of subsidiary deparse functions.  But, at that point I'm done
with recover(), and when stop() finally finishes NOTHING else has been
printed out - no additional traceback, nothing.  So why are we calling
jump_to_top_ex() with traceback=TRUE, what good does the call to
R_GetTraceback() do?

That jump_to_top_ex() code only calls R_GetTraceback() when
R_Interactive or haveHandler are true.  I am certainly running
interactively here, and I did define a handler with options("error"=).
But AFAICT, my handler makes no use whatsoever of any of the work done
by R_GetTraceback().  In fact, I don't see any way it could even if I
wanted it to.  And if I remove my handler with options("error"=NULL),
I STILL get no additional "trace back" output at all.

So it looks to me as if calling jump_to_top_ex() with traceback=TRUE
and spending the extra 41 seconds in R_GetTraceback() NEVER does
anything useful!  Is this actually true, or does R_GetTraceback() have
some important side effect that I'm unaware of?

Thanks for your help!


(gdb) bt
#0  vector2buff (s=0xba73ed0, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1298
#1  deparse2buff (s=0xba73ed0, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1123
#2  0x7ff28807034f in vec2buff (v=, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1402
#3  0x7ff28806ea20 in deparse2buff (s=0xba734d0, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:753
#4  0x7ff28807034f in vec2buff (v=, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1402
#5  0x7ff28806ea20 in deparse2buff (s=0xed9e350, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:753
#6  0x7ff28807034f in vec2buff (v=, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1402
#7  0x7ff28806ea20 in deparse2buff (s=0xed9c818, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:753
#8  0x7ff28806e40c in args2buff (arglist=0xd90b930, 
lineb=, formals=0, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1438
#9  0x7ff28806e822 in deparse2buff (s=0xd90b8c0, d=0x7fffa18d4a00)
at ../../../src/main/deparse.c:1108
#10 0x7ff2880706f9 in deparse2 (call=, abbrev=FALSE, 
cutoff=, backtick=, opts=65, 
nlines=-1) at ../../../src/main/deparse.c:484
#11 deparse1WithCutoff (call=, abbrev=FALSE, 
cutoff=, backtick=, opts=65, 
nlines=-1) at ../../../src/main/deparse.c:221
#12 0x7ff2880a07b5 in R_GetTraceback (skip=0)
at ../../../src/main/errors.c:1312
#13 0x7ff2880a347a in jump_to_top_ex (traceback=TRUE, 
tryUserHandler=, processWarnings=FALSE, 
resetConsole=TRUE, ignoreRestartContexts=13368712)
at ../../../src/main/errors.c:837
#14 0x7ff2880a1ba9 in verrorcall_dflt (call=0x28ea1c0, 
format=, ap=)
at ../../../src/main/errors.c:663
#15 0x7ff2880a239e in Rf_errorcall (call=, 
format=) at ../../../src/main/errors.c:698
#16 0x7ff2880a25c2 in do_stop (call=, 
op=, args=0x10ecae78, rho=)
at ../../../src/main/errors.c:1095
[...]
#79 0x00400819 in _start ()

-- 
Andrew Piskorski 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Description depends line for windows only

[Andrew Redd]

I am developing a package that is only applicable on windows, is there a
line I can put in the depends field of the DESCRIPTION file to tell that
this should only be build for windows?

Are there any other protocols that I should follow?

Thanks,

-- 
Andrew
May the Open Source be with you.

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Rprof loses all system() time

[Andrew Piskorski]

Rprof seems to ignore all time spent inside system() calls.  E.g.,
this simple example actually takes about 10 seconds, but Rprof thinks
the total time is only 0.12 seconds:

> Rprof("sleep-system.out") ; system.time(system(command="sleep 10")) ; 
> Rprof(NULL)
   user  system elapsed
  0.000   0.004  10.015
> summaryRprof("sleep-system.out")$by.total
  total.time total.pct self.time self.pct
"gc"0.12   100  0.12  100
"system.time"   0.12   100  0.000
>

I assume that what's going on here, is that anytime R blocks waiting
for data from a child process, the profiler stops running entirley,
and it then loses track of all that time spend blocking.

This Rprof behavior is frustrating, because I use a database access
layer which essentially works by having R call system() to run a shell
script.  So, if I write a really slow query that takes 10 minutes to
run, as for as Rprof is concerned, those 10 minutes never happened,
the query consumed zero time.  In complicated code, this makes it much
harder to figure out what parts are slow due to slow R code vs. slow
SQL queries.

Why does Rprof behave this way?  Is there something I can do to
work-around or alleviate this?  What do you think it would take to fix
Rprof to track the time spent waiting for system() to finish, and
where in the R source should I look to attempt that?

Thanks!

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] how to change FPU control word?

[Andrew Piskorski]

On Mon, Jul 27, 2009 at 12:41:42PM +0200, Martin Becker wrote:

> is there (already) a platform-independent way for (temporarily!) 
> changing the fpu control word?

> Currently, I am using inline assembler (which works for x86 cpus on
> Linux and Windows (MinGW))

Platform-indepdendent?  I don't know about that.  But on Linux, mostly
x86-64, we use these from C:

  _FPU_GETCW()
  _FPU_SETCW()

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] ARM v7/Linux Port/cross-compile?

[Andrew Piskorski]

On Thu, Aug 27, 2009 at 06:36:38PM -0400, Simon Urbanek wrote:

> It was fairly straight-forward to build R (like any other cross- 
> compilation). The tricky part is to install packages (if you are truly  
> cross-compiling on another architecture) which I solved by using multi- 
> arch R (which contains both arm and the host architecture) and cross- 
> compiling only the binaries (i.e. only packages with native code).  

Simon, could you provide any links to more detailed info on how you
did that, please?  E.g., how did you detect whether a package contains
binaries (native code) or not?  What exactly is a multi-arch R and how
did you build it?

I often need to build R packages for both x86 32 and 64 bit.
Currently, I build both R and all the packages twice in two entirely
separate directory trees, which is both annoying, and overkill for all
the R-code-only packages.  It sounds like you know a better way...

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Load a package without installing it

[Andrew Piskorski]

On Fri, Sep 04, 2009 at 08:39:12AM -0500, Hadley Wickham wrote:

> When developing a package, it's often useful to be able to reload it,
> without re-installing, re-starting R and re-loading.

Why would you ever need to restart R in such a situation?

What I do is make the code change in my package, build it from source
like so:

  R CMD INSTALL $r_lazy_arg -l ../R $Library

and then detach and re-attach it in my R session with:

  detach("package:my.pkg") ; library(my.pkg, lib.loc="/home/me/my-R-stuff/R")

That's it.  Works fine as long as your package doesn't use namespaces;
an additional command is necessary to handle namespaces but I forget
what it is.  Works fine for packages with C code too, as long as
you've correctly set up your package's .First.lib() and .Last.lib()
your to call library.dynam() library.dynam.unload().

I'm not sure whether the detach() will play nicely with package
dependencies, as I've found them annoying in other circumstances and
so tend to never use package dependencies in my code.

It'd be nice not to have to "compile" the R code at the command line
all the time and instead just have R read my source files from their
original location when I call library(), but the above isn't bad.

> To do this I've
> written a little script that inspects the package description and
> loads dependencies, data and code - http://gist.github.com/180883.
> It's obviously not very general (being tailored to my description
> files) and won't work for packages containing C code, but I hope you
> might find it useful nonetheless.

source() and load() both dump code into your top-level workspace,
right?  When I'm developing a package, I'd generally rather have R
access my package's stuff via the search path in the usual manner, not
from some other special location that Production never uses.

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Running two R instances at the same time

[Andrew Piskorski]

On Sat, Sep 05, 2009 at 08:31:18PM +0200, Peter Juhasz wrote:

> I don't understand how is this possible. Maybe there is an issue of
> thread-safety with the R backend, meaning that the two R *interpreter*
> instances are talking to the same backend that's capable of processing
> only one thing at a time?

No.  Particularly since there is no R "backend" involved at all, not
if you're starting up the standard R from Ubuntu 9.04 (rather than
Rserve or something else unusual).  People run multiple R processes
concurrently on the same (multi-core) machine all the time, works
fine.

> Please see http://www.perlmonks.org/?node_id=792460 for an extended
> discussion of the problem, and especially
> http://www.perlmonks.org/?node_id=793506 for excerpts of output and
> actual code.

The most likely explanation seems to be that you have a bug in your
Perl code.  Have you tried using your Perl framework to fork something
OTHER than R?  Have you tried manually starting up two R processes and
running your R code that way?  And, what is the actual R code you're
running?  You don't seem to have shown it anywhere.

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Running two R instances at the same time

[Andrew Piskorski]

On Mon, Sep 07, 2009 at 05:36:38PM +0200, Peter Juhasz wrote:

> But please see http://www.perlmonks.org/index.pl?node_id=793907 ,
> where I posted my R code in its simplest form along with an example
> run which exhibits the symptoms I originally wrote about.

Ah, your two-process serialization is probably happening during that
cenros() call then.  (You may want to run with/without that call to
confirm.)  cenros() is in the NADA package and seems to use survreg()
from the survival package.  The survival package looks bigger than
NADA and includes a bunch of C code, so perhaps one of its C
implementations is using some sort of mutex-locked system call.  If
so, it'd be interesting to know where the serialization is happening.

  http://cran.r-project.org/web/packages/NADA/index.html
  http://cran.r-project.org/web/packages/survival/index.html

You may want to run your code under strace (perhaps with -cfF) and/or
ltrace, to get a list of C-level functions that are actually being
called.  That might give you an idea of where the blocking is
occurring, and could also help the NADA or survival package
maintainers when you ask them about this.

(But, hm, haven't any of your international collaborators run across
this serialization problem before?)

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] R's --enable-threads does nothing?; gdb needs -lpthread

[Andrew Piskorski]

In the R-2-9-branch (svn revision 49914, 2009-09-24), R's configure
script has an "--enable-threads" option.  But, does it do anything
useful?  When I use "--enable-threads=posix", some of the configure
output changes slightly, but it seems to have no effect on the actual
link commands used when building R.  Is that a bug, or am I
misunderstanding what it's supposed to do?

Btw, ldd said that exactly these 4 objects are linked against
libpthread.so.0, regardless of whether I use --enable-threads or not:
 
  ./lib/R/modules/x86_64/R_X11.so
  ./lib/R/library/tcltk/libs/x86_64/tcltk.so
  ./lib/R/site-library/bigmemory/libs/x86_64/bigmemory.so
  ./lib/R/site-library/Rmpi/libs/x86_64/Rmpi.so

Next I tried setting the "LIBS=-lpthread" environment variable when
calling configure - that works.  (Perhaps I should have also set
FLIBS, I'm not sure.)  The generated Makeconf file now has this:

  LIBS =  -ldl -lm -lpthread 

and building with that links my libR.so against libpthread.so.0, which
fixes the gdb problem described below.  Is there any reason you know
of NOT to link R with -lpthread on Linux, any potential downside?

Now, why do I care about this -lpthread stuff in the first place?
It's to work around a gdb limitation.

I have a custom library of C code which I use from R, and my library
needs to be linked with -lpthread.  I know that R isn't thread-safe,
and my code is not itself using threads, but it in turn statically
links in some lower level libraries which insist on having the
-lpthread.

That has worked fine for me for a long time, no problems, except that
gdb won't work!  When I run R under the debugger with "R -d gdb", gdb
only works if my own linked-with-pthread package is NOT loaded.  As
soon as I load it, gdb stops with this error:

   [Thread debugging using libthread_db enabled] 
   Error while reading shared library symbols: 
   Cannot find new threads: generic error 

Googling says that happens because the R binary starts out without
thread support, and gdb doesn't know what to do when my *.so suddenly
brings in the thread stuff.  Since I can't stop using -lpthread in my
library, the obvious fix is for me to build R itself with -lpthread,
so gdb doesn't get confused.

-- 
Andrew Piskorski 
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Installing and compiling C code for R-windows

[Hartley, Andrew]

Hi r-devers, 

This is the first time I've tried to install a package from source on
Windows, so please bear with me. I'm trying to install a package written
(and tested) by a colleague in C for R on linux, and I am trying to
install it on windows following the directions here -
http://cran.r-project.org/doc/manuals/R-admin.html#The-Windows-toolset
and here - http://www.murdoch-sutherland.com/Rtools/

I installed Rtools (but not InnoSetup or LaTeX because of permissions
problems on my PC, although I don't think they are essential). I checked
my environment variables are pointing to the correct tools, and then I
ran the following:

R CMD INSTALL --library="C:/R/library" --no-chm PP

The package is called PP, and contains the following files:
>ls -R PP
PP:
DESCRIPTION  R  man  src  svn-commit.tmp~

PP/R:
PP.R

PP/man:
PP.Rd pp.get.longs.Rd  pp.ppw.Rdpp.strip.extra.Rd
pp.close.file.Rd  pp.open.file.Rd  pp.print.Rd  pp.write.Rd
pp.get.lats.Rdpp.ppa.Rdpp.read.Rd   tmp

PP/src:
PP.c  PP.o  PP.so  makefile.old

The result is (based on my limited knowledge) quite promising, but it
seems to be unable to link libraries. Here's what I get:

* Installing *source* package 'PP' ...
** libs
  making DLL ...
gcc -shared -s -o PP.dll tmp.def PP.o -LC:/PROGRA~1/R/R-29~1.2/bin -lR
Cannot export SwapEndian: symbol not found
Cannot export close_file_c: symbol not found
Cannot export open_file_c: symbol not found
. Etc 

I don't have any experience compiling C code, so I'm a bit stumped. Do I
need to override the default gcc settings by editing the makefile? Or
have I missed something more obvious? Please let me know if you need any
additional info.

Kind regards,
Andy



[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Changing the Prompt for browser()

[Andrew Redd]

Is there a way that I can change the prompt for within a browser() call.  I
often use use code like

> with(obj1,browser())
Browse[1]>

Is there a way that I can set it so that I can get something like

> with(obj1,browser(prompt="obj1"))
obj1[1]>

I know that prompt is not a valid option for browser, but it would be nice
if it were.   There is an option('prompt")  but that does not affect the
prompt for browser.  Can I change this and how?

Thanks,
Andrew

[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] [patch] add is.set parameter to sample()

[Andrew Clausen]

Hi all,

sample() has some well-documented undesirable behaviour.

sample(1:6, 1)
sample(2:6, 1)
...
sample(5:6, 1)

do what you expect, but

sample(6:6, 1)
sample(1:6, 1)

do the same thing.

This behaviour is documented:

 If 'x' has length 1, is numeric (in the sense of 'is.numeric') and
 'x >= 1', sampling _via_ 'sample' takes place from '1:x'.  _Note_
 that this convenience feature may lead to undesired behaviour when
 'x' is of varying length 'sample(x)'.  See the 'resample()'
 example below.

My proposal is to add an extra parameter is.set to sample() to control
this behaviour.  If the parameter is unspecified, then we keep the old
behaviour for compatibility.  If it is TRUE, then we treat the first
parameter x as a set.  If it is FALSE, then we treat it as a set size.
 This means that

sample(6:6, 1, is.set=TRUE)

would return 6 with probability 1.

I have attached a patch to implement this new option.

Cheers,
Andrew
diff --git a/src/library/base/R/sample.R b/src/library/base/R/sample.R
index 8d22469..ddf7cf0 100644
--- a/src/library/base/R/sample.R
+++ b/src/library/base/R/sample.R
@@ -14,13 +14,17 @@
 #  A copy of the GNU General Public License is available at
 #  http://www.r-project.org/Licenses/
 
-sample <- function(x, size, replace=FALSE, prob=NULL)
+sample <- function(x, size, replace=FALSE, prob=NULL, is.set=NULL)
 {
-if(length(x) == 1L && is.numeric(x) && x >= 1) {
+is.natural <- function(x) length(x) == 1L && is.integer(x) && x > 1
+if(is.set == NULL) is.set <- !is.natural(x)
+if(!is.set) {
+	stopifnot(is.natural(x))
 	if(missing(size)) size <- x
 	.Internal(sample(x, size, replace, prob))
 }
 else {
+	stopifnot(length(x) >= 1)
 	if(missing(size)) size <- length(x)
 	x[.Internal(sample(length(x), size, replace, prob))]
 }
diff --git a/src/library/base/man/sample.Rd b/src/library/base/man/sample.Rd
index 3929ff2..811fed2 100644
--- a/src/library/base/man/sample.Rd
+++ b/src/library/base/man/sample.Rd
@@ -12,26 +12,31 @@
   of \code{x} using either with or without replacement.
 }
 \usage{
-sample(x, size, replace = FALSE, prob = NULL)
+sample(x, size, replace = FALSE, prob = NULL, is.set=NULL)
 
 sample.int(n, size, replace = FALSE, prob = NULL)
 }
 \arguments{
   \item{x}{Either a (numeric, complex, character or logical) vector of
-more than one element from which to choose, or a positive integer.}
+elements from which to choose, or a positive integer.  The interpretation
+depends on is.set, or heuristics described below.}
   \item{n}{a non-negative integer, the number of items to choose from.}
   \item{size}{positive integer giving the number of items to choose.}
   \item{replace}{Should sampling be with replacement?}
   \item{prob}{A vector of probability weights for obtaining the elements
 of the vector being sampled.}
+  \item{is.set}{A vector of probability weights for obtaining the elements
+of the vector being sampled.}
 }
 \details{
-  If \code{x} has length 1, is numeric (in the sense of
-  \code{\link{is.numeric}}) and \code{x >= 1}, sampling \emph{via}
-  \code{sample} takes place from
-  \code{1:x}.  \emph{Note} that this convenience feature may lead to
-  undesired behaviour when \code{x} is of varying length
-  \code{sample(x)}.  See the \code{resample()} example below.
+  The \code{is.set} parameter controls whether the \code{x} is interpreted as a
+  set of items to sample from (when \code{is.set} is \code{TRUE}), or the size
+  of the set of samples (when \code{is.set} is \code{FALSE}), in which case
+  the sample set is \code{1:x}.  If \code{is.set} is unspecified, then
+  \code{is.set} is set to \code{FALSE} when \code{x} has length 1, is numeric
+  (in the sense of \code{\link{is.numeric}}) and \code{x >= 1}.  \emph{Note}
+  that when \code{x} is a vector of varying size, leaving \code{is.set} can
+  lead to undesirable behaviour.
 
   By default \code{size} is equal to \code{length(x)}
   so that \code{sample(x)} generates a random permutation
@@ -93,13 +98,9 @@ x <- 1:10
 sample(x[x >  9]) # oops -- length 10!
 try(sample(x[x > 10]))# error!
 
-## This is safer, but only for sampling without replacement
-resample <- function(x, size, ...)
-  if(length(x) <= 1) { if(!missing(size) && size == 0) x[FALSE] else x
-  } else sample(x, size, ...)
-
-resample(x[x >  8])# length 2
-resample(x[x >  9])# length 1
-resample(x[x > 10])# length 0
+## This is safer
+sample(x[x >  8], is.set=TRUE)# length 2
+sample(x[x >  9], is.set=TRUE)# length 1
+sample(x[x > 10], is.set=TRUE)# length 0
 }
 \keyword{distribution}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [patch] add is.set parameter to sample()

[Andrew Clausen]

Hi all,

I forgot to test my patch!  I fixed a few bugs.

Cheers,
Andrew

On 22 March 2010 22:53, Andrew Clausen  wrote:
> Hi all,
>
> sample() has some well-documented undesirable behaviour.
>
> sample(1:6, 1)
> sample(2:6, 1)
> ...
> sample(5:6, 1)
>
> do what you expect, but
>
> sample(6:6, 1)
> sample(1:6, 1)
>
> do the same thing.
>
> This behaviour is documented:
>
>     If 'x' has length 1, is numeric (in the sense of 'is.numeric') and
>     'x >= 1', sampling _via_ 'sample' takes place from '1:x'.  _Note_
>     that this convenience feature may lead to undesired behaviour when
>     'x' is of varying length 'sample(x)'.  See the 'resample()'
>     example below.
>
> My proposal is to add an extra parameter is.set to sample() to control
> this behaviour.  If the parameter is unspecified, then we keep the old
> behaviour for compatibility.  If it is TRUE, then we treat the first
> parameter x as a set.  If it is FALSE, then we treat it as a set size.
>  This means that
>
> sample(6:6, 1, is.set=TRUE)
>
> would return 6 with probability 1.
>
> I have attached a patch to implement this new option.
>
> Cheers,
> Andrew
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [patch] add is.set parameter to sample()

[Andrew Clausen]

Hi Martin,

I re-attached the patch with a filename that will hopefully get
through the filters this time.

I agree that the case that you want to specify an integer is already
well handled with sample.int().  I disagree that the resample() code
for the set case given in the example is trivial.  The user has to
load the code into their program, which is annoying for such basic
functionality.  Moreover, the example code doesn't work for sampling
with replacement, and is poorly documented.  Finally, it isn't obvious
to new users of R what to do with resample().  (They would probably
try using resample() without cutting & pasting it into their program.
And why is it called resample()?  It's a mysterious name, that
suggests some technical concept, like resampling digital audio from
one sampling rate to another.)

So, the upside of my patch is that sample() becomes more convenient,
and the documentation becomes simpler.  What's the downside?  It is
backwards compatible.

sample() is one of the most important functions in R... I teach it to
my undergraduate economics students in the first 20 minutes of their
first R lesson.  It is the first probability/statistics function they
learn.  It is important that it is easy and convenient to use.

My first R problem set that I assigned my students was to do a Monte
Carlo simulation of the Monty Hall problem.  sample()'s surprise
really bites here because Monty has either one or two choices of door
to open.  It's bad enough that there is a surprise, but even worse
that there is no workaround that my students can understand easily.

Cheers,
Andrew

On 25 March 2010 06:53, Martin Maechler  wrote:
>>>>>> "AndrewC" == Andrew Clausen 
>>>>>>     on Tue, 23 Mar 2010 08:04:12 -0400 writes:
>
>    AndrewC> Hi all,
>    AndrewC> I forgot to test my patch!  I fixed a few bugs.
>
> and this time, you even forgot to attach it (in a way to pass
> through the list filters).
>
> Note however, that all this seems unnecessary,
> as we have  sample.int()
> and a trivial definition of resample()
> at least in R-devel, which will be released as R 2.11.0 on
> April 22.
>
> Thank you anyway, for your efforts!
> Martin
>
> Martin Maechler, ETH Zurich
>
>    AndrewC> On 22 March 2010 22:53, Andrew Clausen  
> wrote:
>    >> Hi all,
>    >>
>    >> sample() has some well-documented undesirable behaviour.
>    >>
>    >> sample(1:6, 1)
>    >> sample(2:6, 1)
>    >> ...
>    >> sample(5:6, 1)
>    >>
>    >> do what you expect, but
>    >>
>    >> sample(6:6, 1)
>    >> sample(1:6, 1)
>    >>
>    >> do the same thing.
>    >>
>    >> This behaviour is documented:
>    >>
>    >>     If 'x' has length 1, is numeric (in the sense of 'is.numeric') and
>    >>     'x >= 1', sampling _via_ 'sample' takes place from '1:x'.  _Note_
>    >>     that this convenience feature may lead to undesired behaviour when
>    >>     'x' is of varying length 'sample(x)'.  See the 'resample()'
>    >>     example below.
>    >>
>    >> My proposal is to add an extra parameter is.set to sample() to control
>    >> this behaviour.  If the parameter is unspecified, then we keep the old
>    >> behaviour for compatibility.  If it is TRUE, then we treat the first
>    >> parameter x as a set.  If it is FALSE, then we treat it as a set size.
>    >>  This means that
>    >>
>    >> sample(6:6, 1, is.set=TRUE)
>    >>
>    >> would return 6 with probability 1.
>    >>
>    >> I have attached a patch to implement this new option.
>    >>
>    >> Cheers,
>    >> Andrew
>    >>
>    AndrewC> __
>    AndrewC> R-devel@r-project.org mailing list
>    AndrewC> https://stat.ethz.ch/mailman/listinfo/r-devel
>
diff --git a/src/library/base/R/sample.R b/src/library/base/R/sample.R
index 8d22469..01498c0 100644
--- a/src/library/base/R/sample.R
+++ b/src/library/base/R/sample.R
@@ -14,13 +14,17 @@
 #  A copy of the GNU General Public License is available at
 #  http://www.r-project.org/Licenses/
 
-sample <- function(x, size, replace=FALSE, prob=NULL)
+sample <- function(x, size, replace=FALSE, prob=NULL, is.set=NULL)
 {
-if(length(x) == 1L && is.numeric(x) && x >= 1) {
+is.natural <- function(x) length(x) == 1L && is.numeric(x) && x >= 1
+if(is.null(is.set)) is.set <- !is.natural(x)
+if(!is.set) {
+	stopifnot(is.natural

Re: [Rd] Memory allocation in C/C++ vs R?

[Andrew Runnalls]

Dominick,

On 30/04/10 18:40, Dominick Samperi wrote:
> Just to be sure that I understand, are you suggesting that the R-safe way to 
> do
> things is to not use STL, and to not use C++ memory management and
> exception handling? How can you leave a function in an irregular way without
> triggering a seg fault or something like that, in which case there is no 
> chance
> for recovery anyway?

If you had time to try out your work with CXXR, I would be very
interested to hear your experiences.  Picking up on some of the points
already raised in this thread:

1. CXXR's main program is written in C++, so it should deal with C++
initialisation and teardown correctly.  (However, it has been so far
tested only with gcc and the Intel C++ compiler, so there are doubtless
plenty of submerged rocks still to discover.)

2. Within CXXR, all indirect flows of control are handled using C++
exceptions, and it is certainly the intention that user-supplied code
should also be free to use C++ exceptions (subject to the caution that
is always necessary in using C++ exceptions).

3. As regards memory allocation, in addition to the mechanisms available
in standard R, you can of course use standard C++ 'new' and 'delete'.
Also, objects of any C++ class that inherits directly or indirectly from
CXXR::GCNode will be looked after by CXXR's own internal garbage collection.

4. Regarding protection from garbage collection, the mechanisms of
standard R are available.  But for C++ code, the preferred approach is
to use the CXXR::GCRoot and CXXR::GCStackRoot smart pointers.
Basically an object of one of these types behaves like a plain T*
pointer (where T inherits from CXXR::GCNode), but whatever it points to
is protected from garbage collection - for as long as the pointer exists
and points to that object.

Let me know (offline from this list) if I can be of any help.

Andrew Runnalls

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] How to customize the list of exported functions in a shared library

[Andrew Piskorski]

On Mon, Feb 05, 2007 at 01:28:24AM -0800, Vladimir Eremeev wrote:
> I am writing binding from C library to R.
> I use R 2.4.1, windows XP, and MinGW.

> R CMD SHLIB -d --output=Rsnns.dll  [ list of all C sources]

> R CMD SHLIB -d --output Rsnns.dll Rsnns.c 
> -Wl,-Lc:/mingw/lib,-lfl,-L../sources,-lkernel,-lfunc

You should probably also show us the actual compiler/linker commands
that "R CMD SHLIB" is generating, so we can be sure of what's really
going on.

I'm not sure what you may have to do differently with MinGW on Windows
(what linker does that use anyway?), but for comparison, here's how I
do it for R 2.2.1, on Linux (Ubuntu Dapper 6.06), with gcc 4.0.3, and
Gnu ld 2.16.91:

For my own custom R package's C code, my Makefile says:

  OBJ = ../foo.o $(patsubst %.c,%.o,$(wildcard ../source/[a-z]*.c))
  $(OBJ): %.o:  %.c $(HDRS)
  R.so: $(OBJ) Makevars vis.map
  R CMD SHLIB -o my_pkg_name_R.so $(OBJ)

My Makevars includes this:

  PKG_LIBS = -Wl,--version-script=vis.map

And when I build my R package, the R.so make target above generates
this link command:

  gcc -shared -o my_pkg_name_R.so [lots of *.o filenames here] 
-Wl,--version-script=vis.map -L/usr/lib/R/lib -lR

My vis.map file, which uses Gnu ld syntax, looks like this:

  {
 global: my_pkg_*;
 local:*;
  };

That works, my shared library exports ONLY the symbols starting with
"my_pkg_", everything else remains private.

It's the "--version-script" linker option doing the magic.  Even with
Gnu ld, there are definitely other ways to control symbol visibility,
but that one seemed most convenient in my case.

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] compile 2.4.1 for linux on power cpus

[Andrew Ferris]

Hello,

I've been asked to get R (2.4.1) installed on a IBM p570 server. This is a 
server with power CPUs and is running SLES 10. It currently has 12GB of RAM so 
I'd like to make sure I have the 64 bit version of R so as to take advantage of 
the extra memory. Since it's a power CPU server that means I'll have to compile 
R from source. 

I've searched the r_users list but most of the power CPU correspondence 
concerns either AIX or Macs. SLES 10 uses GCC 4.1.0 by default. Could someone 
please help me with some guidance on the necessary parameters for configure to 
ensure a 64 bit version of R is compiled? 
 

Andrew Ferris
Network Support Analyst
iCAPTURE Research Centre
University of British Columbia


***CONFIDENTIALITY NOTICE***
This electronic message is intended only for the use of the addressee and may 
contain information that is privileged and confidential.  Any dissemination, 
distribution or copying of this communication by unauthorized individuals is 
strictly prohibited. If you have received this communication in error, please 
notify the sender immediately by reply e-mail and delete the original and all 
copies from your system.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] compile 2.4.1 for linux on power cpus

[Andrew Ferris]

Thank you for the reply Peter. 

I've compiled R from source using the following:

./configure --host=powerpc64-power5-linux-gnu 
--build=powerpc64-power5-linux-gnu '--with-blas=-framework blas-3.1.0-11'

and after I've made R I get this:

> .Machine$sizeof.pointer
[1] 4

meanwhile uname -a prints out this:

Linux [hostname] 2.6.16.21-0.8-ppc64 #1 SMP Mon Jul 3 18:25:39 UTC 2006 ppc64 
ppc64 ppc64 GNU/Linux

and in the root filesystem there's a /lib and /lib64. I suspect that R needs to 
have the 64 bit libraries specified so is there a way to do that?

Andrew Ferris
Network Support Analyst
iCAPTURE Research Centre
University of British Columbia

>>> Peter Dalgaard <[EMAIL PROTECTED]> 03/14/07 1:34 PM >>>
Andrew Ferris wrote:
> Hello,
>
> I've been asked to get R (2.4.1) installed on a IBM p570 server. This is a 
> server with power CPUs and is running SLES 10. It currently has 12GB of RAM 
> so I'd like to make sure I have the 64 bit version of R so as to take 
> advantage of the extra memory. Since it's a power CPU server that means I'll 
> have to compile R from source. 
>
> I've searched the r_users list but most of the power CPU correspondence 
> concerns either AIX or Macs. SLES 10 uses GCC 4.1.0 by default. Could someone 
> please help me with some guidance on the necessary parameters for configure 
> to ensure a 64 bit version of R is compiled? 
>  
>
>   
Just running the 64bit version of the OS usually suffices (at least on 
Linux variants). Easiest check on a running R is

 > .Machine$sizeof.pointer
[1] 4

which gives 8 on 64-bit builds.





***CONFIDENTIALITY NOTICE***
This electronic message is intended only for the use of the addressee and may 
contain information that is privileged and confidential.  Any dissemination, 
distribution or copying of this communication by unauthorized individuals is 
strictly prohibited. If you have received this communication in error, please 
notify the sender immediately by reply e-mail and delete the original and all 
copies from your system.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] compile 2.4.1 for linux on power cpus

[Andrew Ferris]

Peter,

First off, as you may have guessed, I don't compile many 64 bit programs so 
thanks again for the help. I'll revert back to powerpc64-unknown-linux-gnu 
which is the default for -build and -host.

Here's the gcc information
[hostname]:/ # which gcc
/usr/bin/gcc
[hostname]:/ # gcc -dumpmachine
powerpc64-suse-linux

>From looking at the GNU documentation for GCC  - IBM RS/6000 and PowerPC 
>Options, I see that it mentions this option:

-m64

Generate code for 32-bit or 64-bit environments of Darwin and SVR4 targets 
(including GNU/Linux). The 32-bit environment sets int, long and pointer to 32 
bits and generates code that runs on any PowerPC variant. The 64-bit 
environment sets int to 32 bits and long and pointer to 64 bits, and generates 
code for PowerPC64, as for -mpowerpc64. 

So would some compiler flags such as these work:

'CC=gcc -m64' 'CXX=g++ -m64' 'FC=gfortran -mc64' 'F77=gfortran -m64' 
'LDFLAGS=-L/lib64'



Andrew Ferris
Network Support Analyst
iCAPTURE Research Centre
University of British Columbia

>>> Peter Dalgaard <[EMAIL PROTECTED]> 03/14/07 5:03 PM >>>
Andrew Ferris wrote:
> Thank you for the reply Peter. 
>
> I've compiled R from source using the following:
>
> ./configure --host=powerpc64-power5-linux-gnu 
> --build=powerpc64-power5-linux-gnu '--with-blas=-framework blas-3.1.0-11'
>
> and after I've made R I get this:
>
>   
>> .Machine$sizeof.pointer
>> 
> [1] 4
>
> meanwhile uname -a prints out this:
>
> Linux [hostname] 2.6.16.21-0.8-ppc64 #1 SMP Mon Jul 3 18:25:39 UTC 2006 ppc64 
> ppc64 ppc64 GNU/Linux
>
>   
So I was wrong in assuming that 64 bit SLES would be set up for 64 bit 
compiles.

> and in the root filesystem there's a /lib and /lib64. I suspect that R needs 
> to have the 64 bit libraries specified so is there a way to do that?
First check the toolchain:

which gcc
gcc -dumpmachine

You might need to revise your path and/or install 64 bit versions of the 
compilers.

Actually, looking at the package list for SLES 10, I see that some 
packages have a -64bit variant, e.g.

glibc-64bit 2.4 
<http://www.novell.com/products/linuxpackages/server10/ppc/glibc-64bit.html> 
(Standard Shared Libraries (from the GNU C Library))
glibc-devel-64bit 2.4 
<http://www.novell.com/products/linuxpackages/server10/ppc/glibc-devel-64bit.html>
 
(Include Files and Libraries Mandatory for Development)

but the gcc package does not, and the gcc-fortran-64bit 4.1.0 
<http://www.novell.com/products/linuxpackages/server10/ppc/gcc-fortran-64bit.html>
 
packages contains only libraries. So my guess is that there is one 
compiler that does both 32 bit and 64 bit compiling, but you need to set 
a compiler flag for the latter.

I don't think messing with --build and --host is likely to do any good.



***CONFIDENTIALITY NOTICE***
This electronic message is intended only for the use of the addressee and may 
contain information that is privileged and confidential.  Any dissemination, 
distribution or copying of this communication by unauthorized individuals is 
strictly prohibited. If you have received this communication in error, please 
notify the sender immediately by reply e-mail and delete the original and all 
copies from your system.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] allocVector reference

[Andrew Piskorski]

On Wed, Mar 14, 2007 at 05:44:04PM -, Matthew Dowle wrote:
> 
> Hi,
> 
> I'm trying to write a function to return an R vector which points
> directly to a contiguous subset of another vector, without taking a
> copy.

Matthew, I don't know the answer to your question, but this all seems
related to support for references in R.  I've included my notes on R
references below.

Ah, I think Jens Oehlschlaegel's "ref" package is what you want:

  http://tolstoy.newcastle.edu.au/R/packages/04/0008.html

  Class refdata is a transparent wrapper to matrices and data.frames
  which allows for memory efficient nested subsetting. I.e. you can
  create a subset of a subset ... of a data.frame without duplicating
  the data in memory, instead only indexing information is duplicated.

> Since SEXPREC is a header followed by the data, rather than the
> header plus a pointer to the data, I'm not sure what I'm trying to
> do is possible.  Is there a way of doing this?  Similar in spirit to
> how the R assignment "x=y" does not copy y until x is modified,

Is you last statement above in fact true?  I was under the impression
that R does NOT do copy on write, that when you make a copy of a
variable, R immediately allocates memory and makes a deep copy of the
value.

But you're using old deprecated "=" for assignment, which is weird, so
maybe you mean when you pass named arguments to a function?  Function
args are evaluated lazily, and I think that is used (I don't know how
exactly) to give copy on write behavior - but only for function
arguments.

My (older) notes on R references:

R does not support reference variables and does not do copy on write -
when you copy a variable, it eagerly allocates all memory and makes a
deep copy.  (I believe S-Plus has the same behavior but have not
checked.)  This can be annoying...

There are however specific exceptions.  For example, R environments
are treated as references, NOT values like most everything else in R.
The July 2006 "attributes of environments" thread makes that clear:

  https://stat.ethz.ch/pipermail/r-devel/2006-July/038352.html

Jens Oehlschlaegel's ref package implements references for both R and
S-Plus:

  http://cran.r-project.org/src/contrib/Descriptions/ref.html
  http://www.maths.lth.se/help/R/.R/library/ref/html/00Index.html
  http://tolstoy.newcastle.edu.au/~rking/R/packages/04/0008.html

Henrik Bengtsson's R.oo package emulates reference variables via R
environments (but perhaps only in the context of his OO framework, I'm
not sure).

  http://www.braju.com/R/
  http://cran.r-project.org/src/contrib/Descriptions/R.oo.html
  http://www.maths.lth.se/help/R/R.oo/

Bengtsson also wrote a 2002 paper, "Implementing support for
references in R":

  http://www.maths.lth.se/help/R/ImplementingReferences/

Further out, see also Tierny's (mostly old) R development notes

  Notes on References, External Objects, or Mutable State for R:
http://www.stat.uiowa.edu/~luke/R/references.html
  Simple References with Finalization:
http://www.stat.uiowa.edu/~luke/R/simpleref.html
  Finalization and Weak References in R:
http://www.stat.uiowa.edu/~luke/R/references/weakfinex.html

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] compile 2.4.1 for linux on power cpus

[Andrew Ferris]

I'm closer but still not quite there. Here's the configure command I'm using:

 ./configure 'CC=gcc -m64' 'CXX=g++ -m64 -mminimal-toc' 'FC=gfortran -mc64 
-fno-optimize-sibling-calls' 'F77=gfortran -m64 -fno-optimize-sibling-calls' 
'LDFLAGS=-L/usr/lib64' R_PAPERSIZE='letter'

I'm using the -mminimal-toc and -fno-optimize-sibling-calls flags because of 
this error during make:

/usr/bin/ld: ../nmath/libnmath.a(gamma.o)(.text+0xf4): sibling call 
optimization to `Rf_chebyshev_eval' does not allow automatic multiple TOCs; 
recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make 
`Rf_chebyshev_eval' extern
/usr/bin/ld: ../nmath/libnmath.a(gamma.o)(.text+0x270): sibling call 
optimization to `Rf_stirlerr' does not allow automatic multiple TOCs; recompile 
with -mminimal-toc or -fno-optimize-sibling-calls, or make `Rf_stirlerr' extern
/usr/bin/ld: ../nmath/libnmath.a(gamma.o)(.text+0x34c): sibling call 
optimization to `Rf_lgammacor' does not allow automatic multiple TOCs; 
recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make 
`Rf_lgammacor' extern
/usr/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
make[3]: *** [R.bin] Error 1
make[3]: Leaving directory `/usr/local/R-2.4.1/src/main'
make[2]: *** [R] Error 2
make[2]: Leaving directory `/usr/local/R-2.4.1/src/main'
make[1]: *** [R] Error 1
make[1]: Leaving directory `/usr/local/R-2.4.1/src'
make: *** [R] Error 1

But those flags aren't helping as those errors persist. Is there anything else 
I can do?

thanks,


Andrew Ferris
Network Support Analyst
iCAPTURE Research Centre
University of British Columbia

>>> Prof Brian Ripley <[EMAIL PROTECTED]> 03/15/07 12:11 AM >>>
On Thu, 15 Mar 2007, Peter Dalgaard wrote:

> Andrew Ferris wrote:
>> Peter,
>>
>> First off, as you may have guessed, I don't compile many 64 bit programs so 
>> thanks again for the help. I'll revert back to powerpc64-unknown-linux-gnu 
>> which is the default for -build and -host.
>>
>> Here's the gcc information
>> [hostname]:/ # which gcc
>> /usr/bin/gcc
>> [hostname]:/ # gcc -dumpmachine
>> powerpc64-suse-linux
>>
>> From looking at the GNU documentation for GCC  - IBM RS/6000 and PowerPC 
>> Options, I see that it mentions this option:
>>
>> -m64
>>
>> Generate code for 32-bit or 64-bit environments of Darwin and SVR4 targets 
>> (including GNU/Linux). The 32-bit environment sets int, long and pointer to 
>> 32 bits and generates code that runs on any PowerPC variant. The 64-bit 
>> environment sets int to 32 bits and long and pointer to 64 bits, and 
>> generates code for PowerPC64, as for -mpowerpc64.
>>
>> So would some compiler flags such as these work:
>>
>> 'CC=gcc -m64' 'CXX=g++ -m64' 'FC=gfortran -mc64' 'F77=gfortran -m64' 
>> 'LDFLAGS=-L/lib64'
>>
>>
>
> That's likely. Or use CFLAGS=-m64, and FFLAGS, CXXFLAGS similarly.

The CC etc forms are preferred, as not all configure scripts use the 
environment CFLAGS.

See the Solaris 64-bit notes in the R-admin manual for proven examples.

>
> I'd try compiling a simple hello.c program first. Try e.g. "gcc -m64
> hello.c" and see what "file a.out" has to say about the result.
>
> You may also find yourself having to install a number of packages  with
> names like foo-64bit_xx.yy to get 64bit C libraries, but configure
> should tell you about any missing bits in due course, once you have it
> convinced not to build for 32bit.
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel 
>

-- 
Brian D. Ripley,  [EMAIL PROTECTED] 
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/ 
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595



***CONFIDENTIALITY NOTICE***
This electronic message is intended only for the use of the addressee and may 
contain information that is privileged and confidential.  Any dissemination, 
distribution or copying of this communication by unauthorized individuals is 
strictly prohibited. If you have received this communication in error, please 
notify the sender immediately by reply e-mail and delete the original and all 
copies from your system.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] compile 2.4.1 for linux on power cpus SOLVED

[Andrew Ferris]

Thank you Ei-ji, 

That seems to have done it.

> .Machine$sizeof.pointer
[1] 8

So in total I've done the following to get it work:

Installed these readline rpms from the SLES10 media:

readline-devel-64bit-5.1-24.4
readline-devel-5.1-24.4

This is the configure command:

./configure CC="gcc -m64" /
CXX="gxx -m64" /
F77="gfortran -m64" / 
FC="gfortran -m64" /
CFLAGS="-mminimal-toc -fno-optimize-sibling-calls -g -O2" /
FFLAGS="-mminimal-toc -fno-optimize-sibling-calls -g -O2" /
LDFLAGS=-L/usr/lib64 /
--without-x

which gets this:

R is now configured for powerpc64-unknown-linux-gnu

  Source directory:  .
  Installation directory:/usr/local

  C compiler:gcc -m64 -std=gnu99  -mminimal-toc 
-fno-optimize-sibling-calls -g -O2
  Fortran 77 compiler:   gfortran -m64  -mminimal-toc 
-fno-optimize-sibling-calls -g -O2

  C++ compiler:  gxx -m64
  Fortran 90/95 compiler:gfortran -m64 -g -O2

  Interfaces supported:
  External libraries:readline
  Additional capabilities:   iconv, MBCS, NLS
  Options enabled:   shared BLAS, R profiling

  Recommended packages:  yes

configure: WARNING: I could not determine CXXPICFLAGS.
configure: WARNING: I could not determine SHLIB_CXXLDFLAGS
configure: WARNING: I could not determine CXXPICFLAGS.
configure: WARNING: you cannot build DVI versions of the R manuals
configure: WARNING: you cannot build PDF versions of the R manuals
configure: WARNING: I could not determine a PDF viewer

And that gets through make with no errors. That's R 2.4.1 on SLES10 on a power5 
CPU server. 

Thank-you to Peter Dalgaard and Prof. Ripley for their help with this.

Andrew Ferris
Network Support Analyst
iCAPTURE Research Centre
University of British Columbia

>>> "Ei-ji Nakama" <[EMAIL PROTECTED]> 3/15/2007 7:01 PM >>>
2007/3/16, Andrew Ferris <[EMAIL PROTECTED]>:
> I'm closer but still not quite there. Here's the configure command I'm using:
>
>  ./configure 'CC=gcc -m64' 'CXX=g++ -m64 -mminimal-toc' 'FC=gfortran -mc64 
> -fno-optimize-sibling-calls' 'F77=gfortran -m64 -fno-optimize-sibling-calls' 
> 'LDFLAGS=-L/usr/lib64' R_PAPERSIZE='letter'

$ uname -a
Linux macg5 2.6.18-3-powerpc64 #1 SMP Mon Dec 4 15:40:16 CET 2006
ppc64 GNU/Linux

$ gcc -v
Using built-in specs.
Target: powerpc-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--program-suffix=-4.1 --enable-__cxa_atexit --enable-clocale=gnu
--enable-libstdcxx-debug --enable-mpfr --disable-softfloat
--enable-targets=powerpc-linux,powerpc64-linux --with-cpu=default32
--enable-checking=release powerpc-linux-gnu
Thread model: posix
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

$ gcc -print-multi-lib
.;@[EMAIL PROTECTED]
64;@[EMAIL PROTECTED]@mstrict-align

$ ./configure CC="gcc -m64" \
   CXX="gxx -m64" \
   F77="gfortran -m64" \
   FC="gfortran -m64" \
   CFLAGS="-mminimal-toc -fno-optimize-sibling-calls -g -O2" \
   FFLAGS="-mminimal-toc -fno-optimize-sibling-calls -g -O2" \
   --without-x

$ file bin/exec/R
bin/exec/R: ELF 64-bit MSB executable, PowerPC 64-bit or cisco 7500,
version 1 (SYSV), for GNU/Linux 2.6.0, dynamically linked (uses shared
libs), for GNU/Linux 2.6.0, not stripped

-- 
EI-JI Nakama  <[EMAIL PROTECTED]>
"\u4e2d\u9593\u6804\u6cbb"  <[EMAIL PROTECTED]>



***CONFIDENTIALITY NOTICE***
This electronic message is intended only for the use of the addressee and may 
contain information that is privileged and confidential.  Any dissemination, 
distribution or copying of this communication by unauthorized individuals is 
strictly prohibited. If you have received this communication in error, please 
notify the sender immediately by reply e-mail and delete the original and all 
copies from your system.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Matrix package: compilation error

[Andrew Robinson]

Hi Rainer,

check the following post for an alternative solution:

http://tolstoy.newcastle.edu.au/R/help/06/01/18908.html

if you would like more detailed instructions, let me know.

Andrew


On Sat, Mar 31, 2007 at 10:10:45PM +0200, Rainer Hurling wrote:
> Thanks, Brian and Martin,
> 
> I think you are both right, Matrix tries to use BSD make (/usr/bin/make) 
> on FreeBSD instead of GNU make (/usr/local/bin/gmake).
> 
> Sorry, but I don't know how to persuade the configure script to use 
> gmake :-(
> 
> Rainer
> 
> 
> Prof Brian Ripley schrieb:
> > This is because of the GNUism in Matrix/src/Makefile
> > 
> > ## get rid of this, once we have 'Depends: R (>= 2.5.0)':
> > ifeq (, $(findstring -lRlapack, $(LAPACK_LIBS)))
> > SOURCES_LAPACK =
> > else
> > SOURCES_LAPACK = zpotf2.f zpotrf.f zlacgv.f
> > endif
> > 
> > I guess you know what you need to do to fix it for BSD make?
> > 
> > On Sat, 31 Mar 2007, Rainer Hurling wrote:
> > 
> >> Trying to compile the package Matrix_0.9975-11.tar.gz with newest
> >> R-2.5.0 alpha (2007-03-31 r40986) on FreeBSD 7.0-CURRENT (i386) I get
> >> the following error:
> >>
> >> -
> >> R CMD INSTALL Matrix_0.9975-11.tar.gz
> >> * Installing to library '/usr/local/lib/R/library'
> >> * Installing *source* package 'Matrix' ...
> >> ** libs
> >> ** arch -
> >> "Makefile", line 10: Missing dependency operator
> >> "Makefile", line 12: Need an operator
> >> "Makefile", line 14: Need an operator
> >> make: fatal errors encountered -- cannot continue
> >> ERROR: compilation failed for package 'Matrix'
> >> ** Removing '/usr/local/lib/R/library/Matrix'
> >> -
> >>
> >>
> >> Under FreeBSD I have installed the LAPACK package (3.0.2) with library
> >> at location
> >>
> >> /usr/local/lib/liblapack.so.4
> >>
> >> Is it possible that the Makefile of package Matrix fails because of that?
> > 
> > Not used unless you asked for it during R's configure.
> >
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Andrew Robinson  
Department of Mathematics and StatisticsTel: +61-3-8344-9763
University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599
http://www.ms.unimelb.edu.au/~andrewpr
http://blogs.mbs.edu/fishing-in-the-bay/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] symbollic differentiation in R

[Andrew Clausen]

Hi all,

I wrote a symbollic differentiation function in R, which can be downloaded
here:

http://www.econ.upenn.edu/~clausen/computing/Deriv.R
http://www.econ.upenn.edu/~clausen/computing/Simplify.R

It is just a prototype.  Of course, R already contains two differentiation
functions: D and deriv.  However, these functions have several limitations.
They can probably be fixed, but since they are written in C, this would
require a lot of work.  Limitations include:
 * The derivatives table can't be modified at runtime, and is only available
in C.
 * The output of "deriv" can not be differentiated again.
 * Neither function can substitute function calls.  eg:
   f <- function(x, y) x + y; deriv(f(x, x^2), "x")
 * They can't differentiate vector-valued functions (although my code also
can't do this yet)

I think these limitations are fairly important.  As it stands, it's rather
difficult to automatically differentiate a likelihood function.  Ideally, I
would like to be able to write

ll <- function(mean, sd)
-sum(log(dnorm(x, mean, sd)))

ll.deriv <- Deriv.function(ll)

I can't get this to work with my code since:
 * since sum can't add a list of vectors (although I could easily write a sum
replacement.)
 * "x" is assumed to be a scalar in this contect.  I'm not sure if there's a
good way to generalize.

The above code would work right now if there were one parameter (so
sum doesn't screw it up) and one scalar data point "x".

Is there an existing way of doing this that is close to being this convenient?
Is it really much easier to solve the limitations I listed with a fresh
R implementation?

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi all,

I wrote a function called relist, which is an inverse to the existing
unlist function:

http://www.econ.upenn.edu/~clausen/computing/relist.R

Some functions need many parameters, which are most easily represented in
complex structures.  Unfortunately, many mathematical functions in R,
including optim, nlm, and grad can only operate on functions whose domain is
a vector.  R has a function to convert complex objects into a vector
representation.  This file provides an inverse operation called "unlist" to
convert vectors back to the convenient structural representation.  Together,
these functions allow structured functions to have simple mathematical
interfaces.

For example, a likelihood function for a multivariate normal model needs a
variance-covariance matrix and a mean vector.  It would be most convenient to
represent it as a list containing a vector and a matrix.  A typical parameter
might look like

list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))

However, optim can't operate on functions that take lists as input; it
only likes vectors.  The solution is conversion:

 initial.param <- list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))

 ll <- function(param.vector)
 {
param <- relist(initial.param, param.vector)
-sum(dnorm(x, mean=param$mean, vcov=param$vcov, log=TRUE))
# note: dnorm doesn't do vcov... but I hope you get the point
 }

 optim(unlist(initial.param), ll)

"relist" takes two parameters: skeleton and flesh.  Skeleton is a sample
object that has the right "shape" but the wrong content.  "flesh" is a vector
with the right content but the wrong shape.  Invoking

relist(skeleton, flesh)

will put the content of flesh on the skeleton.

As long as "skeleton" has the right shape, it should be a precise inverse
of unlist.  These equalities hold:

relist(skeleton, unlist(x)) == x
unlist(relist(skeleton, y)) == y

Is there any easy way to do this without my new relist function?  Is there any
interest in including this in R's base package?  (Or anywhere else?)  Any
comments on the implementation?  

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

On Sun, May 13, 2007 at 01:29:11PM -0400, Andrew Clausen wrote:
> R has a function to convert complex objects into a vector
> representation.  This file provides an inverse operation called "unlist" to
> convert vectors back to the convenient structural representation.

Oops.  I meant to say:

R has a function to convert complex objects into a vector representation called
"unlist".  This file provides an inverse operation called "relist" to convert
vectors back to the convenient structural representation.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi Gabor,

Thanks for the interesting suggestion.  I must confess I got lost -- is
it something like this?
 * unlist() could attach skeleton to every vector it returns.
 * relist() could then use the skeleton attached to the vector to reconstruct
the object.  The interface might be

relist <- function(flesh, skeleton=attributes(flesh)$skeleton)

For example:

par <- list(mean=c(0, 0), vcov(rbind(c(1, 1), c(1, 1
vector.for.optim <- unlist(par)
print(attributes(vector.optim)$skeleton)# the skeleton is stored!
converted.back.again <- relist(par)

Some concerns:
 * the metadata might get lost in some applications -- although it seems
to work fine with optim().  But, if we provide both interfaces (where
skeleton=flesh$skeleton is the default), then there should be no problem.
 * would there be any bad side-effects of changing the existing unlist
interface?  I suppose an option like "save.skeleton" could be added to unlist.
I expect there would be some objections to enabling this as default behaviour,
as it would significantly increase the storage requirements of the output.

Cheers,
Andrew

On Sun, May 13, 2007 at 07:02:37PM -0400, Gabor Grothendieck wrote:
> I suggest you define a "relist" class and then define an unlist
> method for it which stores the skeleton as an attribute.  Then
> one would not have to specify skeleton in the relist command
> so
> 
> relist(unlist(relist(x))) === x
> 
> 1. relist(x) is the same as x except it gets an additional class "relist".
> 2. unlist(relist(x)) invokes the relist method of unlist on relist(x)
> returning another relist object
> 3. relist(unlist(relist(x))) then recreates relist(x)
> 
> 
> On 5/13/07, Andrew Clausen <[EMAIL PROTECTED]> wrote:
> >Hi all,
> >
> >I wrote a function called relist, which is an inverse to the existing
> >unlist function:
> >
> >   http://www.econ.upenn.edu/~clausen/computing/relist.R
> >
> >Some functions need many parameters, which are most easily represented in
> >complex structures.  Unfortunately, many mathematical functions in R,
> >including optim, nlm, and grad can only operate on functions whose domain 
> >is
> >a vector.  R has a function to convert complex objects into a vector
> >representation.  This file provides an inverse operation called "unlist" to
> >convert vectors back to the convenient structural representation.  
> >Together,
> >these functions allow structured functions to have simple mathematical
> >interfaces.
> >
> >For example, a likelihood function for a multivariate normal model needs a
> >variance-covariance matrix and a mean vector.  It would be most convenient 
> >to
> >represent it as a list containing a vector and a matrix.  A typical 
> >parameter
> >might look like
> >
> >   list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
> >
> >However, optim can't operate on functions that take lists as input; it
> >only likes vectors.  The solution is conversion:
> >
> >initial.param <- list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
> >
> >ll <- function(param.vector)
> >{
> >   param <- relist(initial.param, param.vector)
> >   -sum(dnorm(x, mean=param$mean, vcov=param$vcov, log=TRUE))
> >   # note: dnorm doesn't do vcov... but I hope you get the 
> >   point
> >}
> >
> >optim(unlist(initial.param), ll)
> >
> >"relist" takes two parameters: skeleton and flesh.  Skeleton is a sample
> >object that has the right "shape" but the wrong content.  "flesh" is a 
> >vector
> >with the right content but the wrong shape.  Invoking
> >
> >   relist(skeleton, flesh)
> >
> >will put the content of flesh on the skeleton.
> >
> >As long as "skeleton" has the right shape, it should be a precise inverse
> >of unlist.  These equalities hold:
> >
> >   relist(skeleton, unlist(x)) == x
> >   unlist(relist(skeleton, y)) == y
> >
> >Is there any easy way to do this without my new relist function?  Is there 
> >any
> >interest in including this in R's base package?  (Or anywhere else?)  Any
> >comments on the implementation?
> >
> >Cheers,
> >Andrew
> >
> >__
> >R-devel@r-project.org mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-devel
> >

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] optim bug (PR#9684)

[Andrew Clausen]

On Tue, May 15, 2007 at 07:02:56PM +0100, Prof Brian Ripley wrote:
> In R you rarely need to pass additional arguments in programming as 
> lexical scoping can be used to capture them.

You can also use currying, like this:

ll <- function(data) function(params)
{
# compute whatever you want with data and params.
}

Then you can call optim like this:

optim(initial.param, ll(some.data))

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi all,

I've written a new version of relist that includes the suggestions from Gabor
and Martin:

http://www.econ.upenn.edu/~clausen/computing/relist.R

The leading example now looks like this:

initial.param <- list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
initial.param <- as.relistable(initial.param)

ll <- function(param.vector)
{
param <- relist(initial.param)
-sum(dnorm(x, mean=param$mean, vcov=param$vcov, log=TRUE))
# note: dnorm doesn't do vcov... but I hope you get the point
}

optim(unlist(initial.param), ll)

One thing that concerns me is that relist() needs to count how many items are
in a structued object.  In this version, it does that by repeatedly calling

length(unlist(obj))

which is quite inefficient (O(n^2) time, where n is the depth of the
structure).

Is there a clean way of making this faster?  I suppose relist() could
return attach a "length" attribute to the object it returns.

Apart from that, I suppose I should do these things before inclusion:
 * write some documentation (including pointers in unlist's docs)
 * write more relist methods (eg for character)

What's the usual process?  Email a patch to [EMAIL PROTECTED]

Cheers,
Andrew

On Mon, May 14, 2007 at 09:53:31AM +0200, Martin Maechler wrote:
> Nice ideas, Gabor and Andrew.
> 
> While I agree with Andrew that such a utility makes for nicer
> and considerably better maintainable code in examples like his,
> and I do like to provide "inverse operator functions" in R
> whenever sensible,
> OTOH, we have strived to keep R's "base" package as lean and
> clean as possible, so I think this had to go to "utils".
> 
> One further small proposal: I'd use class name  "relistable"
> since that's what the object of this class are
> and hence as.relistable().
> 
> What do other R-develers think?
> Martin
> 
> >>>>> "GaGr" == Gabor Grothendieck <[EMAIL PROTECTED]>
> >>>>> on Mon, 14 May 2007 02:54:22 -0400 writes:
> 
> GaGr> unlist would not attach a skeleton to every vector it
> GaGr> returns, only the relist method of unlist would.
> GaGr> That way just that method needs to be added and no
> GaGr> changes to unlist itself are needed.
> 
> GaGr> Before applying unlist to an object you would coerce
> GaGr> the object to class "relist" to force the relist
> GaGr> method of unlist to be invoked.
> 
> GaGr> Here is an outline of the code:
> 
> GaGr> as.relist <- function(x) {
> GaGr>  if (!inherits(x, "relist")) class(x) <- c("relist", class(x))
> GaGr>  x
> GaGr> }
> 
> GaGr> unlist.relist <- function(x, ...) {
> GaGr>  y <- x
> GaGr>  cl <- class(y)
> GaGr>  class(y) <- cl[- grep("relist", cl)]
> GaGr>  z <- unlist(y)
> GaGr>  attr(z, "relist") <- y
> GaGr>  as.relist(z)
> GaGr> }
> 
> GaGr> relist <- function(x, skeleton = attr(x, "relist")) {
> GaGr>  # simpler version of relist so test can be executed
> GaGr>  skeleton
> GaGr> }
> 
> GaGr> # test
> GaGr> x <- list(a = 1:2, b = 3)
> GaGr> class(as.relist(x))
> GaGr> unlist(as.relist(x))
> GaGr> relist(unlist(as.relist(x)))
> 
> 
> GaGr> On 5/14/07, Andrew Clausen <[EMAIL PROTECTED]> wrote:
> >> Hi GaGr,
> >> 
> >> Thanks for the interesting suggestion.  I must confess I got lost -- is
> >> it something like this?
> >> * unlist() could attach skeleton to every vector it returns.
> >> * relist() could then use the skeleton attached to the vector to 
> reconstruct
> >> the object.  The interface might be
> >> 
> >> relist <- function(flesh, skeleton=attributes(flesh)$skeleton)
> >> 
> >> For example:
> >> 
> >> par <- list(mean=c(0, 0), vcov(rbind(c(1, 1), c(1, 1
> >> vector.for.optim <- unlist(par)
> >> print(attributes(vector.optim)$skeleton)# the skeleton is stored!
> >> converted.back.again <- relist(par)
> >> 
> >> Some concerns:
> >> * the metadata might get lost in some applications -- although it seems
> >> to work fine with optim().  But, if we provide both interfaces (where
> >> skeleton=flesh$skeleton is the default), then there should be no 
> problem.
>

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi all,

For reasons I can't explain, the code I posted worked in my session, but didn't
work when I started a fresh one.  standardGeneric() seems to get confused
by defaults for missing arguments.  It looks for a "missing" method with
this code:

relist <- function(flesh, skeleton=attr(flesh, "skeleton"))
{
standardGeneric("relist")
}

I uploaded yet another new version that adds a wrapper that seems to
resolve standardGeneric()'s confusion like this:

relist <- function(flesh, skeleton=attr(flesh, "skeleton"))
{
# need a wrapper, since standardGeneric gets confused by
# default arguments.
f <- function(flesh, skeleton) standardGeneric("relist")

f(flesh, skeleton)
}

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi Seth,

On Mon, May 21, 2007 at 05:15:10PM -0700, Seth Falcon wrote:
> I will also add that the notion of a default argument on a generic
> function seems a bit odd to me.  If an argument is available for
> dispatch, I just don't see what sense it makes to have a default.  In
> those cases, the default should be handled by the method that has a
> signature with said argument matching the "missing" class.
> 
> What often does make sense is to define a generic function where some
> argument are not available for dispatch.  For example:
> 
> setGeneric("foo", signature="flesh",
>function(flesh, skeleton=attr(flesh, "skeleton") 
>standardGeneric("foo")))

That's an excellent suggestion.  Thanks!  However, I had to set the signature
to c("numeric", "missing") rather than just "numeric".

I have uploaded a new version here:

http://www.econ.upenn.edu/~clausen/computing/relist.R

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] relist, an inverse operator to unlist

[Andrew Clausen]

Hi Gabor,

Can you suggest some examples of how your proposal could be used?
Reshape never returns a vector.

Cheers,
Andrew

On Tue, May 22, 2007 at 07:36:56PM -0400, Gabor Grothendieck wrote:
> One additional idea.
> 
> I wonder if reshape might be promoted to a generic and relist made
> into methods for it.  The unlisted version of an object would be the "long"
> version and the original version of the list would be the "wide" version.
> 
> This would consolidate the two concepts together and make it
> easier to use since the user could leverage his knowledge of
> how reshape works to lists where it would work analogously.
> 
> Essentially reshape(myList, direction = "long") would be
> similar to unlist but would add the attributes need to reverse
> the procedure and reshape(myList, direction = "wide")
> would perform the inversion.
> 
> I am not sure if the reshape package has any bearing here
> as well.
> 
> On 5/22/07, Andrew Clausen <[EMAIL PROTECTED]> wrote:
> >Hi Seth,
> >
> >On Mon, May 21, 2007 at 05:15:10PM -0700, Seth Falcon wrote:
> >> I will also add that the notion of a default argument on a generic
> >> function seems a bit odd to me.  If an argument is available for
> >> dispatch, I just don't see what sense it makes to have a default.  In
> >> those cases, the default should be handled by the method that has a
> >> signature with said argument matching the "missing" class.
> >>
> >> What often does make sense is to define a generic function where some
> >> argument are not available for dispatch.  For example:
> >>
> >> setGeneric("foo", signature="flesh",
> >>function(flesh, skeleton=attr(flesh, "skeleton")
> >>standardGeneric("foo")))
> >
> >That's an excellent suggestion.  Thanks!  However, I had to set the 
> >signature
> >to c("numeric", "missing") rather than just "numeric".
> >
> >I have uploaded a new version here:
> >
> >   http://www.econ.upenn.edu/~clausen/computing/relist.R
> >
> >Cheers,
> >Andrew
> >
> >__
> >R-devel@r-project.org mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-devel
> >

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Optimization in R

[Andrew Clausen]

Hi all,

I've been working on improving R's optim() command, which does general purpose
unconstrained optimization.  Obviously, this is important for many statistics
computations, such as maximum likelihood, method of moments, etc.  I have
focused my efforts of the BFGS method, mainly because it best matches my
current projects.

Here's a quick summary of what I've done:
 * implemented my own version of BFGS in R,
http://www.econ.upenn.edu/~clausen/computing/bfgs.zip
 * written a wrapper for the GNU Scientific Library's optimization function,
multimin(),
http://www.econ.upenn.edu/~clausen/computing/multimin.zip
 * written some tricky functions to compare implementations,
http://www.econ.upenn.edu/~clausen/computing/tests.zip

My own implementation has several advantages over optim()'s implementation
(which you can see in the vmmin() function in
https://svn.r-project.org/R/trunk/src/main/optim.c)
 * the linesearch algorithm (More-Thuente) quickly finds a region of interest
to zoom into.  Moreover, it strikes a much better balance between finding
a point that adequately improves upon the old point, but doesn't waste too
much time finding a much better point.  (Unlike optim(), it uses the standard
Wolfe conditions with weak parameters.)
 * the linesearch algorithm uses interpolation, so it finds an acceptable
point more quickly.
 * implements "box" constraints.
 * easier to understand and modify the code, partly because it's written in R.

Of course, this comes at the (slight?) overhead cost of being written in R.

The test suite above takes the first few functions from the paper

Mor??, Garbow, and Hillstrom, "Testing Unconstrained
Optimization Software", ACM Trans Math Softw 7:1 (March 1981)

The test results appear below, where "*" means "computed the right solution",
and "!" means "got stuck".

testoptim   clausen gsl
--
bard!
beale
brown-scaled
freudenstein-roth
gaussian*
helical-valley  *   *
jennrich-sampson*
meyer   *
powell-scaled   *
rosenbrock  *

The table indiciates that all three implementations of BFGS failed to compute
the right answer in most cases.  I suppose this means they are all quite
deficient.  Of course, this doesn't imply that they perform badly on real
statistics problems -- but in my limited experience with my crude econometric
models, they do perform badly.   Indeed, that's why I started investigating in
the first place.

For what it's worth, I think:
 * the optimization algorithms should be written in R -- the overhead is
small compared to the cost of evaluating likelihood functions anyway, and is
easily made up by the better algorithms that are possible.
 * it would be useful to keep a repository of interesting optimization
problems relevant to R users.  Then R developers can evaluate "improvements".

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization in R

[Andrew Clausen]

Hi Pat,

On Sat, Aug 04, 2007 at 09:59:57AM +0100, Patrick Burns wrote:
> Sounds like a good project.

Thanks :)

> How much extra overhead are you getting from the
> algorithm being in R?

On the Rosenbrock function (which is very quick to evaluate), here are the
system.time() results:

> system.time(bfgs(x0, f, g))
[1] 0.148 0.000 0.149 0.000 0.000
> system.time(optim(x0, f, g, method="BFGS"))
[1] 0.008 0.000 0.008 0.000 0.000

and the function evaluation counts:

> bfgs(x0, f, g)$counts
function gradient 
  95   58 
> optim(x0, f, g, method="BFGS")$counts
function gradient 
 318  100 

So the overhead is clearly much bigger, but still too small to matter for
most (?) applications.

Cheers,
Andrew

PS, my computer is a "Intel(R) Pentium(R) 4 CPU 2.80GHz" with a
1024 KB cache, according to /proc/cpuinfo.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization in R

[Andrew Clausen]

Hi Duncan,

On Sat, Aug 04, 2007 at 09:25:39AM -0400, Duncan Murdoch wrote:
> This is interesting work; thanks for doing it.  Could I make a 
> suggestion?  Why not put together a package containing those test 
> optimization problems, and offer to include other interesting ones as 
> they arise?

Good idea.

> You could also include your wrapper for the gsl function 
> and your own improvements to optim.

I have submitted my gsl multimin() wrapper for inclusion into the Rgsl package,
which seems to be the "right" place for it.  (Its maintainer is currently
enjoying a vacation :)

It would be nice if all of these methods could be accessed with the existing
optim() interface, so that users could easily try different algorithms.
That is, users could specify method="BFGS" or "BFGS-R" or "BFGS-GSL", etc.
Is there a convenient mechanism for packages registering new methods?

One incompatibility with my BFGS implementation is that it returns the
*inverse* Hessian, which is a natural by-product of the BFGS algorithm.
Indeed, R's existing BFGS implementation also calculates the inverse Hessian,
and discards it!  (Disclaimer: as far as I know, there are no theorems that say
that BFGS's inverse Hessians are any good.  In practice, they seem to be.)

The inverse Hessian is more useful than the Hessian for statistics because it
gives the variance-covariance matrix for maximum likelihood estimators.  When
the Hessian is close to being singular (aka "computationally singular"),
solve() sometimes fails to invert it.

I think this means we should change the optim() interface.  For example, an
extra logical parameter, "inv.hessian" could control whether an inv.hessian
matrix is returned.

> On your first point:  I agree that a prototype implementation in R makes 
> sense, but I suspect there exist problems where the overhead would not 
> be negligible (e.g. ones where the user has written the objective 
> function in C for speed).  So I think you should keep in mind the 
> possibility of moving the core of your improvements to C once you are 
> happy with them.

Fair enough.

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Optimization in R

[Andrew Clausen]

Hi Manuel,

My multimin() wrapper will be merged into the Rgsl package.  I expect that the
wrapper doesn't do anything special (compared to the rest of Rgsl) to break the
compilation -- you're just having trouble with my very crude and temporary
Makefile.

Does Rgsl work for you?

If it does, you can probably just copy the relevant files into the right spot,
and build Rgsl the normal way.  (i.e. *.[ch] into src/, *.R into R/, and so
on...)

Cheers,
Andrew

On Sat, Aug 04, 2007 at 03:58:33PM -0400, Manuel Morales wrote:
> I tried installing the multimin function. To get it to compile, I needed
> to change the Makefile to reflect my path and by adding the flags fPIC
> in response to the error: "/usr/bin/ld: vector.o: relocation R_X86_64_32
> against `a local symbol' can not be used when making a shared object;
> recompile with -fPIC"
> 
> However, I get the following running test.R:
> 
> Error in dyn.load(x, as.logical(local), as.logical(now)) : 
> unable to load shared library 
> '/home/mmorales/Desktop/multimin/gsl.so':
>   /usr/lib64/libgsl.so.0: undefined symbol: cblas_ctrmv
> 
> I'm running R-2.5.1 compiled for x86_64 with a custom built ATLAS.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Sligthly OT Re: Makefile for embedding OpenBUGS in R package

[Andrew Clausen]

Hi Hin-Tak,

On Tue, Aug 07, 2007 at 01:10:36AM +0100, Hin-Tak Leung wrote:
> GPL-licensed code dlopen()'ing proprietary-licensed binary-only DLL/so
> is allowed

Do you have any evidence?  (eg: something written on www.fsf.org?)

As far as I know, the normal grounds for allowing GPL code to link with
proprietary code is the text

"However, as a special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary form) with
the major components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies the
executable."

Cheers,
Andrew

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R trunk (2.7) build fails with -fpic, needs -fPIC (PR#10372)

[Andrew Piskorski]

On Thu, Oct 25, 2007 at 12:19:13AM +0200, Peter Dalgaard wrote:

> Fedora 7 seems perfectly happy with -fpic,

Well, Ubuntu 6.06 x86-64 sure seems to hate it, but all I know about
-fpic vs. -fPIC is what I read here:

  
http://gcc.gnu.org/onlinedocs/gcc-4.0.3/gcc/Code-Gen-Options.html#index-fpic-1556
  
http://gcc.gnu.org/onlinedocs/gcc-4.2.2/gcc/Code-Gen-Options.html#index-fpic-1667

> Whic compiler/linker is this?

$ gcc --version | grep gcc
gcc (GCC) 4.0.3 (Ubuntu 4.0.3-1ubuntu5)
$ ld --version | grep ld
GNU ld version 2.16.91 20060118 Debian GNU/Linux

$ COLUMNS=140 dpkg -l binutils gcc-4.0
+++-==-=-==
ii  binutils   2.16.1cvs20060117-1ubuntu2.1  The GNU assembler, linker and 
binary utilities
ii  gcc-4.04.0.3-1ubuntu5The GNU C compiler

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] R trunk (2.7) build fails with -fpic, needs -fPIC (PR#10372)

[Andrew Piskorski]

On Thu, Oct 25, 2007 at 08:47:53AM +0200, Peter Dalgaard wrote:

> Also, make sure that your sources are clean (as in make distclean or
> fresh svn checkout) so that you are not mixing modules built with
> different compiler options.

I did a 'make clean distclean', 'svn up', removed my changes to
configure, and...  It worked!  It now builds successfully with -fpic
(not -fPIC).  So, problem solved, thank you!  And I apologize for
wasting your time with this.

-- 
Andrew Piskorski <[EMAIL PROTECTED]>
http://www.piskorski.com/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] A suggestion for an amendment to tapply

[Andrew Robinson]

Dear R-developers,

when tapply() is invoked on factors that have empty levels, it returns
NA.  This behaviour is in accord with the tapply documentation, and is
reasonable in many cases.  However, when FUN is sum, it would also
seem reasonable to return 0 instead of NA, because "the sum of an
empty set is zero, by definition."

I'd like to raise a discussion of the possibility of an amendment to
tapply.

The attached patch changes the function so that it checks if there are
any empty levels, and if there are, replaces the corresponding NA
values with the result of applying FUN to the empty set.  Eg in the
case of sum, it replaces the NA with 0, whereas with mean, it replaces
the NA with NA, and issues a warning.

This change has the following advantage: tapply and sum work better
together.  Arguably, tapply and any other function that has a non-NA
response to the empty set will also work better together.
Furthermore, tapply shows a warning if FUN would normally show a
warning upon being evaluated on an empty set.  That deviates from
current behaviour, which might be bad, but also provides information
that might be useful to the user, so that would be good.

The attached script provides the new function in full, and
demonstrates its application in some simple test cases.

Best wishes,

Andrew
-- 
Andrew Robinson  
Department of Mathematics and StatisticsTel: +61-3-8344-9763
University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599
http://www.ms.unimelb.edu.au/~andrewpr
http://blogs.mbs.edu/fishing-in-the-bay/ 
## The new function

my.tapply <- function (X, INDEX, FUN=NULL, ..., simplify=TRUE)
{
FUN <- if (!is.null(FUN)) match.fun(FUN)
if (!is.list(INDEX)) INDEX <- list(INDEX)
nI <- length(INDEX)
namelist <- vector("list", nI)
names(namelist) <- names(INDEX)
extent <- integer(nI)
nx <- length(X)
one <- as.integer(1)
group <- rep.int(one, nx)#- to contain the splitting vector
ngroup <- one
for (i in seq.int(INDEX)) {
index <- as.factor(INDEX[[i]])
if (length(index) != nx)
stop("arguments must have same length")
namelist[[i]] <- levels(index)#- all of them, yes !
extent[i] <- nlevels(index)
group <- group + ngroup * (as.integer(index) - one)
ngroup <- ngroup * nlevels(index)
}
if (is.null(FUN)) return(group)
ans <- lapply(split(X, group), FUN, ...)
index <- as.numeric(names(ans))
if (simplify && all(unlist(lapply(ans, length)) == 1)) {
ansmat <- array(dim=extent, dimnames=namelist)
ans <- unlist(ans, recursive = FALSE)
}
else  {
ansmat <- array(vector("list", prod(extent)),
dim=extent, dimnames=namelist)
}
## old : ansmat[as.numeric(names(ans))] <- ans
names(ans) <- NULL
ansmat[index] <- ans
if (sum(table(INDEX) < 1) > 0)
ansmat[table(INDEX) < 1] <- do.call(FUN, list(c(NULL), ...)) 
ansmat
}

## Check its utility

group <- factor(c(1,1,3,3), levels=c("1","2","3"))
x <- c(1,2,3,4)

## Ok with mean?

tapply(x, group, mean)
my.tapply(x, group, mean)

## Ok with sum?

tapply(x, group, sum)
my.tapply(x, group, sum)

## Check that other arguments are carried through

x <- c(NA,2,3,10)

tapply(x, group, sum, na.rm=TRUE)
tapply(x, group, mean, na.rm=TRUE)

my.tapply(x, group, sum, na.rm=TRUE)
my.tapply(x, group, mean, na.rm=TRUE)

## Check that listed groups work ok also

group.2 <- factor(c(1,2,3,3), levels=c("1","2","3"))

tapply(x, list(group, group.2), sum, na.rm=TRUE)
tapply(x, list(group, group.2), mean, na.rm=TRUE)

my.tapply(x, list(group, group.2), sum, na.rm=TRUE)
my.tapply(x, list(group, group.2), mean, na.rm=TRUE)

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] A suggestion for an amendment to tapply

[Andrew Robinson]

These are important concerns.  It seems to me that adding an argument
as suggested by Bill will allow the user to side-step the problem
identified by Brian.

Bill, under what kinds of circumstances would you anticipate a
significant time penalty?  I would be happy to check those out with
some simulations.

If the timing seems acceptable, I can write a patch for tapply.R and
tapply.Rd if anyone in the core is willing to consider them.  Please
contact me on or off list if so.

Best wishes to all,

Andrew




On Tue, Nov 06, 2007 at 07:23:56AM +, Prof Brian Ripley wrote:
> On Tue, 6 Nov 2007, [EMAIL PROTECTED] wrote:
> 
> >Unfortunately I think it would break too much existing code.  tapply()
> >is an old function and many people have gotten used to the way it works
> >now.
> 
> It is also not necessarily desirable: FUN(numeric(0)) might be an error.
> For example:
> 
> >Z <- data.frame(x=rnorm(10), f=rep(c("a", "b"), each=5))[1:5, ]
> >tapply(Z$x, Z$f, sd)
> 
> but sd(numeric(0)) is an error.  (Similar things involving var are 'in the 
> wild' and so would be broken.)
> 
> >This is not to suggest there could not be another argument added at the
> >end to indicate that you want the new behaviour, though.  e.g.
> >
> >tapply <- function (X, INDEX, FUN=NULL, ..., simplify=TRUE,
> >handle.empty.levels = FALSE)
> >
> >but this raises the question of what sort of time penalty the
> >modification might entail.  Probably not much for most situations, I
> >suppose.  (I know this argument name looks long, but you do need a
> >fairly specific argument name, or it will start to impinge on the ...
> >argument.)
> >
> >Just some thoughts.
> >
> >Bill Venables.
> >
> >Bill Venables
> >CSIRO Laboratories
> >PO Box 120, Cleveland, 4163
> >AUSTRALIA
> >Office Phone (email preferred): +61 7 3826 7251
> >Fax (if absolutely necessary):  +61 7 3826 7304
> >Mobile: +61 4 8819 4402
> >Home Phone: +61 7 3286 7700
> >mailto:[EMAIL PROTECTED]
> >http://www.cmis.csiro.au/bill.venables/
> >
> >-Original Message-
> >From: [EMAIL PROTECTED]
> >[mailto:[EMAIL PROTECTED] On Behalf Of Andrew Robinson
> >Sent: Tuesday, 6 November 2007 3:10 PM
> >To: R-Devel
> >Subject: [Rd] A suggestion for an amendment to tapply
> >
> >Dear R-developers,
> >
> >when tapply() is invoked on factors that have empty levels, it returns
> >NA.  This behaviour is in accord with the tapply documentation, and is
> >reasonable in many cases.  However, when FUN is sum, it would also
> >seem reasonable to return 0 instead of NA, because "the sum of an
> >empty set is zero, by definition."
> >
> >I'd like to raise a discussion of the possibility of an amendment to
> >tapply.
> >
> >The attached patch changes the function so that it checks if there are
> >any empty levels, and if there are, replaces the corresponding NA
> >values with the result of applying FUN to the empty set.  Eg in the
> >case of sum, it replaces the NA with 0, whereas with mean, it replaces
> >the NA with NA, and issues a warning.
> >
> >This change has the following advantage: tapply and sum work better
> >together.  Arguably, tapply and any other function that has a non-NA
> >response to the empty set will also work better together.
> >Furthermore, tapply shows a warning if FUN would normally show a
> >warning upon being evaluated on an empty set.  That deviates from
> >current behaviour, which might be bad, but also provides information
> >that might be useful to the user, so that would be good.
> >
> >The attached script provides the new function in full, and
> >demonstrates its application in some simple test cases.
> >
> >Best wishes,
> >
> >Andrew
> >
> 
> -- 
> Brian D. Ripley,  [EMAIL PROTECTED]
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel:  +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UKFax:  +44 1865 272595

-- 
Andrew Robinson  
Department of Mathematics and StatisticsTel: +61-3-8344-9763
University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599
http://www.ms.unimelb.edu.au/~andrewpr
http://blogs.mbs.edu/fishing-in-the-bay/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel