Re: [Rd] Segmentation fault early in compilation of revision 85514
On Mon, Nov 13, 2023 at 12:45 PM Avraham Adler wrote: > > On Mon, Nov 13, 2023 at 1:13 AM Dirk Eddelbuettel wrote: > > > > > > Avi, > > > > Might be toolchain-dependent, might be options-dependent--it built fine > > here. > > Easier for you to vary option two so maybe try that? > > > > Dirk > > Thank you, Dirk. > > I think it was more a PEBCAK issue. When I deleted the entire trunk > folder and started the process from scratch, it compiled properly as > per usual. Although this was revision 85520, so perhaps something > changed in the interim. > Are you using a separate build directory, or building in the src/ directory? The R Installation and Administration manual recommends using a separate build directory: You do not necessarily have to build R in the top-level source directory (say, TOP_SRCDIR). To build in BUILDDIR, run: cd BUILDDIR TOP_SRCDIR/configure make and so on, as described further below. This has the advantage of always keeping your source tree clean and is particularly recommended when you work with a version of R from Subversion. > Regardless, the issue resolved itself. > > Thanks, > > Avi > > ______ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
>
> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
> > Dear all,
> >
> > I have a generic function and a bunch of methods defined in a separate
> > environment. Here is a reduced example:
> >
> > env <- local({
> > # define the generic function and the method
> > myfun <- function(x) UseMethod("myfun")
> > myfun.myclass <- function(x) print("called myfun.myclass”)
> >
> > # register the method
> > .S3method("myfun", "myclass", myfun.myclass)
> >
> > environment()
> >})
> >
> > Since the method has been registered, I hoped that invocation like this
> > would work:
> >
> > env$myfun(structure(0, class = "myclass”))
> >
> > However, this results in a “no applicable method" error.
> >
> > It is my understanding that registerS3method (called by .S3method) will
> > install the method string in the .__S3MethodsTable__. table of the
> > environment where the generic function is defined, and this table is
> > subsequently used by usemethod() inside R, so I am puzzled that the
> > dispatch does not work. I checked and the .__S3MethodsTable__. of env is
> > indeed setup correctly. I also tried manually adding the method string to
> > the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no effect.
> >
> > In fact, the only way to make it work is to define either myfun or
> > myfun.myclas in the global environment, which is something I would like to
> > avoid.
> >
> > Thank you in advance for any pointers!
> >
>
> registerS3method has an additional parameter "envir" which I believe
> would end up set to env in your code. So this works:
>
> > eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
> [1] "called myfun.myclass"
>
> You could probably also call registerS3method with envir specified
> appropriately and get your original expression to work.
>
That doesn't seem to work on 4.1.0 for me. The code below worked for
me in Oct-2020, though I'm not sure what version of R I was using at
the time. I was slow to upgrade to 4.0, so it was probably the latest
3.x version.
env <- new.env()
local({
# define the generic function and the method
myfun <- function(x) { UseMethod("myfun", x) }
# register the method
registerS3method("myfun", "myclass",
function(x) { print("called myfun.myclass") },
envir = env)
}, envir = env)
attach(env)
myfun(structure(0, class = "myclass"))
> Duncan Murdoch
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Demo for linking native routines between R packages
Hi David, On Wed, Jan 6, 2021 at 4:00 AM Jitao David Zhang wrote: > > Dear Davis, Dirk, Jan, and r-devel subscribers, > > I hope you have had a good start in the new year. > > For your information: I have updated the GitHub repository to demonstrate > linking native routines between R packages ( > https://github.com/Accio/demo-linking-native), especially by listing the > prior art and the project by Davis to give the readers more background and > history of the topic. And I took the liberty to acknowledge your > contribution, criticism and help to improve this baby project. > > In any case, I wish there is a link from WRE to a small demo (not > necessarily mine!) where linking native routine is implemented in a > minimalistic example for teaching and demonstration purposes. That will > help people a lot in my opinion. > R-core will probably be reluctant to link to an external example they have very little control over. For example, what if the process changes and the external source doesn't get updated? That said, WRE does list two packages that register native routines from other packages: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Linking-to-native-routines-in-other-packages I'd be open to including a vignette in xts that discusses the process of linking to zoo. That would have the benefit of needing to be updated if the process changes. We can discuss off-list if you're interested in collaborating on it. > Best regards, > David > > -- > Jitao David Zhang > > http://jdzhang.me/ > m...@jdzhang.me > > Schützengasse 20 > 4125 Riehen, Switzerland > > [[alternative HTML version deleted]] > > ______ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R-Forge > GitHub?
On Sat, Jun 29, 2019 at 9:46 PM Joshua Ulrich wrote: > > On Sat, Jun 29, 2019 at 6:06 PM Spencer Graves > wrote: > > > > Hi, Henrik et al.: > > > > > >What's your favorite documentation on how to make two GitHub > > projects from one containing two packages? > > > > > >Currently, "github.com/sbgraves237/Ecdat" consists primarily of a > > directory "pkg" with subdirectories "Ecdat" and "Ecfun" containing the > > two packages. I need to know how to do the following: > > > > > > 1. Extract "github.com/sbgraves237/Ecdat/pkg/Ecfun" to > > create "github.com/sbgraves237/Ecfun". > > > > > > 2. Elevate "github.com/sbgraves237/Ecdat/pkg/Ecdat" to > > "github.com/sbgraves237/Ecdat", discarding the other files in the > > original "github.com/sbgraves237/Ecdat/". > > > > > >This sounds like it could be accomplished relatively easily by > > someone with sufficient understanding of "git" and GitHub. I could use > > suggestions on how to do this -- or at least on how to find > > documentation on how to do this. > > > This is straightforward if the two packages have always been in the > same directory structure. But it doesn't look like that's the case > for your repository, since Ecfun was added around r125. So Ecdat's > history is split and would need to be grafted together. I've done > this for other packages. So it's possible, but it took me some trial > and error. > > I'm giving that a go right now. I'll report back tomorrow morning, at > the latest. > I imported both packages into separate repositories: https://github.com/joshuaulrich/tmp-ecdat https://github.com/joshuaulrich/tmp-ecfun I changed your email address on your R-Forge commits to match your GitHub email address, so R-Forge commits would be associated with your GitHub account. I also omitted the "move" commit from Ecdat, and the "obsolete > GitHub" commits from both packages. I've attached a file with the commands I used, if anyone is interested. You can use my repos by cloning them to your local machine, adding your repos as new remotes, and pushing to them. You would need to run these commands (untested): ### clone my GitHub repo to your machine git clone g...@github.com:joshuaulrich/tmp-ecfun.git Ecdat cd Ecdat ### rename my GitHub repo remote from 'origin' to 'tmp' git remote rename origin tmp ### add your GitHub repo remote as 'origin' ### NOTE: this should be a new, clean repo. ###Rename your existing 'Ecdat' so you don't overwrite it git remote add origin https://github.com/sbgraves237/Ecdat ### push to your GitHub repo git push -u origin master Then you need to run similar commands for Ecfun. Best, Josh > > > >Thanks, > >Spencer > > > > > > On 2019-06-29 14:09, Henrik Bengtsson wrote: > > > On Sat, Jun 29, 2019 at 9:43 AM Spencer Graves > > > wrote: > > >> Hi, Ott et al.: > > >> > > >> > > >> What's the best way to get "Travis CI" to build and test the two > > >> packages, Ecdat and Ecfun, that have long been combined in the Ecdat > > >> project? > > >> > > >> > > >> Following Ott's advice and studying studying Wickham's "R > > >> Packages" (http://r-pkgs.had.co.nz/), I was able to configure RStudio so > > >> it would sync using git with "GitHub.com/sbgraves237/Ecdat". However, > > >> when I tried to configure "Travis CI", it said, "No DESCRIPTION file > > >> found, user must supply their own install and script steps". > > >> > > >> > > >> Earlier in this thread, I think someone suggested I make the > > >> Ecdat and Ecfun packages separate projects on GitHub (though I can't > > >> find that suggestion now). This would not be an issue if it were all > > >> local without version control. With RStudio managing my interface with > > >> GitHub, it now seems quite tricky. > > > I'm 99.999% confident that your life will be much much easier if you > > > keep one R package per repository. If you don't, you'll probably be > > > very lonely when it comes to tools etc. There are built-in 'git' > > > commands, but also git utility tools, for extracting a subset of > > > folders/files from git reposi
Re: [Rd] R-Forge > GitHub?
t;> migration from R-Forge to GitHub seems complete except for the automatic > >> tests provided via "Travis CI". > >> > >> > >> Spencer > >> > >> > >> On 2019-06-28 22:25, Ott Toomet wrote: > >>> Apparently your username/password are wrong. Can you clone/push from > >>> other repos? > >>> > >>> You do not need authorization when cloning a public repo, so even > >>> incorrect credentials may work (haven't tested this though). But for > >>> push you have to have that in order. > >>> > >>> I suggest you create ssh keys, upload those to GH, and use ssh > >>> authorization instead of https. > >>> > >>> Cheers, > >>> Ott > >>> > >>> On Fri, Jun 28, 2019 at 8:18 PM Spencer Graves > >>> mailto:spencer.gra...@prodsyse.com>> wrote: > >>> > >>> Thanks to Duncan, Henrik and Henrik, Brian, and Gábor: > >>> > >>> > >>> I created a local copy of the new GitHub version using the > >>> following: > >>> > >>> git clone > >>> https://sbgraves237:mypassw...@github.com/sbgraves237/Ecdat.git > >>> > >>> > >>> > >>> That worked in the sense that I got a local copy. However, > >>> after > >>> I rolled the version number and did "git commit" on the DESCRIPTION > >>> files, my "git push" command generated the following: > >>> > >>> > >>> remote: Invalid username or password. > >>> fatal: Authentication failed for > >>> 'https://sbgraves237:mypassw...@github.com/sbgraves237/Ecdat.git/' > >>> > >>> > >>> What am I missing? [Note: I used my actual GitHub > >>> password in > >>> place of "mypassword" here, and this "Authentication failed" message > >>> reported the GitHub password I used here.] > >>> > >>> > >>> Thanks, > >>> Spencer > >>> > >>> > >>> p.s. I'm doing this under macOS Mojave 10.14.5. Also, I added > >>> ".onAttach" functions to the R-Forge versions as Brian G. Peterson > >>> suggested. That seemed to work fine. > >>> > >>> > >>> On 2019-06-28 07:13, Duncan Murdoch wrote: > >>> > On 28/06/2019 6:26 a.m., Gábor Csárdi wrote: > >>> > > >>> >> Instead, you can do as Duncan suggested, and put a README in your > >>> >> R-Forge > >>> >> repository, that points to *your* GitHub repositor(y/ies). Then > >>> the > >>> >> https://github.com/rforge/ecdat read only mirror will pick this up > >>> >> and will > >>> >> point there as well. > >>> > > >>> > Just for the record: that was Henrik Singmann's suggestion, I just > >>> > agreed with it. > >>> > > >>> > Duncan Murdoch > >>> > > >>> > >>> > >>> [[alternative HTML version deleted]] > >>> > >>> __ > >>> R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list > >>> https://stat.ethz.ch/mailman/listinfo/r-devel > >>> > >> > >> [[alternative HTML version deleted]] > >> > >> __ > >> R-devel@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
I thought it would be good to summarize my thoughts, since I made a few hypotheses that turned out to be false. This isn't a bug in base R, in either rbind() or `[<-.Date`. To summarize the root cause: base::rbind.data.frame() calls `[<-` for each column of the data.frame, and there is no `[<-.IDate` method to ensure the replacement value is converted to integer. And, in fact, `[<-.Date` calls as.Date() and data.table::as.Date.IDate() calls as.numeric() on the IDate object. So the problem exists, and can be fixed, in data.table. Best, Josh On Mon, May 27, 2019 at 9:34 AM Joshua Ulrich wrote: > > Follow-up (inline) on my comment about a potential issue in `[<-.Date`. > > On Mon, May 27, 2019 at 9:31 AM Michael Chirico > wrote: > > > > Yes, thanks for following up on thread here. And thanks again for clearing > > things up, your email was a finger snap of clarity on the whole issue. > > > > I'll add that actually it was data.table's code at fault on the storage > > conversion -- note that if you use an arbitrary sub-class 'foo' with no > > methods defined, it'll stay integer. > > > > That's because [<- calls as.Date and then as.Date.IDate, and that method > > (ours) has as.numeric(); earlier I had recognized that if we commented that > > line, the issue was "fixed" but I still wasn't understanding the root cause. > > > > My last curiosity on this issue will be in my follow-up thread. > > > > Mike C > > > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich > > wrote: > >> > >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich > >> wrote: > >> > > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > >> > wrote: > >> > > > >> > > Have finally managed to come up with a fix after checking out > >> > > sys.calls() > >> > > from within the as.Date.IDate debugger, which shows something like: > >> > > > >> > > [[1]] rbind(DF, DF) > >> > > [[2]] rbind(deparse.level, ...) > >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > >> > > [[5]] as.Date(value) > >> > > [[6]] as.Date.IDate(value) > >> > > > >> > > I'm not sure why [<- is called, I guess the implementation is to > >> > > assign to > >> > > the output block by block? Anyway, we didn't have a [<- method. And > >> > > [<-.Date looks like: > >> > > > >> > > value <- unclass(as.Date(value)) # <- converts to double > >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class > >> > > > >> > > So we can fix our bug by defining a [<- class; the question that I > >> > > still > >> > > don't see answered in documentation or source code is, why/where is [<- > >> > > called, exactly? > >> > > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > >> > `[<-` call is this line: > >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > > >> > That's where the storage.mode changes from integer to double. > >> > > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > Browse[2]> > >> > debug: xij > >> > Browse[2]> storage.mode(xij) > >> > [1] "integer" > >> > Browse[2]> value[[jj]][ri] > >> > [1] "2019-05-26" > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "integer" > >> > Browse[2]> > >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "double" > >> > > >> To be clear, I don't think this is a bug in rbind() or > >> rbind.data.frame(). The confusion is that rbind.data.frame() calls > >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` > >> method. So the parent class method is dispatched, which converts the > >> storage mode to double. > >> > >> Someone may argue that this is an issue with `[<-.Date`, and that it > >> shouldn't convert the storage.mode from integer to double. > > I don't think this is an issue. The storage mode
Re: [Rd] survival changes
On Sat, Jun 1, 2019 at 5:22 AM Therneau, Terry M., Ph.D. via R-devel wrote: > > In the next version of the survival package I intend to make a non-upwardly > compatable > change to the survfit object. With over 600 dependent packages this is not > something to > take lightly, and I am currently undecided about the best way to go about it. > I'm looking > for advice. > I encountered several issues like this while making a change to xts. I encountered several buggy and inconsistent behaviors while moving attributes from the xts object to the xts object's index attribute. I don't want to "fix" some of these in the next version, because downstream packages may rely on the old behavior. Though it's specific to Python, "API Evolution the Right Way" has many good strategies. https://emptysqua.re/blog/api-evolution-the-right-way/ The "Deleting Features" and "Changing Behavior" sections are most relevant to your situation. > The change: 20+ years ago I had decided not to include the initial x=0,y=1 > data point in > the survfit object itself. It was not formally an estimand and the > plot/points/lines etc > routines could add this on themselves. That turns out to have been a > mistake, and has led > to a steady proliferation of extra bits as I realized that the time axis > doesn't always > start at 0, and later (with multi state) that y does not always start at 1 > (though the > states sum to 1), and later the the error doesn't always start at 0, and > another > realization with cumulative hazard, and ... > The new survfit method for multi-state coxph models was going to add yet > another special > case. Basically every component is turning into a duplicate of "row 1" vs > "all the > others". (And inconsistently named.) > > Three possible solutions > 1. Current working draft of survival_3.0.3: Add a 'version' element to the > survfit object > and a 'survfit2.3' function that converts old to new. All my downstream > functions (print, > plot,...) start with an "if (old) update to new" line. This has allowed me > to stage > updates to the functions that create survfit objects -- I expect it to happen > slowly. > There will also be a survfit3.2 function to go backwards. Both the forward > and backwards > functions leave objects alone if they are currently in the desired format. > This seems reasonable. It would also give you opportunity to warn users if functions that expect a new object receive an old object. That would help them convert any of their functions that rely on the old structure/behavior. You could also add a global option to enable warnings with the default set to FALSE for the first release. Let users know they can set that option to TRUE to identify places where they may need to modify their use cases. > 2. Make a new class "survfit3" and the necessary 'as' functions. The package > would contain > plot.survfit and plot.survfit3 methods, the former a two line "convert and > call the > second" function. > Rather than an entirely new class, I wonder if you could instead make the new class a subclass. I'm not sure whether this has advantages over adding an element, but it's another possibility. > 3. Something I haven't thought of. > I don't have any other ideas, but I would be happy to discuss offline if that would be helpful. > Number 2 has a cleanness about it, but there is a long term nuisance about it > wrt > documentation. Users, not unreasonably, expect the survfit function to > produce a survfit > object, and that is what they look for in the help pages. > > I plan to have 3.0-x on github before userR so that users can begin to play > with it (and > to get feeback before pushing to CRAN), so need to make a decision. > > Terry T. > > > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2019 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
Follow-up (inline) on my comment about a potential issue in `[<-.Date`. On Mon, May 27, 2019 at 9:31 AM Michael Chirico wrote: > > Yes, thanks for following up on thread here. And thanks again for clearing > things up, your email was a finger snap of clarity on the whole issue. > > I'll add that actually it was data.table's code at fault on the storage > conversion -- note that if you use an arbitrary sub-class 'foo' with no > methods defined, it'll stay integer. > > That's because [<- calls as.Date and then as.Date.IDate, and that method > (ours) has as.numeric(); earlier I had recognized that if we commented that > line, the issue was "fixed" but I still wasn't understanding the root cause. > > My last curiosity on this issue will be in my follow-up thread. > > Mike C > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich wrote: >> >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich >> wrote: >> > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico >> > wrote: >> > > >> > > Have finally managed to come up with a fix after checking out sys.calls() >> > > from within the as.Date.IDate debugger, which shows something like: >> > > >> > > [[1]] rbind(DF, DF) >> > > [[2]] rbind(deparse.level, ...) >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) >> > > [[5]] as.Date(value) >> > > [[6]] as.Date.IDate(value) >> > > >> > > I'm not sure why [<- is called, I guess the implementation is to assign >> > > to >> > > the output block by block? Anyway, we didn't have a [<- method. And >> > > [<-.Date looks like: >> > > >> > > value <- unclass(as.Date(value)) # <- converts to double >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class >> > > >> > > So we can fix our bug by defining a [<- class; the question that I still >> > > don't see answered in documentation or source code is, why/where is [<- >> > > called, exactly? >> > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The >> > `[<-` call is this line: >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij >> > >> > That's where the storage.mode changes from integer to double. >> > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij >> > Browse[2]> >> > debug: xij >> > Browse[2]> storage.mode(xij) >> > [1] "integer" >> > Browse[2]> value[[jj]][ri] >> > [1] "2019-05-26" >> > Browse[2]> storage.mode(value[[jj]][ri]) >> > [1] "integer" >> > Browse[2]> >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm >> > Browse[2]> storage.mode(value[[jj]][ri]) >> > [1] "double" >> > >> To be clear, I don't think this is a bug in rbind() or >> rbind.data.frame(). The confusion is that rbind.data.frame() calls >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` >> method. So the parent class method is dispatched, which converts the >> storage mode to double. >> >> Someone may argue that this is an issue with `[<-.Date`, and that it >> shouldn't convert the storage.mode from integer to double. I don't think this is an issue. The storage mode isn't converted if the replacement is the same storage mode. For example: R> x <- .Date(1:5) R> storage.mode(x) [1] "integer" R> x[1L] <- .Date(0L) R> storage.mode(x) [1] "integer" R> x[1L] <- .Date(0) R> storage.mode(x) [1] "double" >> > >> > > Mike C >> > > >> > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico >> > > >> > > wrote: >> > > >> > > > Debugging this issue: >> > > > >> > > > https://github.com/Rdatatable/data.table/issues/2008 >> > > > >> > > > We have custom class 'IDate' which inherits from 'Date' (it just forces >> > > > integer storage for efficiency, hence, I). >> > > > >> > > > The concatenation done by rbind, however, breaks this and returns a >> > > > double: >> > > > >> > > > library(data.table) >>
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich wrote: > > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > wrote: > > > > Have finally managed to come up with a fix after checking out sys.calls() > > from within the as.Date.IDate debugger, which shows something like: > > > > [[1]] rbind(DF, DF) > > [[2]] rbind(deparse.level, ...) > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > > [[5]] as.Date(value) > > [[6]] as.Date.IDate(value) > > > > I'm not sure why [<- is called, I guess the implementation is to assign to > > the output block by block? Anyway, we didn't have a [<- method. And > > [<-.Date looks like: > > > > value <- unclass(as.Date(value)) # <- converts to double > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class > > > > So we can fix our bug by defining a [<- class; the question that I still > > don't see answered in documentation or source code is, why/where is [<- > > called, exactly? > > > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > `[<-` call is this line: > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > > That's where the storage.mode changes from integer to double. > > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > Browse[2]> > debug: xij > Browse[2]> storage.mode(xij) > [1] "integer" > Browse[2]> value[[jj]][ri] > [1] "2019-05-26" > Browse[2]> storage.mode(value[[jj]][ri]) > [1] "integer" > Browse[2]> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > Browse[2]> storage.mode(value[[jj]][ri]) > [1] "double" > To be clear, I don't think this is a bug in rbind() or rbind.data.frame(). The confusion is that rbind.data.frame() calls `[<-` for each column of the data.frame, and there is no `[<-.IDate` method. So the parent class method is dispatched, which converts the storage mode to double. Someone may argue that this is an issue with `[<-.Date`, and that it shouldn't convert the storage.mode from integer to double. > > > Mike C > > > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico > > wrote: > > > > > Debugging this issue: > > > > > > https://github.com/Rdatatable/data.table/issues/2008 > > > > > > We have custom class 'IDate' which inherits from 'Date' (it just forces > > > integer storage for efficiency, hence, I). > > > > > > The concatenation done by rbind, however, breaks this and returns a > > > double: > > > > > > library(data.table) > > > DF = data.frame(date = as.IDate(Sys.Date())) > > > storage.mode(rbind(DF, DF)$date) > > > # [1] "double" > > > > > > This is specific to base::rbind (data.table's rbind returns an integer as > > > expected); in ?rbind we see: > > > > > > The method dispatching is not done via UseMethod(), but by C-internal > > > dispatching. Therefore there is no need for, e.g., rbind.default. > > > The dispatch algorithm is described in the source file > > > (‘.../src/main/bind.c’) as > > > 1. For each argument we get the list of possible class memberships from > > > the class attribute. > > > 2. *We inspect each class in turn to see if there is an applicable > > > method.* > > > 3. If we find an applicable method we make sure that it is identical to > > > any method determined for prior arguments. If it is identical, we proceed, > > > otherwise we immediately drop through to the default code. > > > > > > It's not clear what #2 means -- an applicable method *for what*? Glancing > > > at the source code would suggest it's looking for rbind.IDate: > > > > > > https://github.com/wch/r-source/blob/trunk/src/main/bind.c#L1051-L1063 > > > > > > const char *generic = ((PRIMVAL(op) == 1) ? "cbind" : "rbind"); // should > > > be rbind here > > > const char *s = translateChar(STRING_ELT(classlist, i)); // iterating over > > > the classes, should get to IDate first > > > sprintf(buf, "%s.%s", generic, s); // should be rbind.IDate > > > > > > but adding this method (or even exporting it) is no help [ simply defining > > > rbind.IDate = function(...) as.IDate(NextMethod()) ] > > > > > > Lastly, it appears that as.Date.IDate is called, which is causing the typ
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
On Sun, May 26, 2019 at 4:06 AM Michael Chirico
wrote:
>
> Have finally managed to come up with a fix after checking out sys.calls()
> from within the as.Date.IDate debugger, which shows something like:
>
> [[1]] rbind(DF, DF)
> [[2]] rbind(deparse.level, ...)
> [[3]] `[<-`(`*tmp*`, ri, value = 18042L)
> [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L)
> [[5]] as.Date(value)
> [[6]] as.Date.IDate(value)
>
> I'm not sure why [<- is called, I guess the implementation is to assign to
> the output block by block? Anyway, we didn't have a [<- method. And
> [<-.Date looks like:
>
> value <- unclass(as.Date(value)) # <- converts to double
> .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class
>
> So we can fix our bug by defining a [<- class; the question that I still
> don't see answered in documentation or source code is, why/where is [<-
> called, exactly?
>
Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The
`[<-` call is this line:
value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij
That's where the storage.mode changes from integer to double.
debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij
Browse[2]>
debug: xij
Browse[2]> storage.mode(xij)
[1] "integer"
Browse[2]> value[[jj]][ri]
[1] "2019-05-26"
Browse[2]> storage.mode(value[[jj]][ri])
[1] "integer"
Browse[2]>
debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm
Browse[2]> storage.mode(value[[jj]][ri])
[1] "double"
> Mike C
>
> On Sun, May 26, 2019 at 1:16 PM Michael Chirico
> wrote:
>
> > Debugging this issue:
> >
> > https://github.com/Rdatatable/data.table/issues/2008
> >
> > We have custom class 'IDate' which inherits from 'Date' (it just forces
> > integer storage for efficiency, hence, I).
> >
> > The concatenation done by rbind, however, breaks this and returns a double:
> >
> > library(data.table)
> > DF = data.frame(date = as.IDate(Sys.Date()))
> > storage.mode(rbind(DF, DF)$date)
> > # [1] "double"
> >
> > This is specific to base::rbind (data.table's rbind returns an integer as
> > expected); in ?rbind we see:
> >
> > The method dispatching is not done via UseMethod(), but by C-internal
> > dispatching. Therefore there is no need for, e.g., rbind.default.
> > The dispatch algorithm is described in the source file
> > (‘.../src/main/bind.c’) as
> > 1. For each argument we get the list of possible class memberships from
> > the class attribute.
> > 2. *We inspect each class in turn to see if there is an applicable
> > method.*
> > 3. If we find an applicable method we make sure that it is identical to
> > any method determined for prior arguments. If it is identical, we proceed,
> > otherwise we immediately drop through to the default code.
> >
> > It's not clear what #2 means -- an applicable method *for what*? Glancing
> > at the source code would suggest it's looking for rbind.IDate:
> >
> > https://github.com/wch/r-source/blob/trunk/src/main/bind.c#L1051-L1063
> >
> > const char *generic = ((PRIMVAL(op) == 1) ? "cbind" : "rbind"); // should
> > be rbind here
> > const char *s = translateChar(STRING_ELT(classlist, i)); // iterating over
> > the classes, should get to IDate first
> > sprintf(buf, "%s.%s", generic, s); // should be rbind.IDate
> >
> > but adding this method (or even exporting it) is no help [ simply defining
> > rbind.IDate = function(...) as.IDate(NextMethod()) ]
> >
> > Lastly, it appears that as.Date.IDate is called, which is causing the type
> > conversion:
> >
> > debug(data.table:::as.Date.IDate)
> > rbind(DF, DF) # launches debugger
> > x
> > # [1] "2019-05-26" <-- singleton, so apparently applied to DF$date, not
> > c(DF$date, DF$date)
> > undebug(data.table:::as.Date.IDate)
> >
> > I can't really wrap my head around why as.Date is being called here, and
> > even allowing that, why the end result is still the original class [
> > class(rbind(DF, DF)$date) == c('IDate', 'Date') ]
> >
> > So, I'm beginning to think this might be a bug. Am I missing something?
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2019 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] anyNA() performance on vectors of POSIXct
On Wed, May 1, 2019 at 7:45 AM Harvey Smith wrote:
>
> Inside of the anyNA() function, it will use the legacy any(is.na()) code if
> x is an OBJECT(). If x is a vector of POSIXct, it will be an OBJECT(), but
> it is also TYPEOF(x) == REALSXP. Therefore, it will skip the faster
> ITERATE_BY_REGION, which is typically 5x faster in my testing.
>
> Is the OBJECT() condition really necessary, or could it be moved after the
> switch() for the individual TYPEOF(x) ITERATE_BY_REGION calls?
>
> # script to demonstrate performance difference if x is an OBJECT or not by
> using unclass()
> x.posixct = Sys.time() + 1:1e6
> microbenchmark::microbenchmark(
> any(is.na( x.posixct )),
> anyNA( x.posixct ),
> anyNA( unclass(x.posixct) ),
> unit='ms')
>
>
>
> static Rboolean anyNA(SEXP call, SEXP op, SEXP args, SEXP env)
> {
> SEXP x = CAR(args);
> SEXPTYPE xT = TYPEOF(x);
> Rboolean isList = (xT == VECSXP || xT == LISTSXP), recursive = FALSE;
>
> if (isList && length(args) > 1) recursive = asLogical(CADR(args));
> *if (OBJECT(x) || (isList && !recursive)) {*
> SEXP e0 = PROTECT(lang2(install("is.na"), x));
> SEXP e = PROTECT(lang2(install("any"), e0));
> SEXP res = PROTECT(eval(e, env));
> int ans = asLogical(res);
> UNPROTECT(3);
> return ans == 1; // so NA answer is false.
> }
>
> R_xlen_t i, n = xlength(x);
> switch (xT) {
> case REALSXP:
> {
> if(REAL_NO_NA(x))
> return FALSE;
> ITERATE_BY_REGION(x, xD, i, nbatch, double, REAL, {
> for (int k = 0; k < nbatch; k++)
> if (ISNAN(xD[k]))
> return TRUE;
> });
> break;
> }
>
I'm interested in this as well, because it causes performance
degradation in xts subsetting:
https://github.com/joshuaulrich/xts/issues/296
Would it be possible to special-case POSIXct, and perhaps other types
defined in base+recommended packages?
Best,
Josh
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2019 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Proposal: Internal as.Date.POSIXct()
I would like to propose a .Internal() POSIXct2Date() function, similar to do_POSIXlt2D() in src/main/datetime.c. I created a working prototype, which is now included in Dirk Eddelbuettel's RApiDatetime package: https://github.com/eddelbuettel/rapidatetime This prototype is a combination of asPOSIXlt and POSIXlt2D from R Core. The functions are essentially concatenated, with the extraneous parts removed. Directly converting from POSIXct to Date uses less memory and is roughly twice as fast. You can find the relevant bits in the commit on GitHub: https://github.com/eddelbuettel/rapidatetime/commit/6bc9320451fb1e62d53b5012d33acf538c0f50c6#diff-7df5c1a674e930a4d93ba45f3ce0aaaf Would someone on R Core be interested in adding this function? If so, I can create a patch that includes all the necessary code additions needed to make it a .Internal() function. Thanks for your consideration! Best, Josh -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Bug in POSIXct string representation?
Hi Andreas,
On Thu, Aug 9, 2018 at 2:26 AM, Festl, Andreas wrote:
> Dear all,
>
> I just have identified the following issue which I believe could be a bug in
> R:
>
> Let me illustrate:
>
> First, enable the display of fractional seconds and check that it works:
>> options(digits.secs = 6, digits = 6)
>> as.character(as.POSIXct("2018-08-31 14:15:16.123456"))
> [1] "2018-08-31 14:15:16.123456"
>
> Now create a sequence of POSIXct with stepwidth 0.1sec:
>> test <- as.POSIXct("2018-08-31 14:15:16.00")
>> test_seq <- seq(test, test + 1, by = 1/10)
>
> Calling format with the millisecond conversion specification gives the
> intended result (even though there is a small representation error):
>> format(test_seq, "%F %T.%OS")
> [1] "2018-08-31 14:15:16.16.00" "2018-08-31 14:15:16.16.09"
> "2018-08-31 14:15:16.16.20" "2018-08-31 14:15:16.16.29"
> [5] "2018-08-31 14:15:16.16.40" "2018-08-31 14:15:16.16.50"
> "2018-08-31 14:15:16.16.59" "2018-08-31 14:15:16.16.70"
> [9] "2018-08-31 14:15:16.16.79" "2018-08-31 14:15:16.16.90"
> "2018-08-31 14:15:17.17.00"
>
> However, if I use as.character, the milliseconds seemingly just get cut-off
> after one digit, resulting in incorrect representations:
>> as.character(test_seq)
> [1] "2018-08-31 14:15:16.0" "2018-08-31 14:15:16.0" "2018-08-31 14:15:16.2"
> "2018-08-31 14:15:16.2" "2018-08-31 14:15:16.4" "2018-08-31 14:15:16.5"
> [7] "2018-08-31 14:15:16.5" "2018-08-31 14:15:16.7" "2018-08-31 14:15:16.7"
> "2018-08-31 14:15:16.9" "2018-08-31 14:15:17.0"
>
> It seems to me, that R correctly decides that there is only one significant
> digit after the decimal point, but then incorrectly (due to representation
> error) just cuts off after the first digit.
>
This is known behavior with how POSIXt objects are printed, and has
been discussed before on R-help:
https://stat.ethz.ch/pipermail/r-help/2015-June/429600.html
Basically, the behavior is a combination of truncating fractional
seconds rather than rounding combined with the floating point
representation error you noticed. And truncation is the behavior for
printing whole seconds:
format(as.POSIXct("2018-08-31 14:15:16.9")) # 16s, not 17s
[1] "2018-08-31 14:15:16"
So it would not be consistent to round fractional seconds, unless you
kept track of the rounding error relative to the desired resolution.
There are more details in the R-help thread and the StackOverflow Q&A
it references.
Best,
Josh
> BR,
> Andreas
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Memory leakage from large lists
This looks like a case of FAQ 7.42:
https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-is-R-apparently-not-releasing-memory_003f
On Mon, Jul 16, 2018 at 2:32 PM, Daniel Raduta wrote:
> Hello,
>
> I am experiencing a very noticeable memory leak when using large lists of
> large data. The code below creates a list of matrices, yet the memory does
> not free up after removing that item and performing a garbage collection.
> Is there a something I can do to prevent this memory leak? Any help would
> be greatly appreciated. By the way, if you execute the code, please run it
> in a new R session.
>
> # Start of code
>
>
> # Function that returns memory being used
> MemoryUsed <- function(){
> pid <- Sys.getpid()
> system(paste0("top -n 1 -p ", pid, " -b"), intern = TRUE)[c(7,8)]
> }
>
> # Initial memory (VIRT memory equals about 400,000 bytes on my machine)
> MemoryUsed()
>
> # Create a large list of large data, remove it, and perform garbarge
> collection
> ncols <- 100
> nrows <- 1
> mat <- matrix(seq(nrows * ncols), nrow = nrows, ncol = ncols)
> ls <- lapply(1:1000, function(x) as.data.frame(mat))
> rm(list = setdiff(ls(), 'MemoryUsed'))
> invisible(gc())
>
> # Final memory (now, VIRT memory equals about 4,600,000 bytes on my machine)
> MemoryUsed()
>
> # End of code
> ==
>
> My session info is:
>
> R version 3.4.4 (2018-03-15)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 16.04.3 LTS
>
> Matrix products: default
> BLAS: /usr/lib/libblas/libblas.so.3.6.0
> LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
> LC_PAPER=en_US.UTF-8 LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.4.4 tools_3.4.4yaml_2.1.19
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] install.packages doesn't produce warnings unless qualified with utils::
On Sat, Mar 3, 2018 at 1:33 PM, David Hugh-Jones
wrote:
> Hi all,
>
> Assuming this is an R core issue:
>
> tryCatch(install.packages("clipr", repos = "bullshit"), warning = function
> (w) cat("got a warning"))
> Warning in install.packages :
> unable to access index for repository bullshit/src/contrib:
> cannot open URL 'bullshit/src/contrib/PACKAGES'
> Warning in install.packages :
> package ‘clipr’ is not available (for R version 3.4.3)
> Warning in install.packages :
> unable to access index for repository
> bullshit/bin/macosx/el-capitan/contrib/3.4:
> cannot open URL 'bullshit/bin/macosx/el-capitan/contrib/3.4/PACKAGES'
>
> In other words, lots of warnings, but none are caught.
>
Using R-3.4.3 on 64-bit Ubuntu throws and catches the warning.
R> tryCatch(install.packages("clipr", repos="w"), warning=function(w)
cat("got a warning!\n"))
Installing package into '/home/josh/R/library'
(as 'lib' is unspecified)
got a warning!
> It works if you use the fully qualified version in utils:
>
> tryCatch(utils::install.packages("clipr", repos = "bullshit"), warning =
> function (w) cat("got a warning"))
> got a warning
>
My guess is that something (a package, console, etc) is masking
utils::install.packages().
> Any ideas?
> Cheers,
> David
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] SIGSEGV during startup
Hi,
This problem started as of r73472 ("Merged in the rest of the basic
ALTREP framework."); I tested r73471 and it did not exist.
I'm building R devel on Ubuntu 16.04.3 LTS (64-bit), with the following flags:
CC="gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer"
CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic -mtune=native"
and using
configure --with-valgrind-instrumentation=2
Running 'make' builds the R binary, but R fails to start. So 'make'
ultimately fails when trying to install sysdata.rda during the tools
package build. Here is the traceback:
make[5]: Leaving directory '/home/josh/R/R-build/src/library/tools/src'
make[4]: Leaving directory '/home/josh/R/R-build/src/library/tools'
make[4]: Entering directory '/home/josh/R/R-build/src/library/tools'
installing 'sysdata.rda'
ASAN:SIGSEGV
=
==3543==ERROR: AddressSanitizer: SEGV on unknown address
0x (pc 0x0043b2f2 bp 0x7ffeb5fdc670 sp 0x7ffeb5fdc660
T0)
#0 0x43b2f1 in ALTVEC_DATAPTR ../../../R-svn/src/main/altrep.c:305
#1 0x6b1068 in DATAPTR ../../../R-svn/src/include/Rinlinedfuns.h:105
#2 0x6b1068 in GetNewPage ../../../R-svn/src/main/memory.c:908
#3 0x6b5284 in Rf_allocVector3 ../../../R-svn/src/main/memory.c:2514
#4 0x654a49 in Rf_allocVector ../../../R-svn/src/include/Rinlinedfuns.h:514
#5 0x654a49 in Rf_mkTrue ../../../R-svn/src/main/gram.c:4164
#6 0x6b616c in Rf_InitMemory ../../../R-svn/src/main/memory.c:2137
#7 0x69186a in setup_Rmainloop ../../../R-svn/src/main/main.c:842
#8 0x694488 in Rf_mainloop ../../../R-svn/src/main/main.c:1088
#9 0x41e398 in main ../../../R-svn/src/main/Rmain.c:29
#10 0x7f9d71f4182f in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#11 0x41fbe8 in _start (/home/josh/R/R-build/bin/exec/R+0x41fbe8)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ../../../R-svn/src/main/altrep.c:305
ALTVEC_DATAPTR
==3543==ABORTING
../../../../R-svn/share/make/basepkg.mk:150: recipe for target 'sysdata' failed
make[4]: *** [sysdata] Error 1
make[4]: Leaving directory '/home/josh/R/R-build/src/library/tools'
Makefile:30: recipe for target 'all' failed
make[3]: *** [all] Error 2
make[3]: Leaving directory '/home/josh/R/R-build/src/library/tools'
Makefile:36: recipe for target 'R' failed
make[2]: *** [R] Error 1
make[2]: Leaving directory '/home/josh/R/R-build/src/library'
Makefile:28: recipe for target 'R' failed
make[1]: *** [R] Error 1
make[1]: Leaving directory '/home/josh/R/R-build/src'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1
The context around memory.c:908 is below, and suggests the issue
manifests with the combination of the ALTREP framework and valgrind
instrumentation level >1.
#if VALGRIND_LEVEL > 1
if (NodeClassSize[node_class] > 0)
VALGRIND_MAKE_MEM_NOACCESS(DATAPTR(s),
NodeClassSize[node_class]*sizeof(VECREC));
#endif
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why is as.function() slower than eval(call("function"())?
On Thu, Aug 3, 2017 at 11:32 PM, Gregory Werbin
wrote:
> (Apologies if this is better suited for R-help.)
>
> On my system (macOS Sierra, late 2014 MacBook Pro; R 3.4.1, Homebrew build),
> I found that it is faster to construct a function using eval(call("function",
> ...)) than using as.function(list(...)). Example:
>
> make_fn_1 <- function(a, b) eval(call("function", a, b), env =
> parent.frame())
> make_fn_2 <- function(a, b) as.function(c(a, list(b)), env =
> parent.frame())
>
> a <- as.pairlist(alist(x = , y = ))
> b <- quote(x + y)
>
> library("microbenchmark")
> microbenchmark(make_fn_1(a, b), make_fn_2(a, b))
>
> # Unit: microseconds
> # expr min lqmean median uqmax neval cld
> # make_fn_1(a, b) 1.671 1.8855 2.13297 2.039 2.1950 9.852 100 a
> # make_fn_2(a, b) 3.541 3.7230 4.13400 3.906 4.1055 23.153 100 b
>
> At first I thought the gap was due to the overhead of calling c(a, list(b)).
> But this turns out not to be the case:
>
> make_fn_weird <- function(a, b) as.function(c(a, b), env = parent.frame())
> b_wrapped <- list(b)
>
> make_fn_weirder <- function(a_b) as.function(a_b, env = parent.frame())
> a_b <- c(a, b_wrapped)
>
> microbenchmark(make_fn_1(a, b), make_fn_2(a, b),
>make_fn_weird(a, b_wrapped), make_fn_weirder(a_b))
>
> # Unit: microseconds
> # expr min lqmean median uqmax
> neval cld
> # make_fn_1(a, b) 1.718 1.8990 2.12119 1.9860 2.1605 8.057
> 100 a
> # make_fn_2(a, b) 3.393 3.5865 4.03029 3.6655 3.9615 27.499
> 100 c
> # make_fn_weird(a, b_wrapped) 3.354 3.5005 3.77190 3.6405 3.9425 6.839
> 100 c
> # make_fn_weirder(a_b) 2.488 2.6290 2.83352 2.7215 2.8800 7.007
> 100 b
>
> One IRC user pointed out that as.function() takes its own path through the
> code, namely do_asfunction() (in src/main/coerce.c). What is it about this
> code path that's 50% slower than whatever happens during
> eval(call("function", a, b))?
>
> Obviously this is a trivial micro-optimization and it doesn't matter to 99%
> of users. Mostly asking out of curiosity, but also wondering if there's a
> more general lesson to be learned here.
>
Agreed that this is minor (~2us), but the majority of the difference
seems to be from S3 method dispatch. as.function() is generic and has
to dispatch to as.function.default(). The times are very similar if
you call the method directly.
R> make_fn_3 <- function(a, b) as.function.default(c(a, list(b)), env
= parent.frame())
R> microbenchmark(make_fn_1(a, b), make_fn_2(a, b), make_fn_3(a, b))
Unit: microseconds
expr min lq mean medianuq max neval
make_fn_1(a, b) 1.615 1.7595 12.78339 1.9115 2.145 1077.657 100
make_fn_2(a, b) 3.077 3.3390 19.89423 3.5215 3.862 1589.505 100
make_fn_3(a, b) 1.629 1.7975 15.40389 1.9505 2.227 1335.306 100
Now the difference is <100ns, which is much harder to investigate.
> Thanks!
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] bug in rbind?
I'm not sure whether or not this is a bug, but I did isolate the line where the error is thrown: src/library/base/R/dataframe.R:1395. https://github.com/wch/r-source/blob/01374c3c367fa12f555fd354f735a6e16e5bd98e/src/library/base/R/dataframe.R#L1395 The error is thrown because the line attempts to set a subset of the rownames to NULL, which fails. R> options(error = recover) R> rbind(dfm.names, dfm) Error in rownames(value[[jj]])[ri] <- rownames(xij) : replacement has length zero Enter a frame number, or 0 to exit 1: rbind(dfm.names, dfm) 2: rbind(deparse.level, ...) Selection: 2 Called from: top level Browse[1]> rownames(value[[jj]]) [1] "a" "b" "c" NA NA NA Browse[1]> rownames(xij) NULL Browse[1]> ri [1] 4 5 6 Browse[1]> rownames(value[[jj]])[ri] [1] NA NA NA On Mon, Jan 16, 2017 at 7:50 PM, Krzysztof Banas wrote: > I suspect there may be a bug in base::rbind.data.frame > > Below there is minimal example of the problem: > > m <- matrix (1:12, 3) > dfm <- data.frame (c = 1 : 3, m = I (m)) > str (dfm) > > m.names <- m > rownames (m.names) <- letters [1:3] > dfm.names <- data.frame (c = 1 : 3, m = I (m.names)) > str (dfm.names) > > rbind (m, m.names) > rbind (m.names, m) > rbind (dfm, dfm.names) > > #not working > rbind (dfm.names, dfm) > > Error in rbind(deparse.level, ...) : replacement has length zero > > rbind (dfm, dfm.names)$m > > > [,1] [,2] [,3] [,4] > > 147 10 > > 258 11 > > 369 12 > > a 147 10 > > b 258 11 > > c 369 12 > > > > > > Important: This email is confidential and may be privileged. If you are not > the intended recipient, please delete it and notify us immediately; you > should not copy or use it for any purpose, nor disclose its contents to any > other person. Thank you. > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] wish list: generalized apply
On Thu, Dec 8, 2016 at 3:59 PM, David Winsemius wrote:
>
>> On Dec 8, 2016, at 12:09 PM, John P. Nolan wrote:
>>
>> Dear All,
>>
>> I regularly want to "apply" some function to an array in a way that the
>> arguments to the user function depend on the index on which the apply is
>> working. A simple example is:
>>
>> A <- array( runif(160), dim=c(5,4,8) )
>> x <- matrix( runif(32), nrow=4, ncol=8 )
>> b <- runif(8)
>> f1 <- function( A, x, b ) { sum( A %*% x ) + b }
>> result <- rep(0.0,8)
>> for (i in 1:8) {
>> result[i] <- f1( A[,,i], x[,i] , b[i] )
>> }
>>
>> This works, but is slow. I'd like to be able to do something like:
>>generalized.apply( A, MARGIN=3, FUN=f1, list(x=x,MARGIN=2),
>> list(b=b,MARGIN=1) ), where the lists tell generalized.apply to pass x[,i]
>> and b[i] to FUN in addition to A[,,i].
>>
>> Does such a generalized.apply already exist somewhere? While I can write a
>> C function to do a particular case, it would be nice if there was a fast,
>> general way to do this.
>
> I would have thought that this would achieve the same result:
>
> result <- sapply( seq_along(b) , function(i) { f1( A[,,i], x[,i] , b[i] )} )
>
> Or:
>
> result <- sapply( seq.int( dim(A)[3] ) , function(i) { f1( A[,,i], x[,i] ,
> b[i] )} )
>
> (I doubt it will be any faster, but if 'i' is large, parallelism might help.
> The inner function appears to be fairly efficient.)
You're right, it's slower. Despite how often it's repeated that
"loops in R are slow", they're not *that* slow. They're often faster
than the *apply functions, especially if they have been "compiled" by
compiler::cmpfun().
You really need to know *why* code is slow before trying to make it
faster. I profiled an example that would have a loop with 1e6
iterations and 80%+ of the time was still spent inside f1().
set.seed(21)
nc <- 1e6
nr <- 10
A <- array( runif(5*nr*nc), dim=c(5,nr,nc) )
x <- matrix( runif(nr*nc), nrow=nr, ncol=nc )
b <- runif(nc)
f1 <- compiler::cmpfun(function( A, x, b ) { sum( A %*% x ) + b })
f2 <- compiler::cmpfun({
function(A, x, b, FUN) {
result <- numeric(length(b))
for (i in seq_along(b)) {
result[i] <- FUN( A[,,i], x[,i] , b[i] )
}
return(result)
}
})
Rprof(interval=0.01)
result <- f2(A,x,b,f1)
Rprof(NULL)
summaryRprof()
$by.self
self.time self.pct total.time total.pct
"FUN" 4.2984.28 4.76 93.52
"%*%" 0.47 9.23 0.47 9.23
"f2" 0.33 6.48 5.09100.00
$by.total
total.time total.pct self.time self.pct
"f2"5.09100.00 0.33 6.48
"FUN" 4.76 93.52 4.2984.28
"%*%" 0.47 9.23 0.47 9.23
$sample.interval
[1] 0.01
$sampling.time
[1] 5.09
In this case, almost all the time is spent evaluating f1() ("FUN"),
even after calling compiler::cmpfun on f1() and on a function
containing the loop. Making the looping construct faster is not going
to improve the performance of this code by a significant amount.
I.e., dropping to compiled code will only help if you avoid the R
function call, but then that's not a general solution...
> --
>
> David Winsemius
> Alameda, CA, USA
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] segfault with POSIXlt zone=NULL zone=""
On Tue, Dec 6, 2016 at 10:30 AM, wrote: > Hi Joshua, > > Thank you for minimizing my test case. > >> > Hope I'm not doing something illegal... >> > >> You are. You're changing the internal structure of a POSIXlt object >> by re-ordering the list elements. You should not expect a malformed >> POSIXlt object to behave as if it's correctly formed. You can see >> it's malformed by comparing it's unclass()'d output. >> >> d <- as.POSIXlt(Sys.time()) >> unclass(d) # valid POSIXlt object >> d$zone <- NULL >> d$zone <- "" >> unclass(d) # your malformed POSIXlt object > > I don't know if these questions are not already obvious, but: > > 1. Is there a reasonable way to fail more elegantly when a user makes > this mistake? > It's not just "this mistake". See below. > 2. Should we update the documentation for POSIXlt to warn people that > the optional "zone" list element must precede the optional "gmtoff" > list element, in cases where both are present? > No, because that's not the only way to create a malformed POSIXlt object. Reordering *any* of the elements results in a segfault, and there are probably other things you could do to the internal structure of POSIXlt objects to cause segfaults. Maybe update the documentation to say, "If you update the internal structure of a POSIXlt object, you deserve whatever happens."? ;-) > Thanks, > > Frederick -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] segfault with POSIXlt zone=NULL zone=""
On Tue, Dec 6, 2016 at 6:37 AM, wrote:
> Hi all,
>
> I ran into a segfault while playing with dates.
>
> $ R --no-init-file
> ...
> > library(lubridate); d=as.POSIXlt(floor_date(Sys.time(),"year"));
> d$zone=NULL; d$zone=""; d
>
If you're asking about a bug in R, you should provide a *minimal*
reproducible example (i.e. one without any package dependencies).
This has nothing to do with lubridate, so you can reproduce the
behavior with:
d <- as.POSIXlt(Sys.time())
d$zone <- NULL
d$zone <- ""
d
> Attaching package: ‘lubridate’
>
> The following object is masked from ‘package:base’:
>
> date
>
> Warning message:
> package ‘lubridate’ was built under R version 3.4.0
>
> *** caught segfault ***
> address (nil), cause 'unknown'
>
> Traceback:
> 1: format.POSIXlt(x, usetz = TRUE)
> 2: format(x, usetz = TRUE)
> 3: print(format(x, usetz = TRUE), ...)
> 4: print.POSIXlt(x)
> 5: function (x, ...) UseMethod("print")(x)
>
> Possible actions:
> ...
>
> Hope I'm not doing something illegal...
>
You are. You're changing the internal structure of a POSIXlt object
by re-ordering the list elements. You should not expect a malformed
POSIXlt object to behave as if it's correctly formed. You can see
it's malformed by comparing it's unclass()'d output.
d <- as.POSIXlt(Sys.time())
unclass(d) # valid POSIXlt object
d$zone <- NULL
d$zone <- ""
unclass(d) # your malformed POSIXlt object
> Thanks,
>
> Frederick
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Calling C implementations of rnorm and friends
help me at all in understanding how it works. It should create a >>> function random2(sn, sa, sb, norm, REALSXP); I understand that is a >>> version >>> of the random2 function that returns a real S expression taking sn, sa and >>> sb as parameters. But how does find the actual functional form for the >>> normal distribution? >>> >>> I am asking because I would like to rewrite some of the other functions, >>> such as parameterizing rbeta by the mean and sample size rather than by >>> the >>> number of successes and failures and rgamma by the mean and total time >>> elapsed instead of the number of events. Once I understand how the C >>> source >>> code works, it would be hopefully not very difficult to reparameterize >>> them. >>> >>> Thanks, >>> >>> Luis Usier >>> >>> [[alternative HTML version deleted]] >>> >>> __ >>> R-devel@r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> >> > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] RODBC on Mac & _R_CHECK_FORCE_SUGGESTS_
On Mon, Jun 20, 2016 at 9:19 AM, Spencer Graves
wrote:
>"R CMD check sos" with R 3.3.0 under Mac OS X 10.11.5 ends as
> follows:
>
>
> >* checking package dependencies ... ERROR
> >Package suggested but not available: ‘RODBC’
> >
> >The suggested packages are required for a complete check.
> >Checking can be attempted without them by setting the environment
> >variable _R_CHECK_FORCE_SUGGESTS_ to a false value.
>
>
>Unfortunately, "install.packages('RODBC')" says it's only available
> in source form. When I attempt to install from sources, it fails as
> follows:
>
>
> >checking for sqlext.h... no
> >configure: error: "ODBC headers sql.h and sqlext.h not found"
> >ERROR: configuration failed for package ‘RODBC’
> >* removing
> ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/RODBC’
> >Warning in install.packages :
> > installation of package ‘RODBC’ had non-zero exit status
>
>
>The CRAN checks for RODBC also failed for the same reason
> (https://cloud.r-project.org/web/checks/check_results_RODBC.html).
>
>
>
>I'm not sure how to set "the environment variable
> _R_CHECK_FORCE_SUGGESTS_ to a false value" on my Mac, and I'd rather not do
> it permanently. I tried various versions of "R CMD check
> _R_CHECK_FORCE_SUGGESTS_=FALSE sos_1.3-9.tar.gz" and "R CMD check
> _R_CHECK_FORCE_SUGGESTS_=0 sos_1.3-9.tar.gz", none of which worked.
> Somewhere I found a suggestion to try, "R CMD check --as-cran
> sos_1.3-9.tar.gz". That also failed for me.
>
_R_CHECK_FORCE_SUGGESTS_=FALSE R CMD check sos_1.3-9.tar.gz
>
>Suggestions?
>Thanks,
>Spencer Graves
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R-SIG-Finance] [VC++ calling R] How to create a real-time interactive ticking time-series chart using dygraph via RInside?
Please choose *one* _relevant_ mailing list. Spamming 5 (!)
mailing lists fragments the conversation and makes things difficult
for everyone involved.
On Mon, Apr 11, 2016 at 10:03 AM, Mike Deanza wrote:
> Hi all,
>
>
> I am trying to figure out how to do this in R and I need your help.
>
>
> My journey started from something like the following:
>
>
> http://stackoverflow.com/questions/11365857/real-time-auto-updating-incremental-plot-in-r/1#1
>
>
> n=1000
> df=data.frame(time=1:n,y=runif(n))
> window=100
> for(i in 1:(n©\window)) {
> flush.console()
> plot(df$time,df$y,type='l',xlim=c(i,i+window))
> Sys.sleep(.09)
> }
>
>
> Then I wanted to make it nicer looking so I went to dygraph.
>
>
> And then, I would like to be able to live send tick data from within Visual
> C++ so I started to investigate RInside.
>
>
> Following the example code here:
>
>
> http://dirk.eddelbuettel.com/papers/useR2009RcppRInside.pdf
>
>
> I can open an RInside object in VC++, and then send some data to it, and then
> execute some command in it, and then get data back.
>
>
> It is really great.
>
>
> However, is there a way to have the real-time updating ticking plots to be
> drawn on dygraph inside RInside?
>
>
> It turns out the dygraph package tends to draw onto a browser. That makes the
> real-time updating pretty slow.
>
>
> Is there a way to set the dygraph to plot to a GUI window in VC++? For
> example, a QT or MFC GUI window?
>
>
> My working environment is Win7 64bit ,with VS 2013 and VS2015, QT 5.3 32bit.
>
>
> Could anybody please shed some lights on me?
>
>
> Thanks a lot!
>
>
>
>
>
>
>
> [[alternative HTML version deleted]]
>
>
> ___
> r-sig-fina...@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] %OS on output
On Wed, Feb 24, 2016 at 10:40 AM, Suharto Anggono Suharto Anggono via
R-devel wrote:
> R help on 'strptime' has the following in "Details" section.
> Specific to R is ‘%OSn’, which for output gives the seconds truncated to ‘0
> <= n <= 6’ decimal places (and if ‘%OS’ is not followed by a digit, it uses
> the setting of ‘getOption("digits.secs")’, or if that is unset, ‘n = 3’).
>
> In reality, for output, if '%OS' is not followed by a digit and
> getOption("digits.secs") is unset, the output has no fractional part, as if n
> = 0 is used.
>
That's because n = 0 _is_ used, and appears to have always been the
default (since the logic was added in r37395). So this appears to be
a typo in the documentation (the comment about "n=3" when digits.secs
is unset was added in r37439).
>> getOption("digits.secs")
> NULL
>> z <- strptime("20/2/06 11:16:16.683", "%d/%m/%y %H:%M:%OS")
>> format(z, "%OS")
> [1] "16"
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] should `data` respect default.stringsAsFactors()?
On Thu, Feb 18, 2016 at 6:03 PM, Cook, Malcolm wrote:
> Hi Peter,
>
> Sorry if I was not clear. Perhaps an example will make my point:
>
>> data(iris)
>> class(iris$Species)
> [1] "factor"
>> write.table(iris,'data/myiris.tab')
>> data(myiris)
>> class(myiris$Species)
> [1] "factor"
>> rm(myiris)
>> options(stringsAsFactors = FALSE)
>> data(myiris)
>> class(myiris$Species)
> [1] "factor"
>> myiris<-read.table("data/myiris.tab",header=TRUE)
>> class(myiris$Species)
> [1] "character"
>
> I am surprised to find that in the above
> setting the global option stringsAsFactors = FALSE does NOT effect
> how Species is being read in by the `data` function
> whereas
> setting the global option stringsAsFactors = FALSE DOES effect how
> Species is being read in by read.table
>
> especially since data is documented as calling read.table.
>
To be explicit, it's documented as calling read.table(..., header =
TRUE) in this case, but it actually calls read.table(..., header =
TRUE, as.is = FALSE), which results in class(myiris$Species) of
"factor".
R> myiris<-read.table("data/myiris.tab",header=TRUE,as.is=FALSE)
R> class(myiris$Species)
[1] "factor"
So it seems like adding as.is = FALSE to the call in the documentation
would clear this up.
> In my opinion, one or the other should change (the behavior of data, or the
> documentation).
>
> ,
>
> ~ Malcolm
>
>
> > -Original Message-
> > From: peter dalgaard [mailto:pda...@gmail.com]
> > Sent: Thursday, February 18, 2016 3:32 PM
> > To: Cook, Malcolm
> > Cc: r-de...@stat.math.ethz.ch
> > Subject: Re: [Rd] should `data` respect default.stringsAsFactors()?
> >
> > What the are you on about? data() does many things, only some of
> > which call read.table() et al., and the ones that do have no special
> treatment
> > of stringsAsFactors.
> >
> > -pd
> >
> > > On 18 Feb 2016, at 21:25 , Cook, Malcolm wrote:
> > >
> > > Hiya,
> > >
> > > Probably been debated elsewhere
> > >
> > > I note that R's `data` function does not respect default.stringsAsFactors
> > >
> > > By my lights, it should, especially as it is documented to call
> read.table,
> > which DOES respect.
> > >
> > > Oh, but: http://r.789695.n4.nabble.com/stringsAsFactors-FALSE-
> > tp921891p921893.html
> > >
> > > Compelling. I have to agree.
> > >
> > > So, I change my mind.
> > >
> > > By my lights, `data` should then be documented to NOT respect
> > default.stringsAsFactors.
> > >
> > > Else?
> > >
> > > ~Malcolm Cook
> > >
> > > __________
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> > --
> > Peter Dalgaard, Professor,
> > Center for Statistics, Copenhagen Business School
> > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> > Phone: (+45)38153501
> > Office: A 4.23
> > Email: pd@cbs.dk Priv: pda...@gmail.com
> >
> >
> >
> >
> >
> >
> >
> >
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Confusing print method for Inf dates
On Wed, Oct 21, 2015 at 4:57 PM, Hadley Wickham wrote: > x <- as.Date(Inf, origin = "1970-01-01") > x > #> [1] NA > str(x) > #> Date[1:1], format: NA > unclass(x) > #> [1] Inf > > It's not clear what the correct behaviour is. The documentation for > ?Date has: "It is intended that the date should be an integer,", which > suggests that -Inf and Inf are not valid dates. But if that's true the You omitted the second half of the sentence, which contains important information. The entire sentence is, "It is intended that the date should be an integer, but this is not enforced in the internal representation." Since it's not enforced internally, it doesn't necessarily follow that non-integer values are invalid dates. The rest of the paragraph describes how fractional (internal) dates can be created. Both ?format.Date and ?strptime say that 'NA' dates/times are printed as NA_character_. It might be clearer to say that "invalid" dates/times are printed as NA_character_. > behaviour for max.Date() needs some thought: > > max(as.Date(NA), na.rm = TRUE) > #> Warning in max.default(structure(NA_real_, class = "Date"), na.rm = TRUE): > #> no non-missing arguments to max; returning -Inf > #> [1] NA > > If dates are integers, then there is no date that is smaller than all > other dates, so it's not clear what max() should return - NA? > But they're not integers in the strict sense. > Hadley > > -- > http://had.co.nz/ > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Weird issue when iterating through dates
On Wed, Aug 12, 2015 at 10:55 AM, Gábor Csárdi wrote:
> I am not sure if this is a bug or not.
>
I would argue that this isn't a bug, not even in the documentation of
"for" (even though it might be clearer). ?"for" says that `seq` is
"[A]n expression evaluating to a vector (including a list and an
expression) or to a pairlist or 'NULL'". Date objects aren't strictly
vectors, so they're treated as integer/numeric.
This answer on StackOverflow said that "for" does not copy any of the
iterators attributes (including class), which causes this behavior.
http://stackoverflow.com/a/23278464/271616
To respond to the original question regarding why the code below,
"prints the dates as a string". Quite simply, you convert seq(d1,d2,
by=1) to character, so it's no longer a Date. The fact that
Sys.Date() and as.character(Sys.Date()) both *print* the same thing
does not mean they are the same.
for ( dt in as.character(seq(d1,d2, by=1)) ) {
print(dt)
}
Best,
Josh
> Gabor
>
> On Wed, Aug 12, 2015 at 11:51 AM, Luca Cerone wrote:
>> Following up on this, should I report a bug? can you drive me through
>> the process?
>>
>> Cheers,
>> Luca
>>
>> On Thu, Aug 6, 2015 at 4:55 PM, William Dunlap wrote:
>>>>> Just a quick question: what's the difference between `[.Date` and
>>>>> `[[.Date`?
>>>>> Is it supposed to be the method for accessing the value right?
>>>>
>>>>For Dates and atomic vectors in general they are the same, but ...
>>>
>>> Even for atomic vectors with names they are not quite the same
>>> > c(One=1, Two=2)[[2]]
>>> [1] 2
>>> > c(One=1, Two=2)[2]
>>> Two
>>> 2
>>> (and [[ will only return 1 item, unlike [).
>>>
>>>
>>> Bill Dunlap
>>> TIBCO Software
>>> wdunlap tibco.com
>>>
>>> On Thu, Aug 6, 2015 at 5:36 AM, Gábor Csárdi wrote:
>>>>
>>>> On Thu, Aug 6, 2015 at 6:30 AM, Luca Cerone wrote:
>>>> [...]
>>>> > Just a quick question: what's the difference between `[.Date` and
>>>> > `[[.Date`?
>>>> > Is it supposed to be the method for accessing the value right?
>>>>
>>>> For Dates and atomic vectors in general they are the same, but in
>>>> general they are two different operators that behave differently on
>>>> some data types. E.g. on lists [ selects a sub-list and [[ selects a
>>>> single element.
>>>>
>>>> Gabor
>>>>
>>>> [...]
>>>>
>>>> __
>>>> R-devel@r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R CMD build failure
On Thu, Jul 9, 2015 at 12:26 PM, Therneau, Terry M., Ph.D. wrote: > I have a local library 'dart' that imports "httr". It has routines that > access central patient data such as birth date, so it is heavily used > locally but of no interest to anyone else. > > The httr library (and 300 others) are in a shared directory, referenced by > everyone in the biostatistics group via adding this location to the > .libPaths in their default .Rprofile. (Actually, their .Rprofile starts by > running material from a central one, the libPaths is there). > > When I run R (3.2.0) all is fine, but R CMD build fails with the text below > > * creating vignettes ... ERROR > Error: processing vignette 'dart.Rnw' failed with diagnostics: > chunk 2 (label = auth1) > Error : package ‘httr’ required by ‘dart’ could not be found > Execution halted > > > If I add the requiste directory to my R_LIBS_USER environment variable then > all is fine. However, that's a nuisance since the location changes over > time (e.g. R releases). The system admins have a whole process that keeps > .bashrc, .Rprofle and etc dot references up to date. Plugging into this is > why we use .Rprofile. They are quite willing to make select changes in the > central file, but with >1000 users any suggested changes in the overall > process do not get a warm welcome. > > Any ideas? There is no mention in the Writing R Extentions manual that it > ignores the Rprofile file. If "suck it up and use R_LIBS_USER" is the > answer, well, there are only a few who build packages. > It is mentioned in ?Startup: "'R CMD check' and 'R CMD build' do not always read the standard startup files, but they do always read specific 'Renviron' files. The location of these can be controlled by the environment variables 'R_CHECK_ENVIRON' and 'R_BUILD_ENVIRON'. If these are set their value is used as the path for the 'Renviron' file; otherwise, files '~/.R/check.Renviron' or '~/.R/build.Renviron' or sub-architecture-specific versions are employed." Maybe one of those options could work for you? > PS, I can't use RHOME:/etc/Rprofile.site since the biostat group is not the > only set of R users. Some other groups, for instance, cannot even see our > central area. > > Terry T. > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R CMD check: Uses the superseded package: ‘doSNOW’
On Tue, Feb 10, 2015 at 5:10 AM, Xavier Robin wrote:
> Oh, I completely missed that one.
> It's very neat as it seems to work both on Windows and Unix.
>
It works on both Windows and *nix because it combines functionality
from snow (Windows) and multicore (*nix).
> Thanks!
> Xavier
>
>
> On 10/02/15 10:52, Martyn Plummer wrote:
>> The CRAN package snow is superseded by the parallel package which is
>> distributed with R since version 2.14.0. Here are the release notes
>>
>> \item There is a new package \pkg{parallel}.
>>
>> It incorporates (slightly revised) copies of packages
>> \CRANpkg{multicore} and \CRANpkg{snow} (excluding MPI, PVM and NWS
>> clusters). Code written to use the higher-level API functions in
>> those packages should work unchanged (apart from changing any
>> references to their namespaces to a reference to \pkg{parallel},
>> and links explicitly to \CRANpkg{multicore} or \CRANpkg{snow} on help
>> pages).
>>
>> So you should replace your dependency on doSNOW with doParallel, which
>> is the equivalent foreach adapter for the parallel package.
>>
>> Martyn
>>
>> On Mon, 2015-02-09 at 23:08 +0100, Xavier Robin wrote:
>>> Dear list,
>>>
>>> When I run an R CMD check --as-cran on my package (pROC) I get the
>>> following note:
>>>> Uses the superseded package: ‘doSNOW’
>>> The fact that it uses the doSNOW package is correct as I have the
>>> following example in an .Rd file:
>>>> #ifdef windows
>>>> if (require(doSNOW)) {
>>>> registerDoSNOW(cl <- makeCluster(2, type = "SOCK"))
>>>> ci(roc2, method="bootstrap", parallel=TRUE)
>>>> \dontrun{ci(roc2, method="bootstrap", parallel=TRUE)}
>>>> \dontshow{ci(roc2, method="bootstrap", parallel=TRUE, boot.n=20)}
>>>> stopCluster(cl)
>>>> }
>>>> #endif
>>>> #ifdef unix
>>>> if (require(doMC)) {
>>>> registerDoMC(2)
>>>> \dontrun{ci(roc2, method="bootstrap", parallel=TRUE)}
>>>> \dontshow{ci(roc2, method="bootstrap", parallel=TRUE, boot.n=20)}
>>>> }
>>>> #endif
>>> The "superseded" part is more confusing to me, though. The doSNOW
>>> package seems to be still available on CRAN with no special notice,
>>> listed in the HighPerformanceComputing view likewise, and under active
>>> development (last change a couple of days ago on R-Forge). I could find
>>> no mention of what it has been superseded with. Surprisingly, Google was
>>> no help on this.
>>>
>>> I could see the note is triggered in QC.R file of the tools package.
>>> However this finding is not much help and leaves me just as confused as
>>> before.
>>>
>>> I recall spending quite some time to setup this example to run both
>>> under Windows and Unix. doSNOW was the only way I could get it to work
>>> there. doMC is apparently still available for Unix only. I couldn't get
>>> doRNG to work on either platforms.
>>>
>>> So what is R CMD check noticing me about?
>>> Should I ignore the notice, or take an action? If so, which one?
>>>
>>> Best wishes,
>>> Xavier
>>>
>
>
> --
> Xavier Robin, PhD
> Cellular Signal Integration Group (C-SIG) - Linding Lab
> Biotech Research and Innovation Center (BRIC) - University of Copenhagen
> Anker Engelundsvej, DTU Campus, Building 301, DK-2800 Lyngby, DENMARK
> Mobile: +45 42 799 833
> www.lindinglab.org - www.bric.ku.dk
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [PATCH] Makefile: add support for git svn clones
On Mon, Jan 19, 2015 at 3:13 PM, Nathan Kurz wrote: > On Mon, Jan 19, 2015 at 1:00 PM, Felipe Balbi wrote: >> I just thought that such a small patch which causes no visible change to >> SVN users and allow for git users to build R would be acceptable, but if >> it isn't, that's fine too. > > Felipe --- > > It would appear that you are unaware that you are walking a minefield > of entrenched positions and personality conflicts. For those like > myself who are mystified by the positions taken in this thread, a > partial back story may be helpful. > > In 2012, Han-Tak Leung reported a problem compiling the development > version of R that he had checked out using git's svn compability > feature: https://stat.ethz.ch/pipermail/r-devel/2012-October/065133.html > You forgot to mention that, in February of 2013, Hin-Tak reported that Matrix did not build with the R trunk since October, 2012; but it turned out not to build because he didn't have a subversion checkout. https://mailman.stat.ethz.ch/pipermail/r-devel/2013-February/065858.html > In 2013, Brian Ripley applied a patch with the comment "trap HK Leung > misuse" explicitly to prevent users from being able to do this: > https://github.com/wch/r-source/commit/4f13e5325dfbcb9fc8f55fc6027af9ae9c7750a3 > > Shortly thereafter, Han-Tak tried to start discussion on this list > about that patch, suggesting that preventing the use of non-SVN > mirrors reduced the frequency with which development versions would be > tested: > https://stat.ethz.ch/pipermail/r-devel/2013-March/066128.html > > The opinions expressed on the thread were universally against Leung. > Peter Dalgaard summarized as: > "The generic point is that you are given access to a working tool that > is internal to the core R developers. We are not putting restrictions > on what you do with that access, but if you want to play the game by > other rules than we do, you need to take the consequences. If things > don't work and you start complaining about them being "broken", steps > may be taken to make it clearer who broke them." > https://stat.ethz.ch/pipermail/r-devel/2013-March/066131.html > > As a newcomer hoping to contribute to R who had already encountered > this same compilation issue and considered it was a bug, I am > astounded to learn that it is instead desired and intentional > behavior. > > --nate > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Problem with build and check
On Wed, Nov 12, 2014 at 1:28 PM, Duncan Murdoch
wrote:
> On 12/11/2014 2:11 PM, Therneau, Terry M., Ph.D. wrote:
>>
>> I am getting failure of build and check, for an Rd file that has a long
>> argument list.
>> Guess diagnosis: a quoted string beyond a certain point in the argument
>> list is fatal.
>
>
> No, the problem is that % is a comment marker in .Rd. You need to escape
> those in the dateformat default. Apparently
> prompt() doesn't know this...
>
> Duncan Murdoch
>
prompt.default does add the escapes for the example section, but not
for the usage section.
>>
>> Example: Use the function below, create an Rd file for it with prompt().
>> Move the .Rd
>> file to the man directory (no need to edit it) and try building
>>
>> dart.control <- function(server=c("production", "integration",
>> "development",
>> "http"),
>>out.poll.duration = 5,
>>out.poll.increase = 1.1,
>>out.poll.max = 30,
>>out.poll.timeout = 3600,
>>netrc.path,
>>netrc.server = "ldap",
>>rtype = c("xml", "json"),
>>dateformat= "%Y-%m-%d") {
>>
>> server <- match.arg(server)
>> server
>> }
>>
>> I created a package "dummy" with only this function, and get the following
>> on my Linux box.
>>
>> tmt-local2021% R CMD build dummy
>> * checking for file ‘dummy/DESCRIPTION’ ... OK
>> * preparing ‘dummy’:
>> * checking DESCRIPTION meta-information ... OK
>> Warning: newline within quoted string at dart.control.Rd:11
>> Warning: /tmp/RtmpjPjz9V/Rbuild398d6e382572/dummy/man/dart.control.Rd:46:
>> unexpected
>> section header '\value'
>> Warning: newline within quoted string at dart.control.Rd:11
>> Error in
>> parse_Rd("/tmp/RtmpjPjz9V/Rbuild398d6e382572/dummy/man/dart.control.Rd", :
>> Unexpected end of input (in " quoted string opened at
>> dart.control.Rd:88:16)
>> Execution halted
>>
>> Session info for my version
>> > sessionInfo()
>> R Under development (unstable) (2014-10-30 r66907)
>> Platform: i686-pc-linux-gnu (32-bit)
>>
>> locale:
>>[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>[3] LC_TIME=en_US.UTF-8LC_COLLATE=C
>>[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
>>[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
>>[9] LC_ADDRESS=C LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] stats graphics grDevices utils datasets methods base
>>
>>
>> Terry T.
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] subscripting a data.frame (without changing row order) changes internal row.names
On Mon, Nov 10, 2014 at 12:35 PM, Dr Gregory Jefferis
wrote:
> Dear R-devel,
>
> Can anyone help me to understand this? It seems that subscripting the rows
> of a data.frame without actually changing their order, somehow changes an
> internal representation of row.names that is revealed by e.g.
> dput/dump/serialize
>
> I have read the docs and inspected the (R) code for data.frame, rownames,
> row.names and dput without enlightenment.
>
Look at ?.row_names_info (which is mentioned in the See Also section
of ?row.names) and its type argument. Also see the discussion here:
http://stackoverflow.com/q/26468746/271616
> df=data.frame(a=1:10, b=1)
> dput(df)
> df2=df[1:nrow(df), ]
> # R thinks they are equal (so do I!)
> all.equal(df, df2)
> dput(df2)
>
> Looking at the output of the dputs
>
>> dput(df)
>
> structure(list(a = 1:10, b = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)), .Names =
> c("a",
> "b"), row.names = c(NA, -10L), class = "data.frame")
>>
>> dput(df2)
>
> structure(list(a = 1:10, b = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)), .Names =
> c("a",
> "b"), row.names = c(NA, 10L), class = "data.frame")
>
> we have row.names = c(NA, -10L) in the first case and row.names = c(NA, 10L)
> in the second, so somehow these objects have a different representation
>
> Can anyone explain why? This has come up because
>
The first are "automatic". The second are a compact form of 1:10, as
mentioned in ?row.names. I'm not certain of the root cause/reason,
but the second object will not have "automatic" rownames because you
have subset it with a non-missing 'i'.
>> library(digest)
>> digest(df)==digest(df2)
>
> [1] FALSE
>
> digest uses serialize under the hood, but serialize, dput and dump all show
> the same effect (I've pasted an example below using dump, md5sum from base
> R).
>
> Many thanks for any enlightenment! More generally is there any way to
> calculate a digest of a data.frame that could get round this issue or is
> that not possible?
>
> Best wishes,
>
> Greg.
>
>
> A digest using base R:
>
> library(tools)
> td=tempfile()
> dir.create(td)
> tempfiles=file.path(td,c("df", "df2"))
> dump("df",tempfiles[1])
> dump("df2",tempfiles[2])
> md5sum(tempfiles)
>
> # different md5sum
>
>> sessionInfo() # for my laptop but also observed on R 3.1.2
>
> R version 3.1.1 (2014-07-10)
> Platform: x86_64-apple-darwin13.1.0 (64-bit)
>
> locale:
> [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
>
> attached base packages:
> [1] tools stats graphics grDevices utils datasets methods
> base
>
> other attached packages:
> [1] nat_1.5.14 nat.utils_0.4.2 digest_0.6.4Rvcg_0.9
> devtools_1.6.1 igraph_0.7.1
> [7] testthat_0.9.1 rgl_0.93.1098
>
> loaded via a namespace (and not attached):
> [1] codetools_0.2-9 filehash_2.2-2nabor_0.4.3 parallel_3.1.1
> plyr_1.8.1
> [6] Rcpp_0.11.3 rstudio_0.98.1062 rstudioapi_0.1XML_3.98-1.1
> yaml_2.1.13
>
> --
> Gregory Jefferis, PhD
> Division of Neurobiology
> MRC Laboratory of Molecular Biology
> Francis Crick Avenue
> Cambridge Biomedical Campus
> Cambridge, CB2 OQH, UK
>
> http://www2.mrc-lmb.cam.ac.uk/group-leaders/h-to-m/g-jefferis
> http://jefferislab.org
> http://flybrain.stanford.edu
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Unexpected behavior of identical() with language objects
On Wed, Oct 29, 2014 at 3:26 PM, Winston Chang wrote:
> I ran into this and found the result very surprising:
>
> identical( quote({ a }), quote({ a }) )
> # FALSE
>
> It seems related to curly braces. For example, parens work fine:
> identical( quote(( a )), quote(( a )) )
> # TRUE
>
> Is this expected behavior? I can't seem to find anything in the help
> for identical that relates to this.
>
It's not in ?identical, but ?Paren gives you some pointers.
str(quote((a))) and str(quote({a})) are also informative.
> -Winston
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] "Fastest" way to merge 300+ .5MB dataframes?
The same comment Jeroen Ooms made about your last email also applies to this one: it is better suited to R-help. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sun, Aug 10, 2014 at 1:18 PM, Grant Rettke wrote: > Good afternoon, > > Today I was working on a practice problem. It was simple, and perhaps > even realistic. It looked like this: > • Get a list of all the data files in a directory > • Load each file into a dataframe > • Merge them into a single data frame > > Because all of the columns were the same, the simplest solution in my > mind was to `Reduce' the vector of dataframes with a call to > `merge'. That worked fine, I got what was expected. That is key > actually. It is literally a one-liner, and there will never be index > or scoping errors with it. > > Now with that in mind, what is the idiomatic way? Do people usually do > something else because it is /faster/ (by some definition)? > > Kind regards, > > > Grant Rettke | ACM, ASA, FSF, IEEE, SIAM > g...@wisdomandwonder.com | http://www.wisdomandwonder.com/ > “Wisdom begins in wonder.” --Socrates > ((λ (x) (x x)) (λ (x) (x x))) > “Life has become immeasurably better since I have been forced to stop > taking it seriously.” --Thompson > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Typeof for character vector in dataframe returns integer
On Apr 1, 2014 8:04 PM, "Sandip Nandi" wrote:
>
> Hi ,
>
> I am concerned about the return value of typeof in character column in a
dataframe. I expect it to be of character ,it returns integer instead .
>
Of course typeof returns integer; typeof returns the internal storage type
of an object. Factors are integer vectors with a levels attribute. See
?typeof and ?factor.
As I said, it's not a character column. data.frame coerced it to factor.
> Thanks
>
>
>
>
> On Tue, Apr 1, 2014 at 5:56 PM, Joshua Ulrich
wrote:
>>
>>
>> On Apr 1, 2014 7:48 PM, "Sandip Nandi" wrote:
>> >
>> > Hi ,
>> >
>> > I want to know is this behavior expected and why is that ? Need some
help
>> >
>> > gender <- c("F", "M", "M", "F", "F", "M", "F", "F")
>> > > age<- c(23, 25, 27, 29, 31, 33, 35, 37)
>> > > df<- data.frame(gender,age)
>> > > typeof(df[[1]])
>> > [1] "integer" >>>>>>>>>>>>> Why is this integer . *Should not it be
>> > character ?*
>> > > typeof(df[[2]])
>> > [1] "double"
>> >
>> > > typeof(gender)
>> > [1] "character"
>> > > typeof(age)
>> > [1] "double"
>> > >
>> >
>> > In my code i am trying to do some thing based on typeof and the type
for
>> > character column is strange.
>> >
>> The first column is coerced to factor when stringsAsFactors=TRUE, which
is the default for data.frame. See ?data.frame.
>>
>> > Thanks,
>> > Sandip
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > __
>> > R-devel@r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Typeof for character vector in dataframe returns integer
On Apr 1, 2014 7:48 PM, "Sandip Nandi" wrote:
>
> Hi ,
>
> I want to know is this behavior expected and why is that ? Need some help
>
> gender <- c("F", "M", "M", "F", "F", "M", "F", "F")
> > age<- c(23, 25, 27, 29, 31, 33, 35, 37)
> > df<- data.frame(gender,age)
> > typeof(df[[1]])
> [1] "integer" > Why is this integer . *Should not it be
> character ?*
> > typeof(df[[2]])
> [1] "double"
>
> > typeof(gender)
> [1] "character"
> > typeof(age)
> [1] "double"
> >
>
> In my code i am trying to do some thing based on typeof and the type for
> character column is strange.
>
The first column is coerced to factor when stringsAsFactors=TRUE, which is
the default for data.frame. See ?data.frame.
> Thanks,
> Sandip
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 5:16 PM, Jeroen Ooms wrote: > On Wed, Mar 19, 2014 at 2:59 PM, Joshua Ulrich > wrote: >> >> So implementation isn't a problem. The problem is that you need a way >> to force people not to be able to use different package versions than >> what existed at the time of each R release. I said this in my >> previous email, but you removed and did not address it: "However, you >> would need to find a way to actively _prevent_ people from installing >> newer versions of packages with the stable R releases." Frankly, I >> would stop using CRAN if this policy were adopted. > > I am not proposing to "force" anything to anyone, those are your > words. Please read the proposal more carefully before derailing the > discussion. Below *verbatim* a section from the paper: > Yes "force" is too strong a word. You want a barrier (however small) to prevent people from installing newer (or older) versions of packages than those that correspond to a given R release. I still think you're going to have a very hard time convincing CRAN maintainers to take up your cause, even if you were to build support for it. Especially because there's nothing stopping anyone else from doing it. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 4:28 PM, Jeroen Ooms wrote: > On Wed, Mar 19, 2014 at 11:50 AM, Joshua Ulrich > wrote: >> >> The suggested solution is not described in the referenced article. It >> was not suggested that it be the operating system's responsibility to >> distribute snapshots, nor was it suggested to create binary >> repositories for specific operating systems, nor was it suggested to >> freeze only a subset of CRAN packages. > > > IMO this is an implementation detail. If we could all agree on a particular > set of cran packages to be used with a certain release of R, then it doesn't > matter how the 'snapshotting' gets implemented. It could be a separate > repository, or a directory on cran with symbolic links, or a page somewhere > with hyperlinks to the respective source packages. Or you can put all > packages in a big zip file, or include it in your OS distribution. You can > even distribute your entire repo on cdroms (debian style!) or do all of the > above. > > The hard problem is not implementation. The hard part is that for > reproducibility to work, we need community wide conventions on which > versions of cran packages are used by a particular release of R. Local > downstream solutions are impractical, because this results in > scripts/packages that only work within your niche using this particular > snapshot. I expect that requiring every script be executed in the context of > dependencies from some particular third party repository will make > reproducibility even less common. Therefore I am trying to make a case for a > solution that would naturally improve reliability/reproducibility of R code > without any effort by the end-user. > So implementation isn't a problem. The problem is that you need a way to force people not to be able to use different package versions than what existed at the time of each R release. I said this in my previous email, but you removed and did not address it: "However, you would need to find a way to actively _prevent_ people from installing newer versions of packages with the stable R releases." Frankly, I would stop using CRAN if this policy were adopted. I suggest you go build this yourself. You have all the code available on CRAN, and the dates at which each package was published. If others who care about reproducible research find what you've built useful, you will create the very community you want. And you won't have to force one single person to change their workflow. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 12:59 PM, Jeroen Ooms wrote: > On Wed, Mar 19, 2014 at 5:52 AM, Duncan Murdoch > wrote: > >> I don't see why CRAN needs to be involved in this effort at all. A third >> party could take snapshots of CRAN at R release dates, and make those >> available to package users in a separate repository. It is not hard to set >> a different repository than CRAN as the default location from which to >> obtain packages. >> > > I am happy to see many people giving this some thought and engage in the > discussion. > > Several have suggested that staging & freezing can be simply done by a > third party. This solution and its limitations is also described in the > paper [1] in the section titled "R: downstream staging and repackaging". > > If this would solve the problem without affecting CRAN, we would have been > done this obviously. In fact, as described in the paper and pointed out by > some people, initiatives such as Debian or Revolution Enterprise already > include a frozen library of R packages. Also companies like Google maintain > their own internal repository with packages that are used throughout the > company. > The suggested solution is not described in the referenced article. It was not suggested that it be the operating system's responsibility to distribute snapshots, nor was it suggested to create binary repositories for specific operating systems, nor was it suggested to freeze only a subset of CRAN packages. > The problem with this approach is that when you using some 3rd party > package snapshot, your r/sweave scripts will still only be > reliable/reproducible for other users of that specific snapshot. E.g. for > the examples above, a script that is written in R 3.0 by a Debian user is > not guaranteed to work on R 3.0 in Google, or R 3.0 on some other 3rd party > cran snapshot. Hence this solution merely redefines the problem from "this > script depends on pkgA 1.1 and pkgB 0.2.3" to "this script depends on > repository foo 2.0". And given that most users would still be pulling > packages straight from CRAN, it would still be terribly difficult to > reproduce a 5 year old sweave script from e.g. JSS. > This can be solved by the third party making the repository public. > For this reason I believe the only effective place to organize this staging > is all the way upstream, on CRAN. Imagine a world where your r/sweave > script would be reliable/reproducible, out of the box, on any system, any > platform in any company using on R 3.0. No need to investigate which > specific packages or cran snapshot the author was using at the time of > writing the script, and trying to reconstruct such libraries for each > script you want to reproduce. No ambiguity about which package versions are > used by R 3.0. However for better or worse, I think this could only be > accomplished with a cran release cycle (i.e. "universal snapshots") > accompanying the already existing r releases. > This could be done by a public third-party repository, independent of CRAN. However, you would need to find a way to actively _prevent_ people from installing newer versions of packages with the stable R releases. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Tue, Mar 18, 2014 at 3:24 PM, Jeroen Ooms wrote: > ## Summary > > Extending the r-release cycle to CRAN seems like a solution that would > be easy to implement. Package updates simply only get pushed to the > r-devel branches of cran, rather than r-release and r-release-old. > This separates development from production/use in a way that is common > sense in most open source communities. Benefits for R include: > Nothing is ever as simple as it seems (especially from the perspective of one who won't be doing the work). There is nothing preventing you (or anyone else) from creating repositories that do what you suggest. Create a CRAN mirror (or more than one) that only include the package versions you think they should. Then have your production servers use it (them) instead of CRAN. Better yet, make those repositories public. If many people like your idea, they will use your new repositories instead of CRAN. There is no reason to impose this change on all world-wide CRAN users. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Regarding: stat.math.ethz.ch mailing list memberships reminder
Please read the instructions/warnings before subscribing. "You may enter a privacy password below. This provides only mild security, but should prevent others from messing with your subscription. Do not use a valuable password as it will occasionally be emailed back to you in cleartext." -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Fri, Mar 7, 2014 at 11:17 AM, Sachko Honda wrote: > Please never ever send the password in clear text, never!!! > > Sachko Honda > > Phone: 425.284.7200 > Fax: 425.284.7201 > sachko.ho...@mountainpacificgroup.com > Mountain Pacific Group > 11820 Northup Way, Suite E210 > Bellevue, WA 98005-1926 > > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Possible POSIXlt / wday glitch & bugs.r-project.org status
In an effort to redeem myself, I have found and submitted a patch for what seems to be causing this issue. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sat, Oct 5, 2013 at 1:18 PM, Simon Urbanek wrote: > On Oct 5, 2013, at 4:51 PM, Sean O'Riordain wrote: > >> Some people (luckily not me anymore!) working with mortgages and >> pensions need to calculate up to 40 years into the future for the >> payment schedule. >> > > Just to clarify since the Joshua's comment was ambiguous (and in part plain > wrong) - R's POSIXct has no such limit since it doesn't use integers, so that > is not really the issue here. As the original post suggested there may be a > bug in handing some cases where conversions hit the system libraries (that > may truncate to integers) and some cases may be worked around - and that > remains to be investigated. > > Cheers, > Simon > > > >> On 5 October 2013 02:37, Joshua Ulrich wrote: >>> On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan wrote: >>>> Thanks for the responses and quoting the timezone help file. >>>> >>>> I am assuming that in order to determine the wday element of POSIXlt, R >>>> does the necessary calculations in Julian time (via POSIXct). Based on >>>> this excerpt from ?DateTimeClasses, it looks like R is responsible for >>>> determining time zones post 2037 (the example I gave was in 2038). So it >>>> could be an R issue. >>>> >>> It's an issue with size of the largest number you can store in a >>> signed integer, which is not specific to R. >>> >>>> .POSIXct(.Machine$integer.max, tz="UTC") >>> [1] "2038-01-19 03:14:07 UTC" >>> >>> Dates larger than that cannot be represented by a signed integer. It >>> could be worked around, but it's not trivial because R would have to >>> use something other than the tm C struct. Luckily, there's a decade >>> or two before it starts to become a pressing issue. :) >>> >>>>> ‘"POSIXct"’ objects may also have an attribute ‘"tzone"’, a >>>>> character vector of length one. If set to a non-empty value, it >>>>> will determine how the object is converted to class ‘"POSIXlt"’ >>>>> and in particular how it is printed. This is usually desirable, >>>>> but if you want to specify an object in a particular timezone but >>>>> to be printed in the current timezone you may want to remove the >>>>> ‘"tzone"’ attribute (e.g. by ‘c(x)’). >>>>> >>>>> Unfortunately, the conversion is complicated by the operation of >>>>> time zones and leap seconds (24 days have been 86401 seconds long >>>>> so far: the times of the extra seconds are in the object >>>>> ‘.leap.seconds’). **The details of this are entrusted to the OS >>>>> services where possible. This always covers the period 1970-2037, >>>>> and on most machines back to 1902 (when time zones were in their >>>>> infancy). Outside the platform limits we use our own C code. >>>> >>>> >>>> On 05/10/2013, at 12:59 AM, Scott Kostyshak wrote: >>>> >>>>> On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan >>>>> wrote: >>>>>> Wanted to raise two questions: >>>>>> >>>>>> 1. Is bugs.r-project.org down? I haven't been able to reach it for two >>>>>> or three days: >>>>> >>>>> Yes. Quote from Duncan: >>>>> >>>>> ... the server is currently down. The volunteer who runs the server is >>>>> currently away from his office, so I expect it won't get fixed until he >>>>> gets back in a few days. >>>>> >>>>> https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html >>>>> >>>>> Scott >>>>> >>>>>> >>>>>> ``` >>>>>> ping bugs.r-project.org >>>>>> PING rbugs.research.att.com (207.140.168.137): 56 data bytes >>>>>> Request timeout for icmp_seq 0 >>>>>> Request timeout for icmp_seq 1 >>>>>> Request timeout for icmp_seq 2 >>>>>> Request timeout for icmp_seq 3 >>>>>> Request timeout for icmp_seq 4 >>>>>> Req
Re: [Rd] Possible POSIXlt / wday glitch & bugs.r-project.org status
On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan wrote: > Thanks for the responses and quoting the timezone help file. > > I am assuming that in order to determine the wday element of POSIXlt, R does > the necessary calculations in Julian time (via POSIXct). Based on this > excerpt from ?DateTimeClasses, it looks like R is responsible for determining > time zones post 2037 (the example I gave was in 2038). So it could be an R > issue. > It's an issue with size of the largest number you can store in a signed integer, which is not specific to R. > .POSIXct(.Machine$integer.max, tz="UTC") [1] "2038-01-19 03:14:07 UTC" Dates larger than that cannot be represented by a signed integer. It could be worked around, but it's not trivial because R would have to use something other than the tm C struct. Luckily, there's a decade or two before it starts to become a pressing issue. :) >> ‘"POSIXct"’ objects may also have an attribute ‘"tzone"’, a >> character vector of length one. If set to a non-empty value, it >> will determine how the object is converted to class ‘"POSIXlt"’ >> and in particular how it is printed. This is usually desirable, >> but if you want to specify an object in a particular timezone but >> to be printed in the current timezone you may want to remove the >> ‘"tzone"’ attribute (e.g. by ‘c(x)’). >> >> Unfortunately, the conversion is complicated by the operation of >> time zones and leap seconds (24 days have been 86401 seconds long >> so far: the times of the extra seconds are in the object >> ‘.leap.seconds’). **The details of this are entrusted to the OS >> services where possible. This always covers the period 1970-2037, >> and on most machines back to 1902 (when time zones were in their >> infancy). Outside the platform limits we use our own C code. > > > On 05/10/2013, at 12:59 AM, Scott Kostyshak wrote: > >> On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan wrote: >>> Wanted to raise two questions: >>> >>> 1. Is bugs.r-project.org down? I haven't been able to reach it for two or >>> three days: >> >> Yes. Quote from Duncan: >> >>... the server is currently down. The volunteer who runs the server is >>currently away from his office, so I expect it won't get fixed until he >>gets back in a few days. >> >> https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html >> >> Scott >> >>> >>> ``` >>> ping bugs.r-project.org >>> PING rbugs.research.att.com (207.140.168.137): 56 data bytes >>> Request timeout for icmp_seq 0 >>> Request timeout for icmp_seq 1 >>> Request timeout for icmp_seq 2 >>> Request timeout for icmp_seq 3 >>> Request timeout for icmp_seq 4 >>> Request timeout for icmp_seq 5 >>> Request timeout for icmp_seq 6 >>> ``` >>> >>> 2. Is wday element of POSIXlt meant to be timezone invariant? You would >>> expect the wday element to be invariant to the timezone of a date. That is, >>> the same date/time instant of 5th October 2013 in both Australia/Sydney and >>> UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with >>> 1 min past midnight on 5 October 2013: >>> >>> ``` >>> library(lubridate) >>> d_utc <- ymd_hms(2013100501, tz='UTC') >>> d_local <- ymd_hms(2013100501, tz='Australia/Sydney') >>> as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 >>> as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6 >>> ``` >>> >>> But this isn't always the case. For example, >>> >>> ``` >>> d_utc <- ymd_hms(2038100201, tz='UTC') >>> d_local <- ymd_hms(2038100201, tz='Australia/Sydney') >>> as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 >>> as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5 >>> ``` >>> >>> Is this expected behaviour? I would have expected a properly encoded >>> date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone. >>> >>> Obligatory system dump: >>> >>> ``` sessionInfo() >>> R version 3.0.1 (2013-05-16) >>> Platform: x86_64-apple-darwin12.4.0 (64-bit) >>> >>> locale: >>> [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8 >>> >>> attached base packages: >>> [1] stats graphics grDevices utils datasets methods base >>> >>> other attached packages: >>> [1] lubridate_1.3.0 testthat_0.7.1 devtools_1.3 >>> >>> loaded via a namespace (and not attached): >>> [1] colorspace_1.2-4 dichromat_2.0-0digest_0.6.3 evaluate_0.5.1 >>> [5] ggplot2_0.9.3.1grid_3.0.1 gtable_0.1.2 httr_0.2 >>> [9] labeling_0.2 MASS_7.3-29memoise_0.1munsell_0.4.2 >>> [13] parallel_3.0.1 plyr_1.8 proto_0.3-10 >>> RColorBrewer_1.0-5 >>> [17] RCurl_1.95-4.1 reshape2_1.2.2 scales_0.2.3 stringr_0.6.2 >>> [21] tools_3.0.1whisker_0.3-2 >>> >>> ``` >>> >>> Using R compiled by homebrew [1]. But also experiencing the same bug using >>> R installed on Windows 7 from the CRAN binaries. >>> >>> For those interested, I've also noted this on the `lubridate` Github issues >>>
Re: [Rd] Possible POSIXlt / wday glitch & bugs.r-project.org status
Quoting from ?timezone: Note that except on Windows, the operation of time zones is an OS service, and even on Windows a third-party database is used and can be updated (see the section on ‘Time zone names’). Incorrect results will never be an R issue, so please ensure that you have the courtesy not to blame R for them. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Fri, Oct 4, 2013 at 5:11 AM, Imanuel Costigan wrote: > Wanted to raise two questions: > > 1. Is bugs.r-project.org down? I haven't been able to reach it for two or > three days: > > ``` > ping bugs.r-project.org > PING rbugs.research.att.com (207.140.168.137): 56 data bytes > Request timeout for icmp_seq 0 > Request timeout for icmp_seq 1 > Request timeout for icmp_seq 2 > Request timeout for icmp_seq 3 > Request timeout for icmp_seq 4 > Request timeout for icmp_seq 5 > Request timeout for icmp_seq 6 > ``` > > 2. Is wday element of POSIXlt meant to be timezone invariant? You would > expect the wday element to be invariant to the timezone of a date. That is, > the same date/time instant of 5th October 2013 in both Australia/Sydney and > UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 > min past midnight on 5 October 2013: > > ``` > library(lubridate) > d_utc <- ymd_hms(2013100501, tz='UTC') > d_local <- ymd_hms(2013100501, tz='Australia/Sydney') > as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 > as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6 > ``` > > But this isn't always the case. For example, > > ``` > d_utc <- ymd_hms(2038100201, tz='UTC') > d_local <- ymd_hms(2038100201, tz='Australia/Sydney') > as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 > as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5 > ``` > > Is this expected behaviour? I would have expected a properly encoded > date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone. > > Obligatory system dump: > > ``` >> sessionInfo() > R version 3.0.1 (2013-05-16) > Platform: x86_64-apple-darwin12.4.0 (64-bit) > > locale: > [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] lubridate_1.3.0 testthat_0.7.1 devtools_1.3 > > loaded via a namespace (and not attached): > [1] colorspace_1.2-4 dichromat_2.0-0digest_0.6.3 evaluate_0.5.1 > [5] ggplot2_0.9.3.1grid_3.0.1 gtable_0.1.2 httr_0.2 > [9] labeling_0.2 MASS_7.3-29memoise_0.1munsell_0.4.2 > [13] parallel_3.0.1 plyr_1.8 proto_0.3-10 > RColorBrewer_1.0-5 > [17] RCurl_1.95-4.1 reshape2_1.2.2 scales_0.2.3 stringr_0.6.2 > [21] tools_3.0.1whisker_0.3-2 > > ``` > > Using R compiled by homebrew [1]. But also experiencing the same bug using R > installed on Windows 7 from the CRAN binaries. > > For those interested, I've also noted this on the `lubridate` Github issues > page [2], even though this doesn't appear to be a lubridate issue. > > Thanks for any help. > > [1] http://brew.sh > [2] https://github.com/hadley/lubridate/issues/209 > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Thu, Oct 3, 2013 at 9:44 AM, Jens Oehlschlägel
wrote:
> I agree that quoted integer columns are not the most efficient way of
> delivering csv-files. However, the sad reality is that one receives such
> formats and still needs to read the data. Therefore it is not helpful to
> state that one should 'consider "character" to be the correct colClass in
> case an integer is surrounded by quotes'.
>
> The philosophy of read.table.ffdf is delegating the actual csv-parsing to a
> parse engine 'similarly' parametrized like 'read.table'. It is not 'bad
> coding practice' - but a conscious design decision - to assume that the
> parse engine behaves consistently, which read.table does not yet: it
> automatically recognizes a quoted integer column as 'integer', but when
> asked to explicitly interpret the column as 'integer' it does refuse to do
read.table() does not "automatically recognize a quoted integer column
as 'integer'". If colClasses is not specified, it reads the entire
column into a 'character' vector and then calls type.convert() on it.
type.convert() does all the necessary work to determine what class the
'character' vector should be converted to. If colClasses is
specified, quotes are not interpreted in non-'character' columns.
You want scan() to allocate an 'integer' vector, and then ensure (on
each read from the column in the file) that the value read is a valid
'integer' type, while interpreting quotes (which strtol does not do,
so someone would have to write and test this new functionality).
So your complaint is more with scan() than read.table(). And more
with Strtoi() (and therefore strtol) than scan().
> so. So there is nothing wrong with read.table.ffdf (but something can be
> improved about read.table). It is *not* the 'best solution [...] to rewrite
> read.table.ffdf()' given that it nicely imports such data, see 4+1 ways to
> do so below.
>
> Jens Oehlschlägel
>
>
> # --- first create a csv file for demonstration
> ---
> require(ff)
> file <- "test.csv"
> path <- "c:/tmp"
> n <- 1e2
> d <- data.frame(x=1:n, y=shQuote(1:n))
> write.csv(d, file=file.path(path,file), row.names=FALSE, quote=FALSE)
>
> # --- how to do it with read.table.ffdf
> ---
>
> # 1 let the parse engine ignore colClasses and hope for the best
> fixedengine <- function(file, ..., colClasses=NA){
> read.csv(file, ...)
> }
> df <- read.table.ffdf(file=file.path(path,file), first.rows = 10,
> FUN="fixedengine")
> df
>
> # 2 Suspend colClasses(=NA) for the quoted integer column only
> df <- read.csv.ffdf(file=file.path(path,file), first.rows = 10,
> colClasses=c("integer", NA))
> df
>
> # 3 do your own type conversion using transFUN
> # after reading the problematic column as character
> # Being able to inject regexps is quite powerful isn't it?
> # Or error handlinig in case of varying column format!
> custominterp <- function(d){
> d[[2]] <- as.integer(gsub('"', '', d[[2]]))
> d
> }
> df <- read.table.ffdf(file=file.path(path,file), first.rows = 10,
> colClasses=c("integer", "character"), FUN="read.csv", transFUN=custominterp)
> df
>
> # 4 do your own line parsing and type conversion
> # Here you can even handle non-standard formats
> # such as varying number of columns
> customengine <- function(file, header=TRUE, col.names, colClasses=NA,
> nrows=0, skip=0, fileEncoding="", comment.char = ""){
> l <- scan(file, what="character", nlines=nrows+header, skip=skip,
> fileEncoding=fileEncoding, comment.char = comment.char)
> s <- do.call("rbind", strsplit(l, ","))
> if (header){
> d <- data.frame(as.integer(s[-1,1]),
> as.integer(gsub('"','',s[-1,2])))
> names(d) <- s[1,]
> }else{
> d <- data.frame(as.integer(s[,1]),
> as.integer(gsub('"','',s[,2])))
> }
> if (!missing(col.names))
> names(d) <- col.names
> d
> }
> df <- read.table.ffdf(file=file.path(path,file), first.rows = 10,
> FUN="customengine")
> df
>
> # 5 use a parsing engine that can apply colClasses to quoted integers
> # Unfortunately Henry Bengtson's readDataFrame does not work as a
> # parse engine for read.table.ffdf because read.table.ffdf expects
> # the parse engine to read successive chunks from a file connection
> # while readDataFrame only accepts a filename as input file spec.
> # Yes it has 'skip', but using that would reread the file from scratch
> # for each chunk (O(N^2) costs)
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Tue, Oct 1, 2013 at 11:29 AM, David Winsemius wrote:
>
> On Sep 30, 2013, at 6:38 AM, Joshua Ulrich wrote:
>
>> On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat
>> wrote:
>>> Hi!
>>>
>>>
>>> It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider
>>> quoted integers as an acceptable value for columns for which
>>> colClasses="integer". But when colClasses is omitted, these columns are
>>> read as integer anyway.
>>>
>>> For example, let's consider a file named file.dat, containing:
>>> "1"
>>> "2"
>>>
>>>> read.table("file.dat", colClasses="integer")
>>> Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, :
>>> scan() expected 'an integer' and got '"1"'
>>>
>>> But:
>>>> str(read.table("file.dat"))
>>> 'data.frame': 2 obs. of 1 variable:
>>> $ V1: int 1 2
>>>
>>> The latter result is indeed documented in ?read.table:
>>> Unless ‘colClasses’ is specified, all columns are read as
>>> character columns and then converted using ‘type.convert’ to
>>> logical, integer, numeric, complex or (depending on ‘as.is’)
>>> factor as appropriate. Quotes are (by default) interpreted in all
>>> fields, so a column of values like ‘"42"’ will result in an
>>> integer column.
>>>
>>>
>>> Should the former behavior be considered a bug?
>>>
>> No. If you tell read.table the column is integer and it's actually
>> character on disk, it should be an error.
>
> My reading of the `read.table` help page is that one should expect that when
> there is an 'integer'-class and an `as.integer` function and "integer" is
> the
> argument to colClasses, that `as.integer` will be applied to the values in the
> column. Should I be reading elsewhere?
>
I assume you're referring to the paragraph below.
Possible values are ‘NA’ (the default, when ‘type.convert’ is
used), ‘"NULL"’ (when the column is skipped), one of the
atomic vector classes (logical, integer, numeric, complex,
character, raw), or ‘"factor"’, ‘"Date"’ or ‘"POSIXct"’.
Otherwise there needs to be an ‘as’ method (from package
‘methods’) for conversion from ‘"character"’ to the specified
formal class.
I read that as meaning that an "as" method is required for classes not
already listed in the prior sentence. It doesn't say an "as" method
will be applied if colClasses is one of the atomic, factor, Date, or
POSIXct classes; but I can see how you might assume that, since all
the atomic, factor, Date, and POSIXct classes already have "as"
methods...
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Mon, Sep 30, 2013 at 9:45 AM, Milan Bouchet-Valat wrote:
> Le lundi 30 septembre 2013 à 08:38 -0500, Joshua Ulrich a écrit :
>> On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat
>> wrote:
>> > Hi!
>> >
>> >
>> > It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider
>> > quoted integers as an acceptable value for columns for which
>> > colClasses="integer". But when colClasses is omitted, these columns are
>> > read as integer anyway.
>> >
>> > For example, let's consider a file named file.dat, containing:
>> > "1"
>> > "2"
>> >
>> >> read.table("file.dat", colClasses="integer")
>> > Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,
>> > :
>> > scan() expected 'an integer' and got '"1"'
>> >
>> > But:
>> >> str(read.table("file.dat"))
>> > 'data.frame': 2 obs. of 1 variable:
>> > $ V1: int 1 2
>> >
>> > The latter result is indeed documented in ?read.table:
>> > Unless ‘colClasses’ is specified, all columns are read as
>> > character columns and then converted using ‘type.convert’ to
>> > logical, integer, numeric, complex or (depending on ‘as.is’)
>> > factor as appropriate. Quotes are (by default) interpreted in all
>> > fields, so a column of values like ‘"42"’ will result in an
>> > integer column.
>> >
>> >
>> > Should the former behavior be considered a bug?
>> >
>> No. If you tell read.table the column is integer and it's actually
>> character on disk, it should be an error.
> All values in a CSV file are stored as characters on disk, disregarding
> the fact that they are surrounded by quotes or not. 1 is saved as
> 00110001 (ASCII character #49), not 0001, nor
> 0001 (as would for example imply a 32 bit storage of
> integers).
>
Yes, I'm aware that write.table creates a character representation of
the data on disk. That's its purpose. writeBin is for writing actual
binary representations. I thought you would understand that by
"actually character on disk" I meant "actually a quoted value". I
assumed you would understand my intent.
read.table uses scan to read the file. ?scan says:
The allowed input for a numeric field is optional whitespace
followed either ‘NA’ or an optional sign followed by a decimal or
hexadecimal constant (see NumericConstants), or ‘NaN’, ‘Inf’ or
‘infinity’ (ignoring case). Out-of-range values are recorded as
‘Inf’, ‘-Inf’ or ‘0’.
For an integer field the allowed input is optional whitespace,
followed by either ‘NA’ or an optional sign and one or more digits
(‘0-9’): all out-of-range values are converted to ‘NA_integer_’.
There's no mention of quotes being allowed.
> So, with all due respect, please refrain from formulating such blatantly
> erroneous statements.
>
So, with all due respect, please refrain from formulating such
blatantly pedantic responses to someone trying to help you.
>
> Regards
>
>
>> > This creates problems when combined with read.table.ffdf from package
>> > ff, since this function tries to guess the column classes by reading the
>> > first rows of the file, and then passes colClasses to read.table to read
>> > the remaining rows by chunks. A column of quoted integers is correctly
>> > detected as integer in the first read, but read.table() fails in
>> > subsequent reads.
>> >
>> This sounds like a issue with read.table.ffdf. The column of quoted
>> integers is *incorrectly* detected as integer because they're actually
>> character on disk. read.table.ffdf should rely on how the data are
>> actually stored on disk (via as.is=TRUE), not how read.table might
>> convert them once they're read into R.
>>
>> >
>> > Regards
>> >
>> > __
>> > R-devel@r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> --
>> Joshua Ulrich | about.me/joshuaulrich
>> FOSS Trading | www.fosstrading.com
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat wrote:
> Hi!
>
>
> It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider
> quoted integers as an acceptable value for columns for which
> colClasses="integer". But when colClasses is omitted, these columns are
> read as integer anyway.
>
> For example, let's consider a file named file.dat, containing:
> "1"
> "2"
>
>> read.table("file.dat", colClasses="integer")
> Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, :
> scan() expected 'an integer' and got '"1"'
>
> But:
>> str(read.table("file.dat"))
> 'data.frame': 2 obs. of 1 variable:
> $ V1: int 1 2
>
> The latter result is indeed documented in ?read.table:
> Unless ‘colClasses’ is specified, all columns are read as
> character columns and then converted using ‘type.convert’ to
> logical, integer, numeric, complex or (depending on ‘as.is’)
> factor as appropriate. Quotes are (by default) interpreted in all
> fields, so a column of values like ‘"42"’ will result in an
> integer column.
>
>
> Should the former behavior be considered a bug?
>
No. If you tell read.table the column is integer and it's actually
character on disk, it should be an error.
> This creates problems when combined with read.table.ffdf from package
> ff, since this function tries to guess the column classes by reading the
> first rows of the file, and then passes colClasses to read.table to read
> the remaining rows by chunks. A column of quoted integers is correctly
> detected as integer in the first read, but read.table() fails in
> subsequent reads.
>
This sounds like a issue with read.table.ffdf. The column of quoted
integers is *incorrectly* detected as integer because they're actually
character on disk. read.table.ffdf should rely on how the data are
actually stored on disk (via as.is=TRUE), not how read.table might
convert them once they're read into R.
>
> Regards
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Vignette problem and CRAN policies
As Berend Hasselman already told you, "Then surf to the address given
at the end of each posting; go to the bottom of that page and follow
the instructions for unsubscribing."
Here's the link, so you don't have to scroll to the bottom of this
message: https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Mon, Sep 23, 2013 at 4:57 PM, Eric Malitz wrote:
> take me off here
>
>
> On Mon, Sep 23, 2013 at 4:44 PM, Marc Schwartz wrote:
>
>> Spencer,
>>
>> FYI. I just noted in your post below the error message from WriteXLS
>> regarding TEXT::CSV_XS missing.
>>
>> Please note that in version >=3.0 of WriteXLS (current is 3.2.1), that is
>> no longer required and has been replaced by Text::CSV_PP, which is a Pure
>> Perl module and is included in the WriteXLS CRAN package to reduce the
>> dependencies on nonstandard external Perl modules.
>>
>> Regards,
>>
>> Marc Schwartz
>>
>>
>> On Sep 23, 2013, at 4:28 PM, Spencer Graves
>> wrote:
>>
>> > Hello, All:
>> >
>> >
>> > Professor Ripley is correct as usual: I misunderstood his original
>> statement of the problem.
>> >
>> >
>> > He gave two possible solutions. I could not make the first
>> solution work, and I didn't try the second until someone else on this list
>> explained it in slightly more detail.
>> >
>> >
>> > The correction passed R CMD check on my local computer. It has
>> been "building" on R-Forge since 2013-09-20 19:19:14+02. I hope this
>> completes soon enough for me to meet Ripley's Sept. 25 deadline for this
>> correction to "sos".
>> >
>> >
>> > Thanks again to Prof. Ripley and everyone else who took the time to
>> read my post.
>> >
>> >
>> > Spencer
>> >
>> >
>> > On 9/19/2013 12:00 AM, Prof Brian Ripley wrote:
>> >> This is nothing to do with CRAN policies (nor R).
>> >>
>> >> The issue is that the current upquote.sty does not play with 'ae' fonts
>> as used by default by Sweave. The change is in TeX.
>> >>
>> >> And that was what Spencer Graves was informed.
>> >>
>> >>
>> >> On 19/09/2013 04:35, Spencer Graves wrote:
>> >>> Hello, All:
>> >>>
>> >>>
>> >>> The vignette with the sos package used "upquote.sty", required
>> >>> for R Journal when it was published in 2009. Current CRAN policy
>> >>> disallows "upquote.sty", and I've so far not found a way to pass "R CMD
>> >>> check" with sos without upquote.sty.
>> >>>
>> >>>
>> >>> I changed sos.Rnw per an email exchange with Prof. Ripley without
>> >>> solving the problem; see below. The key error messages (see the
>> results
>> >>> of "R CMD build" below) appear to be "sos.tex:16: LaTeX Error:
>> >>> Environment article undefined" and " sos.tex:558: LaTeX Error:
>> >>> \begin{document} ended by \end{article}." When the article worked, it
>> >>> had bot \begin{document} and \begin{article}, with matching \end
>> >>> statements for both. I've tried commenting out either without success.
>> >>>
>> >>>
>> >>> The current nonworking code is available on R-Forge via anonymous
>> >>> SVN checkout using "svn checkout
>> >>> svn://svn.r-forge.r-project.org/svnroot/rsitesearch/". Any suggestions
>> >>> on how to fix this would be greatly appreciated.
>> >>>
>> >>>
>> >>>Thanks,
>> >>>Spencer
>> >>>
>> >>>
>> >>> ## COMPLETE RESULTS FROM R CMD check
>> >>>
>> >>>
>> >>> Microsoft Windows [Version 6.1.7600]
>> >>> Copyright (c) 2009 Microsoft Corporation. All rights reserved.
>> >>>
>> >>> C:\Users\sgraves>cd 2013
>> >>> C:\Users\sgraves\2013>cd R_pkgs
>> >>> C:\Users\sgraves\2013\R_pkgs>cd sos
>> >>> C:\Users\sgraves\2013\R_pkgs\sos>cd pkg
>> >>> C:\Users\sgraves\2013\R_pkgs\sos\pkg>R CMD build sos
>> >>> * checking for file 'sos/DESCRIPTION' ... OK
&
[Rd] Mailing Lists page, turning off HTML mail
Hello, I just noticed that the link to instructions on turning off HTML mail has been dead for quite some time. The last capture I could find on web.archive.org was in mid-2009. http://web.archive.org/web/20090625155306/http://www.expita.com/nomime.html For reference, the link is in the following sentence: "For more details and instructions on turning off HTML for your e-mail software, see here." Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] What algorithm is R using to calculate mean?
This was also asked on StackOverflow:
http://stackoverflow.com/q/17866149/271616. Here is the answer I
posted:
This appears to be the updating method of West, 1979 [1] and it was
implemented in R-2.3.0 in response to PR#1228 [2].
I'm not positive this is the correct algorithm, since it was suggested
by Martin Maechler but implemented by Brian Ripley. I couldn't find a
reference in the source code or version control logs that listed the
actual algorithm used. It was implemented in cov.c in revision 37389
and in summary.c in revision 37393.
[1] http://dl.acm.org/citation.cfm?doid=359146.359153
[2] https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=1228
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Thu, Jul 25, 2013 at 2:44 PM, Zach Harrington
wrote:
> I am curious to know what algorithm R's mean function uses. Is there some
> reference to the numerical properties of this algorithm?
>
> I found the following C code in summary.c:do_summary():
> case REALSXP:
> PROTECT(ans = allocVector(REALSXP, 1));
> for (i = 0; i < n; i++) s += REAL(x)[i];
> s /= n;
> if(R_FINITE((double)s)) {
> for (i = 0; i < n; i++) t += (REAL(x)[i] - s);
> s += t/n;
> }
> REAL(ans)[0] = s;
> break;
>
> It seems to do a straight up mean:
> for (i = 0; i < n; i++) s += REAL(x)[i];
> s /= n;
>
> Then it adds what i assume is a numerical correction which seems to be the
> mean difference from the mean of the data:
> for (i = 0; i < n; i++) t += (REAL(x)[i] - s);
> s += t/n;
>
> I haven't been able to track this algorithm down anywhere (mean is not a
> great search term).
>
> Any help would be much appreciated,
>
> Zach Harrington
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] as.Date.POSIXct
Would it make sense for as.Date.POSIXct to not assume tz="UTC" if the
POSIXct object has a valid tzone attribute? Current behavior may be
confusing in certain cases, for example:
> (d <- structure(1090450800, tzone="Europe/Berlin",
+ class=c("POSIXct","POSIXt")))
[1] "2004-07-22 01:00:00 CEST"
> as.Date(d)
[1] "2004-07-21"
> as.Date(as.POSIXlt(d))
[1] "2004-07-22"
> as.Date(d, tz=attr(d,'tzone'))
[1] "2004-07-22"
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2013: Applied Finance with R | www.RinFinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Matrix does not build with R trunk since Oct.
roblem - "big" as in >>> "wide-spread". >>> >> >> You are yet to show that this is a problem in R at all. You failed to follow >> the basic instructions in the FAQ. >> >> Cheers, >> Simon >> >> >> >>>> Cheers, >>>> Simon >>>> >>>> >>>>> >>>>> >>>>> Loading required package: Matrix >>>>> Error in namespaceExport(ns, exports) : undefined >>>> exports: .M.classEnv >>>>> Error : require(Matrix) is not TRUE >>>>> ERROR: installing package indices failed >>>>> * removing ‘/svn-loc/R/library/Matrix’ >>>>> * restoring previous ‘/svn-loc/R/library/Matrix’ >>>>> make[2]: *** [Matrix.ts] Error 1 >>>>> make[2]: Leaving directory >>>> `/svn-loc/R/src/library/Recommended' >>>>> make[1]: *** [recommended-packages] Error 2 >>>>> make[1]: Leaving directory >>>> `/svn-loc/R/src/library/Recommended' >>>>> make: *** [stamp-recommended] Error 2 >>>>> >>>>> >>>>> If it matters, here is what r trunk built with: >>>>> ./configure --enable-memory-profiling >>>> --enable-strict-barrier --enable-byte-compiled-packages >>>> --with-valgrind-instrumentation=2 --enable-lto >>>>> >>>>> __ >>>>> R-devel@r-project.org >>>> mailing list >>>>> https://stat.ethz.ch/mailman/listinfo/r-devel >>>> >>>> >>> >>> >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2013: Applied Finance with R | www.RinFinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Who does develop the R core and libs, and how / where is it hosted?
On Mon, Jan 14, 2013 at 2:29 PM, oliver wrote: > Hello, > > I saw Binaries, stable release-souzrces and daily snapshots of R, but > not something like a repository, visible for the public (like on githb for > example). > Go to http://www.r-project.org and click the "Developer Page" link in the left-hand column. > How is the R development handled, what repositories / source code versioning > tools > are used, who are the developers? > Ibid, and Go to http://www.r-project.org and click the "Contributors" link in the left-hand column. > And is there something like a plan with future goals, > which are planned for the next releases? > > Are there areas, where help is needed? > And in which way could support be done? > > > Ciao, >Oliver > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] scientific notation and comparison with character variable
It's expected. From ?"<": If the two arguments are atomic vectors of different types, one is coerced to the type of the other, the (decreasing) order of precedence being character, complex, numeric, integer, logical and raw. > as.character(1e-2) < 0.05 [1] TRUE > as.character(1e-4) < 0.05 [1] FALSE Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Wed, Jan 2, 2013 at 12:38 PM, Tobias Verbeke wrote: > L.S. > > Is the following expected and/or documented? > >> 1e-2 < "0.05" > [1] TRUE >> 1e-4 < "0.05" > [1] FALSE > > Many thanks in advance for any pointer. > > Best, > Tobias > >> sessionInfo() > R Under development (unstable) (2013-01-01 r61512) > Platform: i386-w64-mingw32/i386 (32-bit) > > locale: > [1] LC_COLLATE=English_United States.1252 > [2] LC_CTYPE=English_United States.1252 > [3] LC_MONETARY=English_United States.1252 > [4] LC_NUMERIC=C > [5] LC_TIME=English_United States.1252 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Anybody have a WABAC machine?
grep -E "packageStartupMessage|print0|sprintf" *NEWS You need *NEWS to ensure you search NEWS, ONEWS, and OONEWS. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Fri, Aug 24, 2012 at 1:16 PM, Roebuck,Paul L wrote: > What version of R did the following functions make their debut? > > packageStartupMesssage > paste0 > sprintf > > Need to update some old packages... > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Suggestion for "Writing R Extensions"
On Wed, Aug 22, 2012 at 2:58 PM, Oliver Bandel wrote: > Hello, > > > Zitat von "Joshua Ulrich" (Wed, 22 Aug 2012 > 12:35:51 -0500) > > [...] > >> Would it make sense to add links and/or mention the relevant >> appendices (A and D) of "R Installation and Administration"? > > [...] > > I do not understad, what your question is all about. > > Maybe you should be more verbose. > > Do you want to make the document better, > or do you have questions on how to understand > what it talks about? > The sentences I quoted, which you removed from the thread, point the reader to "R Installation and Administration". Rather than point the reader to the entire manual, I suggest pointing them to the relevant appendices (A and D) of "R Installation and Administration". > > Ciao, >Oliver > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Suggestion for "Writing R Extensions"
Hi, Early in Chapter 1, the manual states: "A computing environment including a number of tools is assumed; the "R Installation and Administration" manual describes what is needed. Under a Unix-alike most of the tools are likely to be present by default, but Microsoft Windows may require careful setup." Would it make sense to add links and/or mention the relevant appendices (A and D) of "R Installation and Administration"? Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Quiz: How to get a "named column" from a data frame
On Tue, Aug 21, 2012 at 2:34 PM, Duncan Murdoch wrote: > On 12-08-18 12:33 PM, Martin Maechler wrote: >>>>>>> >>>>>>> Joshua Ulrich >>>>>>> on Sat, 18 Aug 2012 10:16:09 -0500 writes: >> >> >> > I don't know if this is better, but it's the most obvious/shortest >> I >> > could come up with. Transpose the data.frame column to a 'row' >> vector >> > and drop the dimensions. >> >> R> identical(nv, drop(t(df))) >> > [1] TRUE >> >> Yes, that's definitely shorter, >> congratulations! >> >> One gotta is that I'd want a solution that also works when the >> df has more columns than just one... >> >> Your idea to use t(.) is nice and "perfect" insofar as it >> coerces the data frame to a matrix, and that's really the clue: >> >> Where as df[,1] is losing the names, >> the matrix indexing is not. >> So your solution can be changed into >> >> t(df)[1,] >> >> which is even shorter... >> and slightly less efficient, at least conceptually, than mine, which has >> been >> >> as.matrix(df)[,1] >> >> Now, the remaining question is: Shouldn't there be something >> more natural to achieve that? >> (There is not, currently, AFAIK). > > > I've been offline, so I'm a bit late to this game, but the examples above > fail when df contains a character column as well as the desired one, because > everything gets coerced to a character matrix. You need to select the > column first, then convert to a matrix, e.g. > > drop(t(df[,1,drop=FALSE])) > That's true, but I was assuming a one-column data.frame, which can be achieved via: df <- data.frame(VAR=nv,CHAR=letters[1:3],stringsAsFactors=FALSE) drop(t(df[1])) That said, I prefer the setNames() solution for its efficiency. Best, Josh > Duncan Murdoch > > >> >> Martin >> >> >> > Best, >> > -- >> > Joshua Ulrich | about.me/joshuaulrich >> > FOSS Trading | www.fosstrading.com >> >> >> > On Sat, Aug 18, 2012 at 10:03 AM, Martin Maechler >> > wrote: >> >> Today, I was looking for an elegant (and efficient) way to get a >> named >> >> (atomic) vector by selecting one column of a data frame. Of >> course, >> >> the vector names must be the rownames of the data frame. >> >> >> >> Ok, here is the quiz, I know one quite "cute"/"slick" answer, but >> was >> >> wondering if there are obvious better ones, and also if this >> should >> >> not become more idiomatic (hence "R-devel"): >> >> >> >> Consider this toy example, where the dataframe already has only >> one >> >> column : >> >> >> >>> nv <- c(a=1, d=17, e=101); nv >> >> a d e >> >> 1 17 101 >> >> >> >>> df <- as.data.frame(cbind(VAR = nv)); df >> >> VAR >> >> a 1 >> >> d 17 >> >> e 101 >> >> >> >> Now how, can I get 'nv' back from 'df' ? I.e., how to get >> >> >> >>> identical(nv, ...) >> >> [1] TRUE >> >> >> >> where .. only uses 'df' (and no non-standard R packages)? >> >> >> >> As said, I know a simple solution (*), but I'm sure it is not >> >> obvious to most R users and probably not even to the majority of >> >> R-devel readers... OTOH, people like Bill Dunlap will not take >> >> long to provide it or a better one. >> >> >> >> (*) In my solution, the above '...' consists of 17 letters. >> >> I'll post it later today (CEST time) ... or confirm >> >> that someone else has done so. >> >> >> >> Martin >> >> >> >> __ >> >> R-devel@r-project.org mailing list >> >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Quiz: How to get a "named column" from a data frame
I don't know if this is better, but it's the most obvious/shortest I could come up with. Transpose the data.frame column to a 'row' vector and drop the dimensions. R> identical(nv, drop(t(df))) [1] TRUE Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sat, Aug 18, 2012 at 10:03 AM, Martin Maechler wrote: > Today, I was looking for an elegant (and efficient) way > to get a named (atomic) vector by selecting one column of a data frame. > Of course, the vector names must be the rownames of the data frame. > > Ok, here is the quiz, I know one quite "cute"/"slick" answer, but was > wondering if there are obvious better ones, and > also if this should not become more idiomatic (hence "R-devel"): > > Consider this toy example, where the dataframe already has only > one column : > >> nv <- c(a=1, d=17, e=101); nv > a d e > 1 17 101 > >> df <- as.data.frame(cbind(VAR = nv)); df > VAR > a 1 > d 17 > e 101 > > Now how, can I get 'nv' back from 'df' ? I.e., how to get > >> identical(nv, ...) > [1] TRUE > > where .. only uses 'df' (and no non-standard R packages)? > > As said, I know a simple solution (*), but I'm sure it is not > obvious to most R users and probably not even to the majority of > R-devel readers... OTOH, people like Bill Dunlap will not take > long to provide it or a better one. > > (*) In my solution, the above '...' consists of 17 letters. > I'll post it later today (CEST time) ... or confirm > that someone else has done so. > > Martin > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Reading many large files causes R to crash - Possible Bug in R 2.15.1 64-bit Ubuntu
David, Thank you for providing something reproducible. This line: templateTimes <- as.xts(templateTimes) creates a zero-width xts object (i.e. the coredata is a zero-length vector, but there is a non-zero-length index). So, the to.period(templateTimes) call returns OHLC data of random memory locations. This is the likely cause of the segfaults. Since aggregating "no data" doesn't make sense, I have patched to.period to throw an error when run on zero-width/length objects (revision 690 on R-Forge). The attached file works with the CRAN version of xts because it avoids the issue entirely. Your script will still "hang" on the BAC_0.csv file because as.character.POSIXt can take a long time. Better to just call format() directly (as I do in the attached file). If you have any follow-up questions, please send them to R-SIG-Finance. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Mon, Jul 23, 2012 at 8:41 AM, David Terk wrote: > I'm attaching a runnable script and corresponding data files. This will > freeze at 83%. > > I'm not sure how much simpler to get than this. > > -Original Message- > From: Joshua Ulrich [mailto:josh.m.ulr...@gmail.com] > Sent: Monday, July 23, 2012 9:17 AM > To: David Terk > Cc: Duncan Murdoch; r-devel@r-project.org > Subject: Re: [Rd] Reading many large files causes R to crash - Possible Bug > in R 2.15.1 64-bit Ubuntu > > Well, you still haven't convinced anyone but yourself that it's definitely > an xts problem, since you have not provided any reproducible example... > -- > Joshua Ulrich | about.me/joshuaulrich > FOSS Trading | www.fosstrading.com > > > On Mon, Jul 23, 2012 at 8:14 AM, David Terk wrote: >> Where should this be discussed since it is definitely XTS related? I >> will gladly upload the simplified script + data files to whoever is >> maintaining this part of the code. Fortunately there is a workaround > here. >> >> -Original Message- >> From: Joshua Ulrich [mailto:josh.m.ulr...@gmail.com] >> Sent: Monday, July 23, 2012 8:15 AM >> To: David Terk >> Cc: Duncan Murdoch; r-devel@r-project.org >> Subject: Re: [Rd] Reading many large files causes R to crash - >> Possible Bug in R 2.15.1 64-bit Ubuntu >> >> David, >> >> You still haven't provided a reproducible example. As Duncan already >> said, "if you don't post code that allows us to reproduce the crash, >> it's really unlikely that we'll be able to fix it." >> >> And R-devel is not the appropriate venue to discuss this if it's truly >> an issue with xts/zoo. >> >> Best, >> -- >> Joshua Ulrich | about.me/joshuaulrich FOSS Trading | >> www.fosstrading.com >> >> >> On Mon, Jul 23, 2012 at 12:41 AM, David Terk wrote: >>> Looks like the call to: >>> >>> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL) >>> >>> If what is causing the issue. If variable name is not set, or set to >>> any value other than NULL. Than no hang occurs. >>> >>> -Original Message- >>> From: David Terk [mailto:david.t...@gmail.com] >>> Sent: Monday, July 23, 2012 1:25 AM >>> To: 'Duncan Murdoch' >>> Cc: 'r-devel@r-project.org' >>> Subject: RE: [Rd] Reading many large files causes R to crash - >>> Possible Bug in R 2.15.1 64-bit Ubuntu >>> >>> I've isolated the bug. When the seg fault was produced there was an >>> error that memory had not been mapped. Here is the odd part of the >>> bug. If you comment out certain code and get a full run than comment >>> in >> the code which >>> is causing the problem it will actually run. So I think it is safe to >>> assume something wrong is taking place with memory allocation. Example. >>> While testing, I have been able to get to a point where the code will > run. >>> But if I reboot the machine and try again, the code will not run. >>> >>> The bug itself is happening somewhere in XTS or ZOO. I will gladly >>> upload the data files. It is happening on the 10th data file which >>> is only 225k lines in size. >>> >>> Below is the simplified code. The call to either >>> >>> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL) >>> index(dat.i) <- index(to.period(templateTimes, period=per, k=subper)) >>> >>> is what is causing R to hang or crash. I have been able to replicate >>> this on Windows 7 64 bit and Ubuntu 64 bit. Seems easiest to &g
Re: [Rd] Reading many large files causes R to crash - Possible Bug in R 2.15.1 64-bit Ubuntu
Well, you still haven't convinced anyone but yourself that it's
definitely an xts problem, since you have not provided any
reproducible example...
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Mon, Jul 23, 2012 at 8:14 AM, David Terk wrote:
> Where should this be discussed since it is definitely XTS related? I will
> gladly upload the simplified script + data files to whoever is maintaining
> this part of the code. Fortunately there is a workaround here.
>
> -Original Message-----
> From: Joshua Ulrich [mailto:josh.m.ulr...@gmail.com]
> Sent: Monday, July 23, 2012 8:15 AM
> To: David Terk
> Cc: Duncan Murdoch; r-devel@r-project.org
> Subject: Re: [Rd] Reading many large files causes R to crash - Possible Bug
> in R 2.15.1 64-bit Ubuntu
>
> David,
>
> You still haven't provided a reproducible example. As Duncan already said,
> "if you don't post code that allows us to reproduce the crash, it's really
> unlikely that we'll be able to fix it."
>
> And R-devel is not the appropriate venue to discuss this if it's truly an
> issue with xts/zoo.
>
> Best,
> --
> Joshua Ulrich | about.me/joshuaulrich
> FOSS Trading | www.fosstrading.com
>
>
> On Mon, Jul 23, 2012 at 12:41 AM, David Terk wrote:
>> Looks like the call to:
>>
>> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
>>
>> If what is causing the issue. If variable name is not set, or set to
>> any value other than NULL. Than no hang occurs.
>>
>> -Original Message-
>> From: David Terk [mailto:david.t...@gmail.com]
>> Sent: Monday, July 23, 2012 1:25 AM
>> To: 'Duncan Murdoch'
>> Cc: 'r-devel@r-project.org'
>> Subject: RE: [Rd] Reading many large files causes R to crash -
>> Possible Bug in R 2.15.1 64-bit Ubuntu
>>
>> I've isolated the bug. When the seg fault was produced there was an
>> error that memory had not been mapped. Here is the odd part of the
>> bug. If you comment out certain code and get a full run than comment in
> the code which
>> is causing the problem it will actually run. So I think it is safe to
>> assume something wrong is taking place with memory allocation. Example.
>> While testing, I have been able to get to a point where the code will run.
>> But if I reboot the machine and try again, the code will not run.
>>
>> The bug itself is happening somewhere in XTS or ZOO. I will gladly
>> upload the data files. It is happening on the 10th data file which is
>> only 225k lines in size.
>>
>> Below is the simplified code. The call to either
>>
>> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
>> index(dat.i) <- index(to.period(templateTimes, period=per, k=subper))
>>
>> is what is causing R to hang or crash. I have been able to replicate
>> this on Windows 7 64 bit and Ubuntu 64 bit. Seems easiest to
>> consistently replicate from R Studio.
>>
>> The code below will consistently replicate when the appropriate files
>> are used.
>>
>> parseTickDataFromDir = function(tickerDir, per, subper) {
>> tickerAbsFilenames = list.files(tickerDir,full.names=T)
>> tickerNames = list.files(tickerDir,full.names=F)
>> tickerNames = gsub("_[a-zA-Z0-9].csv","",tickerNames)
>> pb <- txtProgressBar(min = 0, max = length(tickerAbsFilenames),
>> style = 3)
>>
>> for(i in 1:length(tickerAbsFilenames)) {
>> dat.i = parseTickData(tickerAbsFilenames[i])
>> dates <- unique(substr(as.character(index(dat.i)), 1,10))
>> times <- rep("09:30:00", length(dates))
>> openDateTimes <- strptime(paste(dates, times), "%F %H:%M:%S")
>> templateTimes <- NULL
>>
>> for (j in 1:length(openDateTimes)) {
>> if (is.null(templateTimes)) {
>> templateTimes <- openDateTimes[j] + 0:23400
>> } else {
>> templateTimes <- c(templateTimes, openDateTimes[j] + 0:23400)
>> }
>> }
>>
>> templateTimes <- as.xts(templateTimes)
>> dat.i <- merge(dat.i, templateTimes, all=T)
>> if (is.na(dat.i[1])) {
>> dat.i[1] <- -1
>> }
>> dat.i <- na.locf(dat.i)
>> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
>> index(dat.i) <- index(to.period(templateTimes, period=per,
>> k=subper))
>> setTxtProgressBar(pb, i)
>> }
>> close(pb)
>> }
>>
>> parseTickData <- function(inputFile) {
>> DAT.li
Re: [Rd] Reading many large files causes R to crash - Possible Bug in R 2.15.1 64-bit Ubuntu
David,
You still haven't provided a reproducible example. As Duncan already
said, "if you don't post code that allows us to reproduce the crash,
it's really unlikely that we'll be able to fix it."
And R-devel is not the appropriate venue to discuss this if it's truly
an issue with xts/zoo.
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Mon, Jul 23, 2012 at 12:41 AM, David Terk wrote:
> Looks like the call to:
>
> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
>
> If what is causing the issue. If variable name is not set, or set to any
> value other than NULL. Than no hang occurs.
>
> -Original Message-
> From: David Terk [mailto:david.t...@gmail.com]
> Sent: Monday, July 23, 2012 1:25 AM
> To: 'Duncan Murdoch'
> Cc: 'r-devel@r-project.org'
> Subject: RE: [Rd] Reading many large files causes R to crash - Possible Bug
> in R 2.15.1 64-bit Ubuntu
>
> I've isolated the bug. When the seg fault was produced there was an error
> that memory had not been mapped. Here is the odd part of the bug. If you
> comment out certain code and get a full run than comment in the code which
> is causing the problem it will actually run. So I think it is safe to
> assume something wrong is taking place with memory allocation. Example.
> While testing, I have been able to get to a point where the code will run.
> But if I reboot the machine and try again, the code will not run.
>
> The bug itself is happening somewhere in XTS or ZOO. I will gladly upload
> the data files. It is happening on the 10th data file which is only 225k
> lines in size.
>
> Below is the simplified code. The call to either
>
> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
> index(dat.i) <- index(to.period(templateTimes, period=per, k=subper))
>
> is what is causing R to hang or crash. I have been able to replicate this
> on Windows 7 64 bit and Ubuntu 64 bit. Seems easiest to consistently
> replicate from R Studio.
>
> The code below will consistently replicate when the appropriate files are
> used.
>
> parseTickDataFromDir = function(tickerDir, per, subper) {
> tickerAbsFilenames = list.files(tickerDir,full.names=T)
> tickerNames = list.files(tickerDir,full.names=F)
> tickerNames = gsub("_[a-zA-Z0-9].csv","",tickerNames)
> pb <- txtProgressBar(min = 0, max = length(tickerAbsFilenames), style = 3)
>
> for(i in 1:length(tickerAbsFilenames)) {
> dat.i = parseTickData(tickerAbsFilenames[i])
> dates <- unique(substr(as.character(index(dat.i)), 1,10))
> times <- rep("09:30:00", length(dates))
> openDateTimes <- strptime(paste(dates, times), "%F %H:%M:%S")
> templateTimes <- NULL
>
> for (j in 1:length(openDateTimes)) {
> if (is.null(templateTimes)) {
> templateTimes <- openDateTimes[j] + 0:23400
> } else {
> templateTimes <- c(templateTimes, openDateTimes[j] + 0:23400)
> }
> }
>
> templateTimes <- as.xts(templateTimes)
> dat.i <- merge(dat.i, templateTimes, all=T)
> if (is.na(dat.i[1])) {
> dat.i[1] <- -1
> }
> dat.i <- na.locf(dat.i)
> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
> index(dat.i) <- index(to.period(templateTimes, period=per,
> k=subper))
> setTxtProgressBar(pb, i)
> }
> close(pb)
> }
>
> parseTickData <- function(inputFile) {
> DAT.list <- scan(file=inputFile,
> sep=",",skip=1,what=list(Date="",Time="",Close=0,Volume=0),quiet=T)
> index <- as.POSIXct(paste(DAT.list$Date,DAT.list$Time),format="%m/%d/%Y
> %H:%M:%S")
> DAT.xts <- xts(DAT.list$Close,index)
> DAT.xts <- make.index.unique(DAT.xts)
> return(DAT.xts)
> }
>
> DATTick <- parseTickDataFromDir(tickerDirSecond, "seconds",10)
>
> -Original Message-
> From: Duncan Murdoch [mailto:murdoch.dun...@gmail.com]
> Sent: Sunday, July 22, 2012 4:48 PM
> To: David Terk
> Cc: r-devel@r-project.org
> Subject: Re: [Rd] Reading many large files causes R to crash - Possible Bug
> in R 2.15.1 64-bit Ubuntu
>
> On 12-07-22 3:54 PM, David Terk wrote:
>> I am reading several hundred files. Anywhere from 50k-400k in size.
>> It appears that when I read these files with R 2.15.1 the process will
>> hang or seg fault on the scan() call. This does not happen on R 2.14.1.
>
> The code below doesn't do anything other than define a couple of functions.
> Please simplify it to code that creates a file (or multiple files), reads it
> or them, and s
Re: [Rd] Reading many large files causes R to crash - Possible Bug in R 2.15.1 64-bit Ubuntu
Cross-posted on Stack Overflow:
http://stackoverflow.com/q/11596747/271616
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Sun, Jul 22, 2012 at 2:54 PM, David Terk wrote:
> I am reading several hundred files. Anywhere from 50k-400k in size. It
> appears that when I read these files with R 2.15.1 the process will hang or
> seg fault on the scan() call. This does not happen on R 2.14.1.
>
>
>
> This is happening on the precise build of Ubuntu.
>
>
>
> I have included everything, but the issue appears to be when performing the
> scan in the method parseTickData.
>
>
>
> Below is the code. Hopefully this is the right place to post.
>
>
>
> parseTickDataFromDir = function(tickerDir, per, subper, fun) {
>
> tickerAbsFilenames = list.files(tickerDir,full.names=T)
>
> tickerNames = list.files(tickerDir,full.names=F)
>
> tickerNames = gsub("_[a-zA-Z0-9].csv","",tickerNames)
>
> pb <- txtProgressBar(min = 0, max = length(tickerAbsFilenames), style = 3)
>
>
>
> for(i in 1:length(tickerAbsFilenames)) {
>
>
>
> # Grab Raw Tick Data
>
> dat.i = parseTickData(tickerAbsFilenames[i])
>
> #Sys.sleep(1)
>
> # Create Template
>
> dates <- unique(substr(as.character(index(dat.i)), 1,10))
>
> times <- rep("09:30:00", length(dates))
>
> openDateTimes <- strptime(paste(dates, times), "%F %H:%M:%S")
>
> templateTimes <- NULL
>
>
>
> for (j in 1:length(openDateTimes)) {
>
> if (is.null(templateTimes)) {
>
> templateTimes <- openDateTimes[j] + 0:23400
>
> } else {
>
> templateTimes <- c(templateTimes, openDateTimes[j] + 0:23400)
>
> }
>
> }
>
>
>
> # Convert templateTimes to XTS, merge with data and convert NA's
>
> templateTimes <- as.xts(templateTimes)
>
> dat.i <- merge(dat.i, templateTimes, all=T)
>
> # If there is no data in the first print, we will have leading NA's. So
> set them to -1.
>
> # Since we do not want these values removed by to.period
>
> if (is.na(dat.i[1])) {
>
> dat.i[1] <- -1
>
> }
>
> # Fix remaining NA's
>
> dat.i <- na.locf(dat.i)
>
> # Convert to desired bucket size
>
> dat.i <- to.period(dat.i, period=per, k=subper, name=NULL)
>
> # Always use templated index, otherwise merge fails with other symbols
>
> index(dat.i) <- index(to.period(templateTimes, period=per, k=subper))
>
> # If there was missing data at open, set close to NA
>
> valsToChange <- which(dat.i[,"Open"] == -1)
>
> if (length(valsToChange) != 0) {
>
> dat.i[valsToChange, "Close"] <- NA
>
> }
>
> if(i == 1) {
>
> DAT = fun(dat.i)
>
> } else {
>
> DAT = merge(DAT,fun(dat.i))
>
> }
>
> setTxtProgressBar(pb, i)
>
> }
>
> close(pb)
>
> colnames(DAT) = tickerNames
>
> return(DAT)
>
> }
>
>
>
> parseTickData <- function(inputFile) {
>
> DAT.list <- scan(file=inputFile,
> sep=",",skip=1,what=list(Date="",Time="",Close=0,Volume=0),quiet=T)
>
> index <- as.POSIXct(paste(DAT.list$Date,DAT.list$Time),format="%m/%d/%Y
> %H:%M:%S")
>
> DAT.xts <- xts(DAT.list$Close,index)
>
> DAT.xts <- make.index.unique(DAT.xts)
>
> return(DAT.xts)
>
> }
>
>
>
>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problem in compililng c code using R CMD SHLIB
On Wed, Jun 6, 2012 at 8:20 AM, Simon Urbanek wrote: > > On Jun 6, 2012, at 6:16 AM, Arathy Ram wrote: > >> Hello all, >> I was trying to compile a simple C program hello.c using R CMD SHLIB >> hello.c. >> MY R software residing in C:\Program Files\R\R-2.15.0, and also I have >> downloaded the Rtools 2.15.0 and it is in C:\Rtools, MY file(hello.c) is in >> E:\R_dir. >> I have followed the procedures given below >> 1. In a new CMD.exe window I have set the path as >> >> c:\ Rtools\bin; C:\Rtools\MinGW\bin;C:\Program Files\R\R-2.15.0\bin >> >> 2. I have changed the directory to E:\ where my file(hello.c) is residing >> E:\>cd \R_dir >> then I have given the command like this >> E:\>cd \R_dir> R CMD SHLIB hello.c >> Then I am getting the following message >> cygwin warning: >> MS-DOS style path detected: E:/PROGRA~1/R/R-214~1.0/etc/ >> i386/Makeconf >> Preferred POSIX equivalent is: >> /cygdrive/e/PROGRA~1/R/R-214~1.0/etc/i386/Makeconf >> CYGWIN environment variable option "nodosfilewarning" turns off this >> warning. >> Consult the user's guide for more details about POSIX paths: >> http://cygwin.com/cygwin-ug-net/using.html#using-pathnames >> Even though I have gone through several discussions in the same topic I am >> not able to resolve my problem. It would be a great help if somebody knows >> what exactly the problem and its solution and favorable suggestions. > > There is no problem in what you posted - all you posted is merely a note from > cygwin that it prefers different path style, but it's perfectly ok and > doesn't affect anything. So what exactly is the problem you're having? > Probably has something to do with PATH not being set correctly because the Cygwin warning points to R-2.14.0 on E:, not R-2.15.0 on C:. > Cheers, > Simon > Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Patch to add Beta binomial distribution. Mentor needed!
On Tue, May 22, 2012 at 8:27 AM, Joan Maspons wrote: > Hello, > > 2012/5/22 Christophe Dutang > >> Dear Joan, >> >> If you want to contribute to R, you should write a package and submit it >> to CRAN. See http://cran.r-project.org/doc/manuals/R-exts.html >> > > It's necessary to develop a package before to add the functions to R-base? > I just want to add beta binomial and beta negative binomial distributions > which from my point of view are not more strange than distributions > included in R-base. I think they are quite common in order to simulate a > binomial or negative binomial process in an stochastic environment. > In order to add functions to the main R distribution, you have to convince someone on R-core to maintain the code. > In the other hand I have no experience in writing packages so it would take > me more time. > The actuar package contains many additional distributions. You might see if the maintainer(s) would be interested in incorporating your code (if they don't have that functionality already). > Yours, > Joan > > Regards >> >> Christophe >> >> -- >> Christophe Dutang >> Ph.D. student at ISFA, Lyon, France >> website: http://dutangc.free.fr >> >> Le 22 mai 2012 à 12:14, Joan Maspons a écrit : >> >> > Hello, >> > >> > I implemented the Beta binomial distribution following the patterns of >> the >> > binomial distribution code and inspired by JAGS' code [1]. I have studied >> > the code carefully but it's my first run in the R internals. >> > >> > Can somebody review the code and if everything it's ok commit to the >> > repository? >> > >> > [1] >> > >> http://mcmc-jags.hg.sourceforge.net/hgweb/mcmc-jags/mcmc-jags/file/15af65a4be29/src/modules/bugs/distributions/DBetaBin.cc >> > >> > >> > -- >> > Joan Maspons >> > CREAF (Centre de Recerca Ecològica i Aplicacions Forestals) >> > Universitat Autònoma de Barcelona, 08193 Bellaterra (Barcelona), >> Catalonia >> > Tel +34 93 581 2915 j.masp...@creaf.uab.cat >> > http://www.creaf.uab.cat >> > __ >> > R-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > > > -- > Joan Maspons > CREAF (Centre de Recerca Ecològica i Aplicacions Forestals) > Universitat Autònoma de Barcelona, 08193 Bellaterra (Barcelona), Catalonia > Tel +34 93 581 2915 j.masp...@creaf.uab.cat > http://www.creaf.uab.cat > > [[alternative HTML version deleted]] > > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] unlist crashes 32-bit R on WinXP when use.names=TRUE
Hi all, I experienced a crash in R-2.15.0 on 32-bit Windows XP (sessionInfo below) when running the piece of code below. I cannot replicate the error on 64-bit Linux, 64-bit Windows, or 32-bit R running under 64-bit Windows. I do not have, and could not find, a 32-bit version of Linux to test this. > NOW <- Sys.time() > FUTURE <- NOW+1:1e7 > crash <- as.character(FUTURE) Error in unlist(unclass(x)[1L:3L]) : promise already under evaluation: recursive default argument reference or earlier problems? > traceback() Error: C stack usage is too close to the limit > # evaluating an expression at this point would cause R to exit ungracefully Here's an example that avoids a lot of unnecessary code: L1 <- list(one=1:1e6, two=1:1e6, three=1:1e6) # no issue with smaller list elements U1 <- unlist(L1, recursive=TRUE, use.names=TRUE) C1 <- c(L1, recursive=TRUE, use.names=TRUE) L2 <- list(one=1:1e7, two=1:1e7, three=1:1e7) # crashes after ~2min with error above U2 <- unlist(L2, recursive=TRUE, use.names=TRUE) C2 <- c(L2, recursive=TRUE, use.names=TRUE) # no issue if use.names=FALSE U3 <- unlist(L2, recursive=TRUE, use.names=FALSE) C3 <- c(L2, recursive=TRUE, use.names=FALSE) I took a look at do_unlist and do_c_dflt in bind.c, but I stopped at NewExtractNames because it is a bit beyond my current understanding. Any thoughts? > sessionInfo() R version 2.15.0 (2012-03-30) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base Please let me know if I forgot anything or if there's anything I can do to help. Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com R/Finance 2012: Applied Finance with R www.RinFinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] url, readLines, source behind a proxy
Hi Renaud, On Wed, Apr 18, 2012 at 12:22 AM, Renaud Gaujoux wrote: > Hi Henrik, > > > Could anybody behind a proxy check if the issue can be reproduced? > My proxy is in fact provided by cntml, which acts as a local proxy that > takes care of tricky authentication protocols with the actual university > proxy, not natively supported by my system (Ubuntu). Anybody in this case? > I can replicate this on a WinXP system, where I normally have to use the --internet2 flag to get internet access through a proxy. ?download.file has a section on "Setting Proxies", which describes how to use environment variables to set proxy information. Setting http_proxy='http://my.proxy.com/' was enough for me to get R CMD check to run successfully with the --as-cran flag. > Thanks. > Renaud > Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com R/Finance 2012: Applied Finance with R www.RinFinance.com > On Tue, 17 Apr 2012, Henrik Bengtsson wrote: > >> On Tue, Apr 17, 2012 at 1:01 AM, Renaud Gaujoux >> wrote: >> > Hi, >> > >> > when I run R CMD check with flag --as-cran, the process hangs at stage: >> > >> > * checking CRAN incoming feasibility ... >> >> Doesn't it time-out eventually? I'm not behind a proxy but when I've >> been running 'R CMD check' whenon very poor 3G connection, it had >> eventually timed out. >> >> /Henrik >> >> > >> > I am pretty sure it is a proxy issue. >> > I looked at the check code in the tools package and it seems that the issue >> > is in the local function `.repository_db()` (defined in >> > `tools:::.check_package_CRAN_incoming()`), which eventually calls `url()` >> > with argument open="rb", that hangs probably because it does not use the >> > proxy settings. >> > I had a similar issue with `source()`, which apparently uses internal >> > network functions (not as download.file), but is supposed to work behind a >> > proxy (correct?). >> > Does anybody else have this problem? >> > >> > I was wondering if there is a way around, as I would like to be able to use >> > --as-cran for my checks. >> > Thank you. >> > >> > Renaud >> > >> > -- >> > Renaud Gaujoux >> > Computational Biology - University of Cape Town >> > South Africa >> > >> > __ >> > R-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Smalltalk binding to R
Hi Francisco, R-devel probably isn't the best venue for this. If you're looking for a GSoC student/mentor and you visit the R Project webpage on GSoC's site: http://www.google-melange.com/gsoc/org/google/gsoc2012/rproject you will see a link to the "Ideas page >>": http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2012&s=gsoc which says you should have sent this email to the GSoC-R Google group. Please join and continue the discussion there. I have already cc'd the GSoC-R group. Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com R/Finance 2012: Applied Finance with R www.RinFinance.com On Sat, Mar 24, 2012 at 12:41 PM, Francisco Garau wrote: > Hi - any person interested in building a binding between Smalltalk and R, > please contact myself or Hernan. > > This could be done as the Google Summer of Code (GSoC). > > http://www.google-melange.com/gsoc/homepage/google/gsoc2012 > http://gsoc2012.esug.org/projects/r-statistics > > For those who don't know Smalltalk it is an Object Oriented language that > was created back in the '80s in the Xerox Parc research center. Although it > never achieved high popularity it remains strong in some niche areas and > growing in popularity thanks to a very cool Web development framework. > > Smalltalk has only one disadvantage: once you try it, you don't want to > leave it! > > Please, join us in helping connect the world of Smalltalk and all the > amazing R libraries. > > - Francisco > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] How to install sqldf to R with version 2.10?
On Sat, Mar 17, 2012 at 9:56 AM, zhu free wrote: > Hi, > How to install sqldf to R with version 2.10? I used the R in the cluster of > university and there seems no way to update the R version to 2.14. However, > I do need sqldf. I tried to install and there is the problem of namespace. > How could I solve the problem of namespace and run sqldf on R with version > 2.10? Thank you! > Download and unpack the sqldf source, add a NAMESPACE file, then build the Windows binary using R-2.10.x and Rtools210 or Rtools211. All these steps are documented throughout the "Writing R Extensions" manual. http://cran.r-project.org/doc/manuals/R-exts.html HTH, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com R/Finance 2012: Applied Finance with R www.RinFinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problems with iconv
On Wed, Dec 14, 2011 at 8:15 AM, RogerP wrote: > For some reason when I click on reply and sign-in - the message I'm replying > to is not carried forward. As you requested that include your message I > copied and pasted it from my email. > I guess it's some Nabble nonsense then... >> A suitably comprehensive iconv function is essential. The R usage requires >> iconv to be able to translate between "latin1" and "UTF-8", to recognize >> "" >> (as the current encoding) and "ASCII", and to translate to and from the >> Unicode wide-character formats "UCS-[24][BL]E" — this is true for glibc >> but >> not of most commercial Unixes. However, you can make use of GNU libiconv >> (possibly as a plug-in replacement: see >> http://www.gnu.org/software/libiconv/). >> >> Well, that's just what I did. I downloaded libiconv and compiled and >> linked >> it. Oh, so where to put it. H, don't see it anywhere here. Do you? >> > The location isn't mentioned in Appendix A, which you cite a portion > of above, but it is mentioned in Solaris section of Appendix C, which > you cite below. > >> Also from the manual: >> >> /You will need GNU libiconv and readline: the Solaris version of iconv is >> not sufficiently powerful. >> >> For the Solaris Studio compilers a little juggling of paths was needed to >> ensure GNU libiconv (in /usr/local) was used rather than the Solaris >> iconv: >> > This seems to indicate GNU libiconv should be installed in /usr/local, > which is the location specified in the libiconv installation > instructions: > > As usual for GNU packages: > $ ./configure --prefix=/usr/local > $ make > $ make install > > Did you try the suggestion two paragraphs above this in Appendix C? > "Some people have reported that the Solaris libintl needs to be > avoided, for example by using --disable-nls or --with-included-gettext > or using libintl from OpenCSW." > > As it turns out I put iconv and libiconv.so and libiconv.so.2.5.1 in > /usr/local/bin and /usr/local/lib. Just for grins I copied these files to > /usr/local and tried again with the same results. > > This is my configure statement: > > ./configure --with-blas=-library=sunperf --with-lapack --with-readline=no > --x-includes=/usr/X11/include --x-libraries=/usr/X11/lib --prefix=/usr/local > --disable-nls --with-included-gettext > > So, yes, I did try the suggestions mentioned in Appendix C. > >> CC="cc -xc99" >> CFLAGS="-O -xlibmieee" >> F77=f95 >> FFLAGS=-O4 >> CXX="CC -library=stlport4" >> CXXFLAGS=-O >> FC=f95 >> FCFLAGS=$FFLAGS >> FCLIBS="-lfai -lfsu" >> R_LD_LIBRARY_PATH="/usr/local/lib:/opt/csw/gcc4/lib:/opt/csw/lib" >> >> For a 64-bit target add -m64 to the compiler macros and use something like >> LDFLAGS=-L/usr/local/lib/sparcv9 or LDFLAGS=-L/usr/local/lib/amd64 as >> appropriate. / >> >> Well, I did that. Here are some of my options from the config.site: >> > >> >> So, here it is - all the revelent documentation on iconv. If I've missed >> anything please let me know. If you see where in the documentation I >> missed >> some cryptic clue on how to get iconv to work or where to put it or it's >> headers, also please let me know. >> > Did you take the steps required to resolve the circular dependency > between libiconv and gettext? > > I used the --disable-nls and --with-included-gettext options. I also > downloaded the more up-to-date cairo package. > I was referring to the instructions on the libiconv page: http://www.gnu.org/software/libiconv/ "On systems other than GNU/Linux, the iconv program will be internationalized only if GNU gettext has been built and installed before GNU libiconv." That seems to suggest that libiconv *may* require you to install GNU gettext before installing GNU libiconv. Have you done that? > BTW, is there a program I can run to test my iconv binary? I think that > would rule out any problem with the compilation. > > Thanks for you suggestions and for any future help. This is frustrating > because I have compiled R, just not with a workable iconv, which prevents me > from updating and adding packages. > > Roger > Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problems with iconv
As Uwe asked, please cite the original messages. On Tue, Dec 13, 2011 at 4:10 PM, RogerP wrote: > Sorry, but IMHO saying "read the manual" does not constitute actual help. > > But here it is from the manual: > > A suitably comprehensive iconv function is essential. The R usage requires > iconv to be able to translate between "latin1" and "UTF-8", to recognize "" > (as the current encoding) and "ASCII", and to translate to and from the > Unicode wide-character formats "UCS-[24][BL]E" — this is true for glibc but > not of most commercial Unixes. However, you can make use of GNU libiconv > (possibly as a plug-in replacement: see > http://www.gnu.org/software/libiconv/). > > Well, that's just what I did. I downloaded libiconv and compiled and linked > it. Oh, so where to put it. H, don't see it anywhere here. Do you? > The location isn't mentioned in Appendix A, which you cite a portion of above, but it is mentioned in Solaris section of Appendix C, which you cite below. > Also from the manual: > > /You will need GNU libiconv and readline: the Solaris version of iconv is > not sufficiently powerful. > > For the Solaris Studio compilers a little juggling of paths was needed to > ensure GNU libiconv (in /usr/local) was used rather than the Solaris iconv: > This seems to indicate GNU libiconv should be installed in /usr/local, which is the location specified in the libiconv installation instructions: As usual for GNU packages: $ ./configure --prefix=/usr/local $ make $ make install Did you try the suggestion two paragraphs above this in Appendix C? "Some people have reported that the Solaris libintl needs to be avoided, for example by using --disable-nls or --with-included-gettext or using libintl from OpenCSW." > CC="cc -xc99" > CFLAGS="-O -xlibmieee" > F77=f95 > FFLAGS=-O4 > CXX="CC -library=stlport4" > CXXFLAGS=-O > FC=f95 > FCFLAGS=$FFLAGS > FCLIBS="-lfai -lfsu" > R_LD_LIBRARY_PATH="/usr/local/lib:/opt/csw/gcc4/lib:/opt/csw/lib" > > For a 64-bit target add -m64 to the compiler macros and use something like > LDFLAGS=-L/usr/local/lib/sparcv9 or LDFLAGS=-L/usr/local/lib/amd64 as > appropriate. / > > Well, I did that. Here are some of my options from the config.site: > > > So, here it is - all the revelent documentation on iconv. If I've missed > anything please let me know. If you see where in the documentation I missed > some cryptic clue on how to get iconv to work or where to put it or it's > headers, also please let me know. > Did you take the steps required to resolve the circular dependency between libiconv and gettext? > One things for sure - you can't say I've not read the documentation! > > Roger > Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Please explain your workflow from R code -> package -> R code -> package
On Sat, Sep 10, 2011 at 11:23 AM, steven mosher wrote: > All I need now is a tool to go through the 4 packages I already > created without Roxygen and spit out source files with the Roxygen > comments in them... > > really lazy. > > That's what Rd2roxygen does... Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com > > On Fri, Sep 9, 2011 at 11:41 AM, Hadley Wickham wrote: >>> | In other languages, I've seen to write the documentation inside the >>> | code files and then post-process to make the documentation. Is there >>> | a similar thing for R, to unify the R code development and >>> | documentation/package-making process? >>> >>> You can also follow the cool kids who these days tie some of this together >>> using roxygen. >> >> It's not the cool kids who are doing this, it's the lazy kids ;) >> Roxygen(2) does remove a considerable amount of replication between >> code and documentation (e.g. replicating function usage in two >> places), and the close proximity between code and documentation does >> make it easier to remember to update your documentation when the code >> changes. >> >> Roxygen2 adds a few other tools for reducing duplication like >> templates, the ability to inherit parameter documentation from other >> function, and the family tag to automatically add seealso references >> between all members of a related family of functions. These are >> things that are painful to do by hand and add a significance >> maintenance burden. >> >> I agree that there's no silver bullet, but good tools certainly can >> make life easier. >> >> Hadley >> >> -- >> Assistant Professor / Dobelman Family Junior Chair >> Department of Statistics / Rice University >> http://had.co.nz/ >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Please explain your workflow from R code -> package -> R code -> package
On Fri, Sep 9, 2011 at 11:38 AM, Paul Johnson wrote: > Hi, > > I'm asking another one of those questions that would be obvious if I > could watch your work while you do it. > > I'm having trouble understanding the workflow of code and package maintenance. > > Stage 1. Make some R functions in a folder. This is in a Subversion repo > > R/trunk/myproject > > Stage 2. Make a package: > > After the package.skeleton, and R check, I have a new folder with the > project in it, > > R/trunk/myproject/mypackage > DESCRIPTION > man > R > > I to into the man folder and manually edit the Rd files. I don't > change anything in the R folder because I think it is OK so far. > > And eventually I end up with a tarball mypackage_1.0.tar.gz. > > Stage 3. How to make the round trip? I add more R code, and > re-generate a package. > > package.skeleton obliterates the help files I've already edited. > See ?dump and ?prompt. > So keeping the R code in sync with the documentation appears to be a hassle. > > In other languages, I've seen to write the documentation inside the > code files and then post-process to make the documentation. Is there > a similar thing for R, to unify the R code development and > documentation/package-making process? > Yes. See the roxygen / roxygen2 and devtools packages. > pj > > -- > Paul E. Johnson > Professor, Political Science > 1541 Lilac Lane, Room 504 > University of Kansas > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Small bug in install.packages?
It is 8.3 on my install of 2.13.0 (release) and I get the same results
with the current R-patched.
> R.home("bin")
[1] "C:/PROGRA~1/R/R-213~1.0PA/bin/i386"
> sessionInfo()
R version 2.13.0 Patched (2011-06-20 r56188)
Platform: i386-pc-mingw32/i386 (32-bit)
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
On Tue, Jun 28, 2011 at 4:42 PM, Hadley Wickham wrote:
>> Isn't R.home() 8.3 path anyway?
>
> I don't think so:
>
>> R.home("bin")
> [1] "C:/Program Files/R/R-2.13.0/bin/i386"
>> sessionInfo()
> R version 2.13.0 Patched (2011-06-09 r56106)
> Platform: i386-pc-mingw32/i386 (32-bit)
>
> locale:
> [1] LC_COLLATE=English_United States.1252
> [2] LC_CTYPE=English_United States.1252
> [3] LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United States.1252
>
> attached base packages:
> [1] grDevices datasets utils stats graphics methods base
>
> other attached packages:
> [1] RMark_2.0.5 rj_0.5.0-5 devtools_0.2 codetools_0.2-8
> [5] plotrix_3.2-2 nlme_3.1-101 msm_1.0.1
>
> loaded via a namespace (and not attached):
> [1] grid_2.13.0 lattice_0.19-26 mvtnorm_0.9-9991 rJava_0.8-8
> [5] splines_2.13.0 survival_2.36-9 tools_2.13.0
>
>> BTW: I think you probably meant more something like
>> paste(shQuote(file.path(R.home("bin"), "R")), "CMD INSTALL") since the above
>> won't work ..
>
> Ooops, yes.
>
> Hadley
>
>
> --
> Assistant Professor / Dobelman Family Junior Chair
> Department of Statistics / Rice University
> http://had.co.nz/
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] median and data frames
On Fri, Apr 29, 2011 at 9:25 AM, Martin Maechler wrote: >>>>>> Paul Johnson >>>>>> on Thu, 28 Apr 2011 00:20:27 -0500 writes: > > > On Wed, Apr 27, 2011 at 12:44 PM, Patrick Burns > > wrote: > >> Here are some data frames: > >> > >> df3.2 <- data.frame(1:3, 7:9) > >> df4.2 <- data.frame(1:4, 7:10) > >> df3.3 <- data.frame(1:3, 7:9, 10:12) > >> df4.3 <- data.frame(1:4, 7:10, 10:13) > >> df3.4 <- data.frame(1:3, 7:9, 10:12, 15:17) > >> df4.4 <- data.frame(1:4, 7:10, 10:13, 15:18) > >> > >> Now here are some commands and their answers: > > >>> median(df4.4) > >> [1] 8.5 11.5 > >>> median(df3.2[c(1,2,3),]) > >> [1] 2 8 > >>> median(df3.2[c(1,3,2),]) > >> [1] 2 NA > >> Warning message: > >> In mean.default(X[[2L]], ...) : > >> argument is not numeric or logical: returning NA > >> > >> > >> > >> The sessionInfo is below, but it looks > >> to me like the present behavior started > >> in 2.10.0. > >> > >> Sometimes it gets the right answer. I'd > >> be grateful to hear how it does that -- I > >> can't figure it out. > >> > > > Hello, Pat. > > > Nice poetry there! I think I have an actual answer, as opposed to the > > usual crap I spew. > > > I would agree if you said median.data.frame ought to be written to > > work columnwise, similar to mean.data.frame. > > > apply and sapply always give the correct answer > > >> apply(df3.3, 2, median) > > X1.3 X7.9 X10.12 > > 2 8 11 > > [...] > > exactly > > > mean.data.frame is now implemented as > > > mean.data.frame <- function(x, ...) sapply(x, mean, ...) > > exactly. > > My personal oppinion is that mean.data.frame() should never have > been written. > People should know, or learn, to use apply functions for such a > task. > > The unfortunate fact that mean.data.frame() exists makes people > think that median.data.frame() should too, > and then > > var.data.frame() > sd.data.frame() > mad.data.frame() > min.data.frame() > max.data.frame() > ... > ... > > all just in order to *not* to have to know sapply() > > > No, rather not. > > My vote is for deprecating mean.data.frame(). > > Martin > I agree. However, sd() isn't currently (as of R-2.13.0) generic and it operates by column for matrix and data.frame objects, so it behaves a bit more like mean() and is similarly inconsistent from the other listed functions. I have no input on how this should be handled, but thought it may be worth addressing. Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] FW: [Rcpp-devel] Question on 5.6 Interfacing C++ code
Please, please, please read the documentation before sending more
questions to the list. You also have the source code, so you can look
at what "R CMD build" and "R CMD INSTALL" are doing.
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
On Thu, Apr 21, 2011 at 10:16 AM, Sean Robert McGuffee
wrote:
>
> So, how is the package turning it's name into those commands?
> Does the installation automatically list the src directory and iteratively
> run a loop over each file and call 'R CMD SHLIB objectOfIterator' ?
> The reason this is so important is because itšs easy to get things to work
> via a terminal command 'R CMD SHLIB someSourceFile.cppš, so I want to be
> able to take things that work that way and put them inside a package. It
> seems peculiar to me that R needs to access functions through a C wrapper
> too. Išm not sure why it canšt access C++ functions directly. What R is
> doing is over my head because it is calling functions that were not compiled
> into it. I mean, if I want to call a C or C++ function from C++ code, I have
> to convince my compiler that I have a header and all definitions behind the
> declarations in my source files to compile my own programs. I donšt know how
> R works when the program is compiled way in advance and is then somehow
> calling on declarations made later in another place. I think objective C/C++
> allows for this type of thing where you can write code to call something
> that is declared but not yet defined. However, Išm not sure what R is doing?
> Is R doing the same thing a compiler would do and creating itšs own binary
> instructions for the launch of a function, or is it creating a new
> executable and launching that as itšs own application and then somehow
> communicating with it?
> Sean
>
>
> On 4/21/11 7:52 AM, "Dirk Eddelbuettel" wrote:
>
>>
>> On 21 April 2011 at 07:16, Duncan Murdoch wrote:
>> | On 11-04-20 11:33 AM, Sean Robert McGuffee wrote:
>> | > Hi, apparently I sent my question about using R and C++ to the wrong
>> list,
>> | > ironically seeing as that list was called Rcpp. Anyway, I was directed to
>> | > post my question here. To summarize my current question, I have found two
>> | > commands that I want to be able to put into a package. The commands are
>> 'R
>> | > CMD SHLIB X.cc X_main.cc' and
>> | > 'dyn.load(paste("X",.Platform$dynlib.ext,sep="")),' which I would like to
>> | > run when my package is installed and maybe have the second command run
>> again
>> | > when my package is to be used. I've been trying to figure out the
>> | > documentation and learn through examples, but I'm just not getting it and
>> | > have been trying for weeks.
>> | > Does anyone on this site have any suggestions for me?
>> |
>> | Assuming those lines work on their own, just do the following:
>> |
>> | 1. Put those *.cc files into the src directory of your package. (You
>> | may need to create it.)
>> |
>> | 2. Put useDynLib(foo) into the NAMESPACE file of your foo package.
>> |
>> | 3. Call those functions using .C("X", args, PACKAGE="foo").
>> |
>> | That's it.
>>
>> We told Sean this twice or three times already over in this thread
>>
>> http://thread.gmane.org/gmane.comp.lang.r.rcpp/1808
>>
>> but the message does not seem to sink in. He keeps asking where to put 'R
>> CMD SHLIB' and doesn't seem to hear when we say there is none in a package...
>>
>> Dirk
>>
>> | Duncan Murdoch
>> |
>> | > Thanks, Sean
>> | >
>> | > |On 20 April 2011 at 10:20, Sean Robert McGuffee wrote:
>> | > |
>> | > |
>> | > | Hi, thanks!
>> | > |
>> | > |>On 4/20/11 10:03 AM, "Steve Lianoglou"
>> | > wrote:
>> | > |> Hi,
>> | > |>
>> | > |> On Wed, Apr 20, 2011 at 9:49 AM, Sean Robert McGuffee
>> | > |> wrote:
>> | > |>> Hi, I have a quick couple of questions about some of the
>> documentation
>> | > on
>> | > |>> the web page:
>> | > |>>
>> | >
>> http://cran.r-project.org/doc/manuals/R-exts.html#Linking-GUIs-and-other-fro
>> | > n
>> | > |>> t_002dends-to-R
>> | > |>> under the heading:
>> | > |>> 5.6 Interfacing C++ code
>> | > |>>
>> | > |>> Question 1:
>> | > |>> If Išm at a terminal, I can type the instructions they suggest:
>> | > |>> R
Re: [Rd] duplicates() function
On Fri, Apr 8, 2011 at 10:15 AM, Duncan Murdoch wrote: > On 08/04/2011 11:08 AM, Joshua Ulrich wrote: >> >> How about: >> >> y<- rep(NA,length(x)) >> y[duplicated(x)]<- match(x[duplicated(x)] ,x) > > That's a nice solution for vectors. Unfortunately for me, I have a matrix > (which duplicated() handles by checking whole rows). So a better example > that I should have posted would be > > x <- cbind(1, c(9,7,9,3,7) ) > > and I'd still like the same output > For a matrix, could you apply the same strategy used in duplicated()? y <- rep(NA,NROW(x)) temp <- apply(x, 1, function(x) paste(x, collapse="\r")) y[duplicated(temp)] <- match(temp[duplicated(temp)], temp) >> duplicated(x) > > [1] FALSE FALSE TRUE FALSE TRUE > >> duplicates(x) > > [1] NA NA 1 NA 2 > > > Duncan Murdoch > >> -- >> Joshua Ulrich | FOSS Trading: www.fosstrading.com >> >> >> >> On Fri, Apr 8, 2011 at 9:59 AM, Duncan Murdoch >> wrote: >> > I need a function which is similar to duplicated(), but instead of >> > returning >> > TRUE/FALSE, returns indices of which element was duplicated. That is, >> > >> >> x<- c(9,7,9,3,7) >> >> duplicated(x) >> > [1] FALSE FALSE TRUE FALSE TRUE >> > >> >> duplicates(x) >> > [1] NA NA 1 NA 2 >> > >> > (so that I know that element 3 is a duplicate of element 1, and element >> > 5 is >> > a duplicate of element 2, whereas the others were not duplicated >> > according >> > to our definition.) >> > >> > Is there a simple way to write this function? I have an ugly >> > implementation in R that loops over all the values; it would make more >> > sense >> > to redo it in C, if there isn't a simple implementation I missed. >> > >> > Duncan Murdoch >> > >> > __ >> > R-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-devel >> > > > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] duplicates() function
How about: y <- rep(NA,length(x)) y[duplicated(x)] <- match(x[duplicated(x)] ,x) -- Joshua Ulrich | FOSS Trading: www.fosstrading.com On Fri, Apr 8, 2011 at 9:59 AM, Duncan Murdoch wrote: > I need a function which is similar to duplicated(), but instead of returning > TRUE/FALSE, returns indices of which element was duplicated. That is, > >> x <- c(9,7,9,3,7) >> duplicated(x) > [1] FALSE FALSE TRUE FALSE TRUE > >> duplicates(x) > [1] NA NA 1 NA 2 > > (so that I know that element 3 is a duplicate of element 1, and element 5 is > a duplicate of element 2, whereas the others were not duplicated according > to our definition.) > > Is there a simple way to write this function? I have an ugly > implementation in R that loops over all the values; it would make more sense > to redo it in C, if there isn't a simple implementation I missed. > > Duncan Murdoch > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] as.POSIXlt.factor and '...'
I noticed a difference in how as.POSIXct handled the 'format' argument when 'x' is character versus when 'x' is a factor. For example: myFormat <- "%d-%m-%Y" myDateStr <- format(Sys.Date()+1:5,myFormat) as.POSIXct(myDateStr, format=myFormat) as.POSIXct(factor(myDateStr), format=myFormat) It seems to be caused by '...' not being passed in as.POSIXlt.factor. Would it make sense to change the function to pass '...'? I.e. from: as.POSIXlt.factor <- function(x, ...) as.POSIXlt(as.character(x)) to: as.POSIXlt.factor <- function(x, ...) as.POSIXlt(as.character(x), ...) Regards, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] calling browser on error
I believe options(error=recover) will do what you want.
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
On Fri, Oct 15, 2010 at 1:27 PM, Vadim Ogranovich
wrote:
> Dear R-developers,
>
> I am trying to figure out a way to call browser() when an error occur, and
> naturally I want the browser() to be called in the environment of the error.
>
> I tried something simple in vain:
>
>> f <- function() { x <- 1; stop('ok') }
>> tryCatch(f(), error=browser())
> Called from: tryCatch(f(), error = browser())
> ## if browser() was called in the local environment of f then 'x' would be
> set, but it's not
> Browse[1]> x
> Error: object 'x' not found
> Browse[1]> Q
>
> Is there a way to make it work? What do people do to 'set an on-error
> breakpoint'?
>
> Thanks,
> Vadim
>
>
> Note: This email is for the confidential use of the named addressee(s) only
> and may contain proprietary, confidential or privileged information. If you
> are not the intended recipient, you are hereby notified that any review,
> dissemination or copying of this email is strictly prohibited, and to please
> notify the sender immediately and destroy this email and any attachments.
> Email transmission cannot be guaranteed to be secure or error-free. Jump
> Trading, therefore, does not make any guarantees as to the completeness or
> accuracy of this email or any attachments. This email is for informational
> purposes only and does not constitute a recommendation, offer, request or
> solicitation of any kind to buy, sell, subscribe, redeem or perform any type
> of transaction of a financial product.
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R-2.10.1 : There is no make file. (PR#14192)
On Fri, Jan 22, 2010 at 8:35 AM, wrote: > Full_Name: N. Srinivasan > Version: R-2.10.1 > OS: Ubuntu 9.10, 64 bits > Submission from: (NULL) (68.110.238.141) > > > ,/configure runs fine. > > But there is no make file mentioned in the INSTALL file. > > Usually this kind of information is provided in the README file. > > I am a donor to the R Foundation. > > Ubuntu uses the Synaptic. It only installs the R version 2.9.0 > > So, I have to compile R-2.10.1 from the source. > No you do not. See the R FAQ: http://cran.r-project.org/doc/FAQ/R-FAQ.html#Are-there-Unix-binaries-for-R_003f Best, Josh -- http://www.fosstrading.com > Will appreciate help on how to install R-2.10.1 > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Embed R, and provide a function to user scripts
Hi Saptarshi,
See ?SHLIB and ?dyn.load.
HTH,
Josh
--
http://www.fosstrading.com
On Thu, Aug 13, 2009 at 9:01 PM, Saptarshi Guha wrote:
> Essentially I'd like to load the C function into the load table
> After that, I can create an R function
> rboo=function(x) .Call("boo",x)
>
> then user script cann call rboo
>
> Hope that clarifies things.
> Thanks in advance
>
> Saptarshi Guha
>
>
> On Thu, Aug 13, 2009 at 9:07 PM, Saptarshi Guha
> wrote:
>
>> Hello,
>> I'm not sure how to go about this. Suppose I have a function
>>
>> SEXP boo(SEXP x){
>> //do something
>> }
>>
>> Also, I have an executable which embeds R in itself. I would like to
>> provide a R function to user code to call that calls 'boo' e.g
>>
>> ##user supplied expression that is given to me
>>
>> x<-1
>> boo(x)
>>
>> One way is to create a library with function boo that performs .Call to
>> boo, but is there a way that I can do something like
>>
>> ainstall("rboo", boo)
>>
>> and calls to rboo in the language will be sent to 'boo' ?
>>
>> Thank you
>> Saptarshi
>>
>>
>>
>> Saptarshi Guha
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rJava fails compilation on R-2.9.1 but not R-2.7.1 on Debian Lenny
Hi Mark,
The first thing that jumps out to me is that 2.7.1 compiles JRI, while
2.9.1 does not (for some reason, autodetect decides not to compile).
HTH,
Josh
--
http://www.fosstrading.com
On Wed, Jul 15, 2009 at 1:46 PM, Mark Kimpel wrote:
> I have been futzing around for days tying to get rJava to install on
> my freshly build Debian Lenny installation. I have R-2.9.1 installed
> from source in my R_HOME directory and R-2.7.1 installed via apt-get
> install r-recommended. I was tried many different things, but by
> accident started up R-2.7.1 as root and viola, successful install. I
> immediately exited out of R-2.7.1 and started R-2.9.1 as root and
> compilation fails. The file
>
> The output is voluminous, but is included below. I had originally
> thought that my problem had to do with setting environmental
> variables, but now it seems not.
>
> Thanks,
> Mark
>
> mkimpel-debian-xps /usr/lib/jvm/java-6-sun/include: su root
> Password:
> mkimpel-debian-xps:/usr/lib/jvm/java-6-sun/include#
> /home/mkimpel/R_HOME/R-2.9.1/R-build/bin/R
> WARNING: ignoring environment value of R_HOME
>
> R version 2.9.1 (2009-06-26)
> Copyright (C) 2009 The R Foundation for Statistical Computing
> ISBN 3-900051-07-0
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under certain conditions.
> Type 'license()' or 'licence()' for distribution details.
>
> Natural language support but running in an English locale
>
> R is a collaborative project with many contributors.
> Type 'contributors()' for more information and
> 'citation()' on how to cite R or R packages in publications.
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
>> install.packages('rJava')
> Warning in install.packages("rJava") :
> argument 'lib' is missing: using '/usr/local/lib/R/site-library'
> --- Please select a CRAN mirror for use in this session ---
> Loading Tcl/Tk interface ... done
> trying URL 'http://cran.fhcrc.org/src/contrib/rJava_0.6-3.tar.gz'
> Content type 'application/x-gzip' length 240527 bytes (234 Kb)
> opened URL
> ==
> downloaded 234 Kb
>
> * Installing *source* package ‘rJava’ ...
> checking for gcc... gcc -std=gnu99
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc -std=gnu99 accepts -g... yes
> checking for gcc -std=gnu99 option to accept ISO C89... none needed
> checking how to run the C preprocessor... gcc -std=gnu99 -E
> checking for grep that handles long lines and -e... /bin/grep
> checking for egrep... /bin/grep -E
> checking for ANSI C header files... yes
> checking for sys/wait.h that is POSIX.1 compatible... yes
> checking for sys/types.h... yes
> checking for sys/stat.h... yes
> checking for stdlib.h... yes
> checking for string.h... yes
> checking for memory.h... yes
> checking for strings.h... yes
> checking for inttypes.h... yes
> checking for stdint.h... yes
> checking for unistd.h... yes
> checking for string.h... (cached) yes
> checking sys/time.h usability... yes
> checking sys/time.h presence... yes
> checking for sys/time.h... yes
> checking for unistd.h... (cached) yes
> checking for an ANSI C-conforming const... yes
> checking whether time.h and sys/time.h may both be included... yes
> configure: checking whether gcc -std=gnu99 supports static inline...
> yes
> checking Java support in R... present:
> interpreter : '/usr/bin/java'
> archiver : '/usr/bin/jar'
> compiler : '/usr/bin/javac'
> header prep.: '/usr/bin/javah'
> cpp flags : '-I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/linux'
> java libs : '-L$(JAVA_HOME)/lib/amd64/server
> -L$(JAVA_HOME)/lib/amd64 -L$(JAVA_HOME)/../lib/amd64 -L
> -L/usr/java/packages/lib/amd64 -L/lib -L/usr/lib -ljvm'
> checking whether JNI programs can be compiled... yes
> checking JNI data types... ok
> checking whether JRI should be compiled (autodetect)... no
> checking whether debugging output should be enabled... no
> checking whether memory profiling is desired... no
> checking whether threads support is requested... no
> checking whether callbacks support is requested... no
> checking whether JNI cache support is requested... no
> checking whether JRI is requested... no
> configure: creating ./config.status
> config.status: creating src/Makevars
> config.status: creating R/zzz.R
> config.status: creating src/config.h
> ** libs
> gcc -std=gnu99 -I/home/mkimpel/R_HOME/R-2.9.1/R-build/lib64/R/include
> -I. -I/usr/lib/jvm/java-1.5.0-gcj-4.3-1.5.0.0/jre/../include
> -I/usr/lib/jvm/java-1.5.0-gcj-4.3-1.5.0.0/jre/../include/linux
> -I/usr/local/include -fpic -g -O2 -c Rglue.c -o Rglue.o
> In file included
Re: [Rd] simple add error (PR#13699)
Gostan,
This is not a bug. You're asking for 20 decimal digits of precision,
which is impossible with double-precision floating point arithmetic.
http://fr.wikipedia.org/wiki/Virgule_flottante
Best,
Josh
--
http://www.fosstrading.com
On Wed, May 13, 2009 at 7:35 AM, wrote:
> Full_Name: Gostan Thierry
> Version: 2.6.1 (2007-11-26)
> OS: Windows XP
> Submission from: (NULL) (193.49.190.42)
>
>
> I cannot explain why R seems to have problems adding two big numbers.
>
> sprintf("%f",10^4+10^19) gives "10010240.00"
> instead of "1001.00"
>
> problems seems to arrive when i'm trying to add a big and a small number...
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
