Re: [R-pkg-devel] [External] RcmdrPlugin.HH_1.1-48.tar.gz
On Wed, Mar 6, 2024 at 1:03 AM Richard M. Heiberger wrote:
>
> Thank you Duncan, Jeff, Ivan.
>
> I did all that Duncan and Jeff suggested, plus a bit more that appeared to be
> necessary.
> All of what I did is documented in the RcmdrPlugin.HH/NEWS file.
>
> Ivan's comments were received after I sent 1.1-50 to CRAN and it was accepted.
>
I recommend you revert all the changes you made that are documented in
the package NEWS, and at minimum follow Ivan's advice to use
exportPattern("^[^\\.]") instead of exportPattern("."). It would be
even better to follow the advice in Writing R Extensions and list each
exported object individually.
> I suggest that my notes in the NEWS file, perhaps augmented with Ivan's
> comments,
> might be added to utils/man/globalVariables.Rd and to the
> "
> section ‘Package
> structure’ in the ‘Writing R Extensions’ manual.
> "
>
That section of Writing R Extensions specifically says not to do what you did.
Beware of patterns which include names starting with a period: some
of these are internal-only variables and should never be exported,
e.g. ‘.__S3MethodsTable__.’ (and loading excludes known cases).
Duncan pointed out that '.__global__' is an internal-only variable
created by globalVariables(), which means it should never be exported
by a package. Imagine the number of conflicts there would be if every
package that used globalVariables() exported the '.__global__'
object... there would probably be thousands, yikes!
It's possible that future versions of 'R CMD check' will error if
there are any incorrectly exported internal variables (like
'.__global__'), which would cause your package to fail.
Best,
Josh
>
> > On Mar 6, 2024, at 01:38, Ivan Krylov wrote:
> >
> > В Tue, 5 Mar 2024 22:41:32 +
> > "Richard M. Heiberger" пишет:
> >
> >> Undocumented code objects:
> >> '.__global__'
> >> All user-level objects in a package should have documentation
> >> entries. See chapter 'Writing R documentation files' in the 'Writing R
> >> Extensions' manual.
> >
> > This object is not here for the user of the package. If you don't
> > export it, there will be no WARNING about it being undocumented. This
> > variable is exported because of exportPattern(".") in the file
> > NAMESPACE. The lone dot is a regular expression that matches any name
> > of an R object.
> >
> > If you don't want to manually list your exports in the NAMESPACE file
> > (which can get tedious) or generate it (which takes additional
> > dependencies and build steps), you can use exportPattern('^[^\\.]') to
> > export everything except objects with a name starting with a period:
> > https://cran.r-project.org/doc/manuals/R-exts.html#Specifying-imports-and-exports
> >
> > --
> > Best regards,
> > Ivan
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Changing package maintainers
The process is described in the 6th bullet of the "Source Packages"
section of the CRAN Policies:
https://cran.r-project.org/web/packages/policies.html
"When a new maintainer wishes to take over a package, this should be
accompanied by the written agreement of the previous maintainer
(unless the package has been formally orphaned)."
So the current maintainer needs to send CRAN an email with the name
and email address of the new maintainer.
Best,
Josh
On Thu, Feb 8, 2024 at 10:38 AM Josiah Parry wrote:
>
> I intend to change the maintainer of my package {sfdep}
> https://cran.r-project.org/package=sfdep.
> Is the process as simple as submitting a new release where the DESCRIPTION
> file changes who has the role of `aut`?
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Segmentation fault early in compilation of revision 85514
On Mon, Nov 13, 2023 at 12:45 PM Avraham Adler wrote: > > On Mon, Nov 13, 2023 at 1:13 AM Dirk Eddelbuettel wrote: > > > > > > Avi, > > > > Might be toolchain-dependent, might be options-dependent--it built fine > > here. > > Easier for you to vary option two so maybe try that? > > > > Dirk > > Thank you, Dirk. > > I think it was more a PEBCAK issue. When I deleted the entire trunk > folder and started the process from scratch, it compiled properly as > per usual. Although this was revision 85520, so perhaps something > changed in the interim. > Are you using a separate build directory, or building in the src/ directory? The R Installation and Administration manual recommends using a separate build directory: You do not necessarily have to build R in the top-level source directory (say, TOP_SRCDIR). To build in BUILDDIR, run: cd BUILDDIR TOP_SRCDIR/configure make and so on, as described further below. This has the advantage of always keeping your source tree clean and is particularly recommended when you work with a version of R from Subversion. > Regardless, the issue resolved itself. > > Thanks, > > Avi > > ______ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] LICENSE file in an R package for CRAN submission
On Wed, Aug 16, 2023 at 11:41 AM Vincent Goulet wrote: > > Although late to the party, I thought I might chime in with a different take > on the question. > > Most answers seem to pertain to the actual license used for the package. For > my part, I read the question as: "GitHub wants me to have a LICENSE file at > the root of the repository, but R CMD check complains about it. What should I > do?" > > If you need to use the LICENSE file in your package, Ben Bolker already > pointed you at the correct way to do it. > > If the file is just there to please GitHub (and in any case), here's what I > tend to do: put the source code of your package in a subdirectory (say 'src') > of the repository. You may then put the GitHub files (README.md, LICENSE, > whatever else) at the top level without interfering with R CMD build/check. > The added benefit is that you will then build the package in the directory of > the project (say 'mypkg') with 'R CMD build src' and end up with a file > .../mypkg/mypkg_x.y-z.tar.gz, rather than in the parent directory of 'mypkg'. > Another alternative is put everything you don't want to be included in your package in the .Rbuildignore file. For example: https://github.com/joshuaulrich/xts/blob/main/.Rbuildignore > (Mind you, I build my packages manually. The tidyverse tools may very well > take care of this sort of things automatically and I wouldn't know.) > I also build my packages manually (with a makefile) and have my own personal preferences, so the patterns in my file may need to be changed for others' packages. > Hope this helps, > > v. > > Vincent Goulet > Professeur titulaire > École d'actuariat, Université Laval > > > Le 9 août 2023 à 11:06, Emanuele Cordano a > > écrit : > > > > Dear list, > > > > is there a way to put the LICENSE file within an R package like in Github, > > I have an R package on Github with a a LICENSE file compliant to Github and > > containing the text of the licence citing in the DESCRIPION file. But when > > I check the package , I obatained the following output: > > > > * checking top-level files ... NOTE > > File > > LICENSE > > > > is not mentioned in the DESCRIPTION file. > > > > How can I solve this? > > Thank you > > best > > Emanuele Cordano > > -- > > Emanuele Cordano, PhD > > Environmental Engineer / Ingegnere per l' Ambiente e il territorio nr. > > 3587 (Albo A - Provincia di Trento) > > cell: +39 3282818564 > > email: emanuele.cord...@gmail.com,emanuele.cord...@rendena100.eu, > > emanuele.cord...@eurac.edu > > PEC: emanuele.cord...@ingpec.eu > > URL: www.rendena100.eu > > LinkedIn: https://www.linkedin.com/in/emanuele-cordano-31995333 > > GitHub: https://github.com/ecor > > > > [[alternative HTML version deleted]] > > > > ______ > > R-package-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-package-devel > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Segmentation Fault cause 'memory not mapped' on Debian only
Hi David, In addition to running under valgrind as Dirk suggests below, you should also run under UBSAN and ASAN. You can do all 3 (and more) using Winston Cheng's docker images: https://github.com/wch/r-debug/ That should help you narrow down the problematic code. Best, Josh On Fri, Apr 21, 2023 at 2:33 PM Dirk Eddelbuettel wrote: > > > Hi David, > > On 21 April 2023 at 19:17, D Z wrote: > | Hi all, > | I wanted to publish my RITCH package (https://github.com/DavZim/RITCH) to > CRAN, which has Rcpp code. > > That would be great! > > | It has, in it’s CICD Pipeline, Tests for Macos, Windows, and Ubuntu (devel, > release,, and oldrel-1), which all succeed on Github Actions. > | When I submitted the package to CRAN, I was notified, that I got the > following error on Debian (the tests on Windows throw no such bug, neither do > the Github Actions tests): > | > | > |*** caught segfault *** > | > | address 0x55939b7ca000, cause 'memory not mapped' > | > | Segmentation fault > | > | The full Report is currently available here: > https://win-builder.r-project.org/incoming_pretest/RITCH_0.1.14_20230420_223157/Debian/00check.log > (note the error happens in the test_filter_itch.R unit tests). > | > | I looked for the error online but couldn’t find anything useful, and as I > am not able to reproduce the segfault locally, cannot debug it. > | Do you have any ideas or hunches what might cause this and how to resolve > it? > > "Absence of error" on one platform does proove correctness on all, the hint > from the segfault should be taken seriously. I just updated my older > checkout of your repo, built a current tarball and simply invoked > >R CMD check --use-valgrind RITCH_0.1.14.tar.gz > > which you could do on CI too (I can help as needed, I would also pull the CI > runners forward from Ubuntu 20.04 to 22.04). When we do the above, the > examples output file RITCH-Ex.Rout ends in > > ==2269271== LEAK SUMMARY: > ==2269271==definitely lost: 1,363,969 bytes in 5,473 blocks > ==2269271==indirectly lost: 177,235 bytes in 1,281 blocks > ==2269271== possibly lost: 101,924,267 bytes in 148 blocks > ==2269271==still reachable: 89,572,890 bytes in 17,950 blocks > ==2269271== suppressed: 0 bytes in 0 blocks > ==2269271== Reachable blocks (those to which a pointer was found) are not > shown. > ==2269271== To see them, rerun with: --leak-check=full --show-leak-kinds=all > ==2269271== > ==2269271== ERROR SUMMARY: 30 errors from 30 contexts (suppressed: 0 from 0) > > and the tests end in > > ==2273761== LEAK SUMMARY: > ==2273761==definitely lost: 5,223,981 bytes in 141,151 blocks > ==2273761==indirectly lost: 2,241,700 bytes in 2,767 blocks > ==2273761== possibly lost: 1,201,493,644 bytes in 270 blocks > ==2273761==still reachable: 133,909,136 bytes in 19,110 blocks > ==2273761== suppressed: 0 bytes in 0 blocks > ==2273761== Reachable blocks (those to which a pointer was found) are not > shown. > ==2273761== To see them, rerun with: --leak-check=full --show-leak-kinds=all > ==2273761== > ==2273761== ERROR SUMMARY: 43 errors from 39 contexts (suppressed: 0 from 0) > > That probably warrants a few more close looks. > > Hth, Dirk > > -- > dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Replacing dplyr by stats functions
On Wed, Apr 19, 2023 at 9:19 AM A. Kucharski wrote: > > Hi, > > I am developing my own package. I have a problem with its new version. > Checking on the CRAN server gives 2 warnings like this (critpath is my > package): > Warning: replacing previous import 'dplyr::lag' by 'stats::lag' when loading > 'critpath' > Warning: replacing previous import 'dplyr::filter' by 'stats::filter' when > loading 'critpath' > > On my machine these warnings show up during Check but before Build starts so > at the end I get a message that the number of warnings is 0. I don't use lag > or filter at all in my package although I use both of these packages. > Reordering the dplyr and stats package names in the Depends of the > Description file didn't help. Is it a problem with my code or a conflict > between those two packages? Please help me remove these warnings. > It's a conflict between stats and dplyr. The dplyr lag() function masks the base R stats lag() generic, which breaks S3 method dispatch for every package that has a lag() method when dplyr is attached. You should use Imports instead of Depends unless you need the package attached, which is a rare case. The latest version of critpath on CRAN uses `import()` in the NAMESPACE file. That imports every function in the entire package. Use `importFrom()` to import only the functions you use from the package. That should help avoid these types of conflicts. > Best regards > > Adam > > Wysłane z aplikacji Poczta dla systemu Windows > > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Tools for reducing dependencies?
Hi Duncan, On Sun, Mar 12, 2023 at 6:34 AM Duncan Murdoch wrote: > > The rgl package has a really large number of dependencies, especially if > I include soft (Suggested) dependencies. It would be nice to reduce > that number. > > To do that, I can imagine a tool that gives the following information: > > - For each export from rgl, which dependencies are needed? > > - For each dependency, which exports are supported? > > - Are there other ways to get the same support with fewer > dependencies? E.g. if rgl used devtools::install_github, it could be > replaced with remotes::install_github. > > Does a tool already exist that addresses these questions? > I've never used it, but there's some relevant functionality in https://github.com/r-lib/itdepends Best, Josh > Duncan Murdoch > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Rd cross-references to Suggested package
Hi all,
I'd like to link to a help page of a package in my package's Suggests.
WRE, section 2.5 says,
"Historically (before R version 4.1.0), links of the form
\link[pkg]{foo} and \link[pkg:bar]{foo} used to be interpreted as
links to files foo.html and bar.html in package pkg, respectively. For
this reason, the HTML help system looks for file foo.html in package
pkg if it does not find topic foo, and then searches for the topic in
other installed packages. To test that links work both with both old
and new systems, the pre-4.1.0 behaviour can be restored by setting
the environment variable _R_HELP_LINKS_TO_TOPICS_=false.
"Packages referred to by these ‘other forms’ should be declared in the
DESCRIPTION file, in the ‘Depends’, ‘Im
ports’, ‘Suggests’ or ‘Enhances’ fields."
This seems to imply that it's possible... though I don't understand
when I need to set _R_HELP_LINKS_TO_TOPICS_=false in order to test
that the link is done correctly. I'm using \link[pkg]{foo} in R 4.2.1.
I ran R CMD build/INSTALL/check with and without that env var set to
false. Both times the suggested package was not installed on my
library path, so I had to set _R_CHECK_FORCE_SUGGESTS_=false for R CMD
check --as-cran.
I didn't notice a difference in output from R CMD check. Both runs had:
* checking Rd cross-references ... NOTE
Package unavailable to check Rd xrefs: ‘timeSeries’
I'd appreciate any thoughts and/or pointers to other documentation.
Best,
Josh
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Searching examples in source code
Hi Ben, On Sat, May 7, 2022 at 4:24 PM Ben Engbers wrote: > > Hi, > > My package (RBaseX) is written entirely in R. The performance is not bad > but to further improve the performance I want to investigate to what > extent use of C++ makes sense. Problem is that I have little experience > with C++ and none with Rcpp. So I am looking for examples. > On my (linux) system I installed several packages that needed to be > compiled. So it is likely that I already have examples on my system. > I strongly recommend you profile your code to determine where there are performance bottlenecks before writing any new code. Especially before adding compiled code to your package. The infamous Knuth quote: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified." Best, Josh > My first question is if there is a useful linux command to search all > the source code of installed packages on my system. > The second question is if there is a command to search all packages at > https://cran.r-project.org/web/packages/available_packages_by_name.html? > > This question is not only relevant to C++ examples. It would also be > nice if you could search for occurrences of commands in R code. > > Ben > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] S3 dispatch does not work for generics defined inside an environment
On Wed, Jun 30, 2021 at 5:17 AM Duncan Murdoch wrote:
>
> On 30/06/2021 5:22 a.m., Taras Zakharko wrote:
> > Dear all,
> >
> > I have a generic function and a bunch of methods defined in a separate
> > environment. Here is a reduced example:
> >
> > env <- local({
> > # define the generic function and the method
> > myfun <- function(x) UseMethod("myfun")
> > myfun.myclass <- function(x) print("called myfun.myclass”)
> >
> > # register the method
> > .S3method("myfun", "myclass", myfun.myclass)
> >
> > environment()
> >})
> >
> > Since the method has been registered, I hoped that invocation like this
> > would work:
> >
> > env$myfun(structure(0, class = "myclass”))
> >
> > However, this results in a “no applicable method" error.
> >
> > It is my understanding that registerS3method (called by .S3method) will
> > install the method string in the .__S3MethodsTable__. table of the
> > environment where the generic function is defined, and this table is
> > subsequently used by usemethod() inside R, so I am puzzled that the
> > dispatch does not work. I checked and the .__S3MethodsTable__. of env is
> > indeed setup correctly. I also tried manually adding the method string to
> > the global .__S3MethodsTable__. inside .BaseNamespaceEnv to no effect.
> >
> > In fact, the only way to make it work is to define either myfun or
> > myfun.myclas in the global environment, which is something I would like to
> > avoid.
> >
> > Thank you in advance for any pointers!
> >
>
> registerS3method has an additional parameter "envir" which I believe
> would end up set to env in your code. So this works:
>
> > eval(expression(myfun(structure(0, class = "myclass"))), envir = env)
> [1] "called myfun.myclass"
>
> You could probably also call registerS3method with envir specified
> appropriately and get your original expression to work.
>
That doesn't seem to work on 4.1.0 for me. The code below worked for
me in Oct-2020, though I'm not sure what version of R I was using at
the time. I was slow to upgrade to 4.0, so it was probably the latest
3.x version.
env <- new.env()
local({
# define the generic function and the method
myfun <- function(x) { UseMethod("myfun", x) }
# register the method
registerS3method("myfun", "myclass",
function(x) { print("called myfun.myclass") },
envir = env)
}, envir = env)
attach(env)
myfun(structure(0, class = "myclass"))
> Duncan Murdoch
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Advice on R-forge to Github migration
On Sun, Jan 31, 2021 at 2:17 PM Duncan Murdoch wrote: > > Thanks to everyone who commented. A few replies inline: > > On 31/01/2021 1:21 p.m., Joshua Ulrich wrote: > > I've moved history and issues from R-Forge to GitHub for half a dozen > > R packages. I might be able to do this rgl. At minimum, I could help > > you do it. > > Thanks for the offer (and the more detailed one offline, where you > recommend cloning the repository rather than forking it). Just for > future reference, could you tell me the benefits of cloning over forking? > We can discuss more offline, but at a high level: - if you fork my repo, mine is the head of the network graph - if you clone it and then push it to a repo in your account, yours is the head of the network graph Basically, I don't want your repo to be a fork of mine. That would confuse users about where issues/PRs should be. I plan to delete my repo once you've loaded the copy to your account. > Duncan Murdoch > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Advice on R-forge to Github migration
On Sun, Jan 31, 2021 at 12:10 PM Duncan Murdoch wrote: > > rgl has been on R-forge for a long time, but I am now planning on > migrating it to Github. I really dislike git, but Github offers enough > benefits, and nowadays I'm familiar enough with them, that I think I'd > be better off there. > > The easiest way to do this would be to do almost nothing: just declare > the the dmurdoch/rgl fork of r-forge/rgl is now where all new changes > will be committed. > > Can anyone else who has done this migration tell me if there there any > disadvantages to this that I don't know about? What I know: > > - I'll lose the bug reports and forum discussions that were sent to > R-forge. > - I'll need to do a bit of work to change dmurdoch/rgl to a more > standard R package layout, but this should be quite easy: basically > just moving the files in pkg/rgl to the top level. I assume "git mv" > will keep their history if I do this. > I've moved history and issues from R-Forge to GitHub for half a dozen R packages. I might be able to do this rgl. At minimum, I could help you do it. Yes, git mv will retain the file history. > Duncan Murdoch > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Demo for linking native routines between R packages
Hi David, On Wed, Jan 6, 2021 at 4:00 AM Jitao David Zhang wrote: > > Dear Davis, Dirk, Jan, and r-devel subscribers, > > I hope you have had a good start in the new year. > > For your information: I have updated the GitHub repository to demonstrate > linking native routines between R packages ( > https://github.com/Accio/demo-linking-native), especially by listing the > prior art and the project by Davis to give the readers more background and > history of the topic. And I took the liberty to acknowledge your > contribution, criticism and help to improve this baby project. > > In any case, I wish there is a link from WRE to a small demo (not > necessarily mine!) where linking native routine is implemented in a > minimalistic example for teaching and demonstration purposes. That will > help people a lot in my opinion. > R-core will probably be reluctant to link to an external example they have very little control over. For example, what if the process changes and the external source doesn't get updated? That said, WRE does list two packages that register native routines from other packages: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Linking-to-native-routines-in-other-packages I'd be open to including a vignette in xts that discusses the process of linking to zoo. That would have the benefit of needing to be updated if the process changes. We can discuss off-list if you're interested in collaborating on it. > Best regards, > David > > -- > Jitao David Zhang > > http://jdzhang.me/ > m...@jdzhang.me > > Schützengasse 20 > 4125 Riehen, Switzerland > > [[alternative HTML version deleted]] > > ______ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] unexpected CRAN pretest failure
On Thu, Dec 17, 2020 at 7:25 AM Rossum, Bart-Jan van wrote: > > Dear Uwe, > > Thanks for your reaction. > I installed the latest available R-devel version (2020-12-15 r79633) on my > own windows pc, but even then the checks pas cleanly. > So, unfortunately I'm not able to reproduce the issue. > > To be sure, I also retried on Winbuilder, which has a slightly different > version (r79643), but there the issue is still present. > > Could there be something else I need to do to be able to reproduce this > locally? > I assume you used the pre-compiled version here: https://cran.r-project.org/bin/windows/base/rdevel.html That version probably lags behind the subversion repo for about a day. So you need to build the latest R-devel from the latest source in the subversion repo. It always has the latest version. Rocker and r-hub help make this much easier than setting everything up locally on your machine. That said, it looks like the pre-compiled version is at r79643 now, so you should be able to use it to reproduce the issue. Best, Josh > Regards, > Bart-Jan > > -Original Message- > From: Uwe Ligges > Sent: Thursday, December 17, 2020 13:29 > To: Rossum, Bart-Jan van ; > r-package-devel@r-project.org > Subject: Re: [R-pkg-devel] unexpected CRAN pretest failure > > This is form a change in R-devel. > Use a more recent R-devel to reproduce the issue. > > Best, > Uwe Ligges > > On 14.12.2020 15:02, Rossum, Bart-Jan van wrote: > > Dear community, > > > > When trying to update my CRAN > > (https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fweb%2Fpackages%2FstatgenGxE%2Findex.htmldata=04%7C01%7Cbart-jan.vanrossum%40wur.nl%7C7cee644adb0a4aebb06008d8a28746e3%7C27d137e5761f4dc1af88d26430abb18f%7C0%7C0%7C637438049192516675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=XsyDDtwJk%2FXlDxxWxx2ylcx9RX%2BVOb9XWSZruwo42HY%3Dreserved=0) > > package in ran into unexpected error on Windows. > > I had tested before submission on R-hub, which went fine, but CRAN > > complained, and this was confirmed on Winbuilder. > > I noticed a slight difference in R-version used on CRAN/Winbuilder and > > R-hub. > > However, the error itself seems to come from an lme4 function. > > I'm quite clueless on how to debug/fix this. > > > > CRAN and Winbuilder: > > R Under development (unstable) (2020-12-13 r79623) > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2FnK0OMOQ378SIdata=04%7C01%7Cbart-jan.vanrossum%40wur.nl%7C7cee644adb0a4aebb06008d8a28746e3%7C27d137e5761f4dc1af88d26430abb18f%7C0%7C0%7C637438049192516675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=WpzlD%2B7%2BsggCPKJUrjvV4ZchNF12tXAXfrVMjRlh10c%3Dreserved=0 > > > > R-hub: > > R Under development (unstable) (2020-11-30 r79529) > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbuilder.r-hub.io%2Fstatus%2FstatgenGxE_1.0.4.tar.gz-fcc1e205a5fb4fd09559d301ee3502c9data=04%7C01%7Cbart-jan.vanrossum%40wur.nl%7C7cee644adb0a4aebb06008d8a28746e3%7C27d137e5761f4dc1af88d26430abb18f%7C0%7C0%7C637438049192516675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=aWev8qrAGU5yO%2FP%2FGqqb3EVqdYYkGq%2FYrUf0b2ZqrDw%3Dreserved=0 > > > > Any pointers are appreciated, > > Bart-Jan > > > > __ > > R-package-devel@r-project.org mailing list > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-package-develdata=04%7C01%7Cbart-jan.vanrossum%40wur.nl%7C7cee644adb0a4aebb06008d8a28746e3%7C27d137e5761f4dc1af88d26430abb18f%7C0%7C0%7C637438049192516675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=VA%2BpMcbmLDsLsjeZj5ucESwPHfgylxG59HHYkr9MOiw%3Dreserved=0 > > > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How to remove rJava from Ecfun when it's not called directly or indirectly
On Sat, Oct 10, 2020 at 10:57 PM Spencer Graves
wrote:
>
> Hello, All:
>
>
> "R CMD check Ecfun_0.2-4.tar.gz" fails under Windows 10 with "Error:
> package or namespace load failed for 'Ecfun': .onLoad failed in
> loadNamespace() for 'rJava'".
>
>
> However, I cannot find where Ecfun calls rJava. This is in:
>
>
> https://github.com/sbgraves237/Ecfun
>
>
> I get nothing from "grep 'rJava'" in the DESCRIPTION and NAMESPACE
> files plus in the man and R subdirectories. I ran
> tools:package_dependencies recursively starting with Ecfun until I got
> all NULLs and could not find rJava anywhere.
>
You need to include suggested package dependencies in your search.
tools::package_dependencies("Ecfun",
which = c("Depends", "Imports", "LinkingTo", "Suggests"),
recursive = TRUE)
You will find rJava in that list of ~1650 dependencies.
>
> Thanks,
> Spencer Graves
>
>
> 00install.out
>
>
> * installing *source* package 'Ecfun' ...
> ** using staged installation
> ** R
> ** inst
> ** byte-compile and prepare package for lazy loading
> ** help
> *** installing help indices
> ** building package indices
> ** installing vignettes
> ** testing if installed package can be loaded from temporary location
> *** arch - i386
> Error: package or namespace load failed for 'Ecfun':
> .onLoad failed in loadNamespace() for 'rJava', details:
>call: inDL(x, as.logical(local), as.logical(now), ...)
>error: unable to load shared object 'C:/Program
> Files/R/R-4.0.2/library/rJava/libs/i386/rJava.dll':
>LoadLibrary failure: %1 is not a valid Win32 application.
>
> Error: loading failed
> Execution halted
> *** arch - x64
> ERROR: loading failed for 'i386'
> * removing 'C:/Users/spenc/Documents/R/Ecfun/Ecfun.Rcheck/Ecfun'
>
>
> 00check.log
>
>
> * using log directory 'C:/Users/spenc/Documents/R/Ecfun/Ecfun.Rcheck'
> * using R version 4.0.2 (2020-06-22)
> * using platform: x86_64-w64-mingw32 (64-bit)
> * using session charset: ISO8859-1
> * checking for file 'Ecfun/DESCRIPTION' ... OK
> * this is package 'Ecfun' version '0.2-4'
> * checking package namespace information ... OK
> * checking package dependencies ... OK
> * checking if this is a source package ... OK
> * checking if there is a namespace ... OK
> * checking for executable files ... OK
> * checking for hidden files and directories ... OK
> * checking for portable file names ... OK
> * checking whether package 'Ecfun' can be installed ... ERROR
> Installation failed.
> See 'C:/Users/spenc/Documents/R/Ecfun/Ecfun.Rcheck/00install.out' for
> details.
> * DONE
> Status: 1 ERROR
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Dependency needs to be loaded manually even its specified in the package
On Fri, Sep 18, 2020 at 11:56 AM Dirk Eddelbuettel wrote: > > > On 18 September 2020 at 18:38, Nuria Perez-Zanon wrote: > | I am maintaining a package call CSTools which is aimed for > | post-processing climate simulations. > [...] > | library(CSTools) > | library(qmap) > > You never use library() in a package. Rather, you declare dependency > relationsships via DESCRIPTION (and likely NAMESPACE). See "Writing R > Extensions" for all the details. > And here's the relevant section: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Registering-S3-methods Best, Josh > > Dirk > > -- > https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of `:::` in a package for code run in a parallel cluster
On Sun, Sep 13, 2020 at 3:19 PM Duncan Murdoch wrote:
>
> On 13/09/2020 3:51 p.m., David Kepplinger wrote:
> > Dear list members,
> >
> > I submitted an update for my package and got automatically rejected by the
> > incoming checks (as expected from my own checks) for using `:::` calls to
> > access the package's namespace.
> > "There are ::: calls to the package's namespace in its code. A package
> > *almost* never needs to use ::: for its own objects:…" (emphasis mine)
> >
> > This was a conscious decision on my part as the package runs code on a
> > user-supplied parallel cluster and I consider cluster-exporting the
> > required functions a no-go as it would potentially overwrite objects in the
> > clusters R sessions. The package code does not own the cluster and hence
> > the R sessions. Therefore overwriting objects could potentially lead to
> > unintended behaviour which is opaque to the user and difficult to debug.
> >
> > Another solution to circumvent the R CMD check note is to export the
> > functions to the public namespace but mark them as internal. This was also
> > suggested in another thread on this mailing list (c.f. "Etiquette for
> > package submissions that do not automatically pass checks?"). I do not
> > agree with this work-around as the methods are indeed internal and should
> > never be used by users. Exporting truly internal functions for the sake of
> > satisfying R CMD check is a bad argument, in particular if there is a
> > clean, well-documented, solution by using `:::`
>
> Who is calling this function: package code or user code? I assume it's
> a bit of a mix: your package writes a script that calls the function
> when it runs in user space. (It would help if you gave an explicit
> example of when you need to use this technique.)
>
> If my assumption is correct, there are other simple workarounds besides
> exporting the functions. Instead of putting
>
> pkg:::foo(args)
>
> into your script, put
>
> pkg::callInternal("foo", args)
>
> where pkg::callInternal is an exported function that can look up
> unexported functions in the namespace.
>
Another possibility is what quantmod::newTA() does.
https://github.com/joshuaulrich/quantmod/blob/a8e9cb87825c0997a8468f5105db6c507b26ac5d/R/newTA.
It's a function that creates a user-facing function. The created
function needs to access unexported objects from the quantmod
namespace. newTA() accomplishes that by setting the environment of
the function it returns to the quantmod namespace.
https://github.com/joshuaulrich/quantmod/blob/a8e9cb87825c0997a8468f5105db6c507b26ac5d/R/newTA.R#L98
That gives the user's new function access to the unexported charting
objects it needs to work.
Hope that helps.
Best,
Josh
> You may argue that you prefer pkg:::foo for some reason: to which I'd
> respond that you are being rude to the CRAN volunteers. I've offered
> two options (one in the previous thread, a different one here), and
> there was a third one in that thread offered by Ivan Krylov. Surely one
> of these is good enough for your needs, and you shouldn't force CRAN to
> handle you specially.
>
> Duncan
>
> >
> > I argue `:::` is the only clean solution to this problem and no dirty
> > work-arounds are necessary. This is a prime example of where `:::` is
> > actually useful and needed inside a package. If the R community disagrees,
> > I think R CMD check should at least emit a WARNING instead of a NOTE and
> > elaborate on the problem and accepted work-arounds in "Writing R
> > extensions". Or keep emitting a NOTE but listing those nebulous reasons
> > where `:::` would be tolerated inside a package. Having more transparent
> > criteria for submitting to CRAN would be really helpful to the entire R
> > community and probably also reduce the traffic on this mailing list.
> >
> > Best,
> > David
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-package-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] CRAN, API packages, Keys and tests
On Fri, Sep 4, 2020 at 9:20 AM Joshua Ulrich wrote: > > Hi Rainer, > > You could set an environment variable with the API key, and only run > the tests if the variable is set. > Sorry, I missed that you're already doing this. I confirm that it works for CRAN. I also set my API key as a private/secure variable on TravisCI, so the tests will run there. IIRC, testthat does something similar. You could look at that pattern, since it works on CRAN for many packages. > Here's an example that I use: > https://github.com/joshuaulrich/quantmod/blob/master/tests/test_getSymbols.R#L4 > > Best, > Josh > > On Fri, Sep 4, 2020 at 3:25 AM Rainer M Krug wrote: > > > > I know this has been asked a few times - but I can’t find anything which > > might help me. > > > > I have a package (https://github.com/rkrug/ROriginStamp) which wraps around > > API calls to OriginStamp (https://originstamp.com) to obtain trusted time > > stamps. > > > > The package works, but the tests are the problem on remote machines. > > > > My question is: > > > > 1) I have disabled all tests and all examples if there is no environmental > > variable set with the API Key - is this OK for CRAN? > > 2) If not, how can I make this CRAN compliant? > > > > Thanks, > > > > Rainer > > > > > > -- > > Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, > > UCT), Dipl. Phys. (Germany) > > > > Orcid ID: -0002-7490-0066 > > > > Department of Evolutionary Biology and Environmental Studies > > University of Zürich > > Office Y34-J-74 > > Winterthurerstrasse 190 > > 8075 Zürich > > Switzerland > > > > Office: +41 (0)44 635 47 64 > > Cell: +41 (0)78 630 66 57 > > email: rainer.k...@uzh.ch > > rai...@krugs.de > > Skype: RMkrug > > > > PGP: 0x0F52F982 > > > > > > > > > > > > -- > > Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, > > UCT), Dipl. Phys. (Germany) > > > > Orcid ID: -0002-7490-0066 > > > > Department of Evolutionary Biology and Environmental Studies > > University of Zürich > > Office Y34-J-74 > > Winterthurerstrasse 190 > > 8075 Zürich > > Switzerland > > > > Office: +41 (0)44 635 47 64 > > Cell: +41 (0)78 630 66 57 > > email: rainer.k...@uzh.ch > > rai...@krugs.de > > Skype: RMkrug > > > > PGP: 0x0F52F982 > > > > __ > > R-package-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-package-devel > > > > -- > Joshua Ulrich | about.me/joshuaulrich > FOSS Trading | www.fosstrading.com -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] CRAN, API packages, Keys and tests
Hi Rainer, You could set an environment variable with the API key, and only run the tests if the variable is set. Here's an example that I use: https://github.com/joshuaulrich/quantmod/blob/master/tests/test_getSymbols.R#L4 Best, Josh On Fri, Sep 4, 2020 at 3:25 AM Rainer M Krug wrote: > > I know this has been asked a few times - but I can’t find anything which > might help me. > > I have a package (https://github.com/rkrug/ROriginStamp) which wraps around > API calls to OriginStamp (https://originstamp.com) to obtain trusted time > stamps. > > The package works, but the tests are the problem on remote machines. > > My question is: > > 1) I have disabled all tests and all examples if there is no environmental > variable set with the API Key - is this OK for CRAN? > 2) If not, how can I make this CRAN compliant? > > Thanks, > > Rainer > > > -- > Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, > UCT), Dipl. Phys. (Germany) > > Orcid ID: -0002-7490-0066 > > Department of Evolutionary Biology and Environmental Studies > University of Zürich > Office Y34-J-74 > Winterthurerstrasse 190 > 8075 Zürich > Switzerland > > Office: +41 (0)44 635 47 64 > Cell: +41 (0)78 630 66 57 > email: rainer.k...@uzh.ch > rai...@krugs.de > Skype: RMkrug > > PGP: 0x0F52F982 > > > > > > -- > Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, > UCT), Dipl. Phys. (Germany) > > Orcid ID: -0002-7490-0066 > > Department of Evolutionary Biology and Environmental Studies > University of Zürich > Office Y34-J-74 > Winterthurerstrasse 190 > 8075 Zürich > Switzerland > > Office: +41 (0)44 635 47 64 > Cell: +41 (0)78 630 66 57 > email: rainer.k...@uzh.ch > rai...@krugs.de > Skype: RMkrug > > PGP: 0x0F52F982 > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] visible binding for '<<-' assignment
On Thu, Sep 3, 2020 at 4:36 PM Ben Bolker wrote:
>
> Is there a reason that this slightly more explicit version wouldn't work?
>
> pali_string_fix <- function() {
> assign("pali_alphabet", stringi::stri_unescape_unicode(pali_alphabet),
> .GlobalEnv)
> }
>
Using assign will also cause R CMD check to throw a NOTE that you will
need to explain upon pkg submission to CRAN.
>
> On 9/3/20 5:25 PM, Dan Zigmond wrote:
> > Thanks, Gabor. I want these to be easily available to package users though
> > – that's why they are in the package. So I would rather not "hide" them in
> > a local environment. This is fundamentally a data package, so access to
> > this data is the primary point of installing it.
> >
> > Is there any other solution?
> >
> > Dan
> >
> > .
> > --
> > Dan Zigmond
> > d...@shmonk.com
> >
> >
> >
> > On Thu, Sep 3, 2020 at 1:40 PM Gábor Csárdi wrote:
> >
> >> Store the cached data in an environment within the package:
> >>
> >> pali_data <- new.env(parent = emptyenv())
> >>
> >> pali_string_fix <- function() {
> >>pali_data$alphabet <-
> >> stringi::stri_unescape_unicode(pali_alphabet)
> >> ...
> >> }
> >>
> >> Gabor
> >>
> >> On Thu, Sep 3, 2020 at 9:33 PM Dan Zigmond wrote:
> >>>
> >>> Hi, all. I am developing a package that includes some global variables.
> >>> Because these are non-ASCII, I have escaped them. But then because these
> >>> are difficult to read, I want to provide an easy way for users to
> >> unescape
> >>> all of them up front. Thus I have code like to create and save the data
> >> in
> >>> global variables in one file:
> >>>
> >>> pali_vowels <-
> >>>c("a", "\u0101", "i", "\u012b", "u", "\u016b", "e", "o")
> >>> pali_consonants <-
> >>>c("k", "kh", "g", "gh", "\u1e45",
> >>> "c", "ch", "j", "jh", "\u00f1",
> >>> "\u1e6d", "\u1e6dh", "\u1e0d", "\u1e0dh", "\u1e47",
> >>> "t", "th", "d", "dh", "n",
> >>> "p", "ph", "b", "bh", "m",
> >>> "y", "r", "l", "v", "s", "h", "\u1e37", "\u1e43")
> >>> pali_alphabet <-c(pali_vowels, pali_consonants)
> >>> use_data(pali_alphabet, overwrite = TRUE)
> >>>
> >>> and then I try to export a function like this in another file:
> >>>
> >>> pali_string_fix <- function() {
> >>>pali_alphabet <<-
> >>> stringi::stri_unescape_unicode(pali_alphabet)
> >>># Several more of these...
> >>>}
> >>>
> >>> The idea is that users can run pali_string_fix() once when they load the
> >>> package and then they won't need to deal with all the Unicode escape
> >>> sequences after that.
> >>>
> >>> However, this is getting rejected by the CRAN checks with the message:
> >>>
> >>> * checking R code for possible problems ... [4s] NOTE
> >>> pali_string_fix: no visible binding for '<<-' assignment to
> >>>'pali_alphabet'
> >>>
> >>> I'm guessing this is because the data and the function are defined in
> >>> different files, so even though those globals are defined by my package,
> >>> that isn't obvious when the check is run on this code.
> >>>
> >>> Does anyone have advice for how to fix this?
> >>>
> >>> Dan
> >>>
> >>> .
> >>> -
> >>> Dan Zigmond
> >>> d...@shmonk.com
> >>>
> >>> [[alternative HTML version deleted]]
> >>>
> >>> __
> >>> R-package-devel@r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >>
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-package-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] url checks not working for me
l.pone.0142533 > > > (moved to > > > > > https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0142533) > > > From: inst/CITATION Status: 200 Message: OK URL: https://geneactiv.org/ > > > (moved to https://www.activinsights.com/) From: inst/doc/GGIR.html > > Status: > > > 200 Message: OK URL: https://www.accelting.com/ (moved to > > > https://accelting.com/) From: inst/doc/GGIR.html Status: 200 Message: OK > > > URL: https://www.physiology.org/doi/10.1152/japplphysiol.00421.2014 > > (moved > > > to https://journals.physiology.org/doi/10.1152/japplphysiol.00421.2014) > > > From: inst/CITATION Status: 200 Message: OK > > > > > > [[alternative HTML version deleted]] > > > > > > __ > > > R-package-devel@r-project.org mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-package-devel > > > > > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Etiquette for package submissions that do not automatically pass checks?
On Fri, Aug 14, 2020 at 2:54 PM Duncan Murdoch wrote:
>
> On 14/08/2020 3:08 p.m., Cesko Voeten wrote:
> > A while ago, I submitted an update to my package 'buildmer' that does not
> > pass R CMD check. This is deliberate. The package contains functionality to
> > run on cluster nodes that were set up by the user and needs to access its
> > own internal functions from there. In previous versions of the package, I
> > had maintained a list of those functions and clusterExport()ed them, but
> > that had the side effect of overwriting any same-named user objects on the
> > user-provided cluster nodes, which I thought was poor form. The update
> > therefore accesses these functions using ':::', which triggers a check
> > warning.
> >
> > I thought the etiquette was to explain this in the 'Comments' box when
> > submitting, but this gave me the same automated message that the package
> > does not pass checks and that I should fix it or reply-all and explain.
> > This led me to believe that I should not have used the 'Comments' box for
> > this purpose, hence I resubmitted the package leaving the comments box
> > empty, and I replied-all to the subsequent e-mail I got with an explanation
> > similar to the above.
>
> It seems to me that what you should have done is "reply-all and
> explain", as the automated message said.
>
> > It has now been a while since I sent that e-mail (ten days), and I have yet
> > to hear back. I was wondering if the message had gotten lost, if they
> > simply haven't gotten around to it yet (I have no idea how much mail they
> > receive on a daily basis, but I'd think it's a lot more than I do), or if I
> > should have handled this differently. Only CRAN can answer the first two
> > questions, but before I bother them: was this the correct procedure, or
> > should I simply have done something differently?
> >
>
> You can see the state of your submission using the foghorn package.
> cran_incoming("buildmer") currently shows your package is in the
> "archive", which means "package rejected: it does not pass the checks
> cleanly and the problems are unlikely to be false positives".
>
> I only see version 1.7 there, which may indicate that you resubmitted
> exactly the same package (down to the version number). As the
> instructions at
> https://cran.r-project.org/web/packages/policies.html#Re_002dsubmission
> say, "Increasing the version number at each submission reduces confusion
> so is preferred even when a previous submission was not accepted."
>
> What I'd suggest now is that you do nothing more for a day or two,
> because CRAN members who aren't on holiday might read and respond to
> your message. If you don't hear anything, then I'd start over again,
> with a new version number, and an explanation in the comments, and
> likely a followup reply-all.
>
You have more than a few days. As it says on CRAN:
"CRAN submission is offline from Aug 14 to Aug 24, 2020 (CRAN team
vacation and maintainance work)"
> Alternatively, you could export those troublesome functions from your
> package but document them as for internal use only. Renaming them with
> a name starting with "." will make them harder for users to stumble
> upon, but you can still access them using buildmer::.something, you
> shouldn't need clusterExport(). Then you will meet the technical
> requirement and not need any explanation.
>
> Duncan Murdoch
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] help troubleshooting travis-ci issues?
On Sun, Aug 4, 2019 at 4:54 PM Ben Bolker wrote: > > On 2019-08-04 5:43 p.m., Iñaki Ucar wrote: > > On Sun, 4 Aug 2019 at 23:26, Ben Bolker wrote: > >> > >> > >> I'm having some trouble with travis builds of a package (local checks > >> work fine) and hoping if I can get suggestions for diagnosis & > >> troubleshooting ... > >> > >> The package is 'broom.mixed'; I hadn't changed it in a while, but > >> recently got a minor pull request which seemed reasonable. I believe the > >> problems I'm having now are dependency and/or caching problems, not > >> actually related to the changes in the package. > >> > >> The failing build is here > >> > >> https://travis-ci.org/bbolker/broom.mixed/builds/567597217 > > > > I see various "package x was installed by an R version with different > > internals; it needs to be reinstalled for use with this R version". > > > > Top of the page, right side, you should see More options > Caches. Try > > removing them and restart the build. > > > > Iñaki > > I tried that already, thanks. That's why I'm now suspecting it's some > kind of mismatch between the Rutter PPAs and the default (?) repos that > Travis uses. It would be a little painful, but I could edit out the > rstan dependency and try the builds with and without Rutter's PPA to > test that hypothesis ... > Try changing: sudo add-apt-repository -y "ppa:marutter/rrutter" sudo add-apt-repository -y "ppa:marutter/c2d4u" to: sudo add-apt-repository -y "ppa:marutter/rrutter3.5" sudo add-apt-repository -y "ppa:marutter/c2d4u3.5" The R C API changed between R-3.4.x and R-3.5.x, which created incompatibility in packages installed using different versions. Any R > 3.6.x should use the "ppa:*3.5" PPAs. > cheers > Ben Bolker > > > > > > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] R-Forge > GitHub?
On Sat, Jun 29, 2019 at 9:46 PM Joshua Ulrich wrote: > > On Sat, Jun 29, 2019 at 6:06 PM Spencer Graves > wrote: > > > > Hi, Henrik et al.: > > > > > >What's your favorite documentation on how to make two GitHub > > projects from one containing two packages? > > > > > >Currently, "github.com/sbgraves237/Ecdat" consists primarily of a > > directory "pkg" with subdirectories "Ecdat" and "Ecfun" containing the > > two packages. I need to know how to do the following: > > > > > > 1. Extract "github.com/sbgraves237/Ecdat/pkg/Ecfun" to > > create "github.com/sbgraves237/Ecfun". > > > > > > 2. Elevate "github.com/sbgraves237/Ecdat/pkg/Ecdat" to > > "github.com/sbgraves237/Ecdat", discarding the other files in the > > original "github.com/sbgraves237/Ecdat/". > > > > > >This sounds like it could be accomplished relatively easily by > > someone with sufficient understanding of "git" and GitHub. I could use > > suggestions on how to do this -- or at least on how to find > > documentation on how to do this. > > > This is straightforward if the two packages have always been in the > same directory structure. But it doesn't look like that's the case > for your repository, since Ecfun was added around r125. So Ecdat's > history is split and would need to be grafted together. I've done > this for other packages. So it's possible, but it took me some trial > and error. > > I'm giving that a go right now. I'll report back tomorrow morning, at > the latest. > I imported both packages into separate repositories: https://github.com/joshuaulrich/tmp-ecdat https://github.com/joshuaulrich/tmp-ecfun I changed your email address on your R-Forge commits to match your GitHub email address, so R-Forge commits would be associated with your GitHub account. I also omitted the "move" commit from Ecdat, and the "obsolete > GitHub" commits from both packages. I've attached a file with the commands I used, if anyone is interested. You can use my repos by cloning them to your local machine, adding your repos as new remotes, and pushing to them. You would need to run these commands (untested): ### clone my GitHub repo to your machine git clone g...@github.com:joshuaulrich/tmp-ecfun.git Ecdat cd Ecdat ### rename my GitHub repo remote from 'origin' to 'tmp' git remote rename origin tmp ### add your GitHub repo remote as 'origin' ### NOTE: this should be a new, clean repo. ###Rename your existing 'Ecdat' so you don't overwrite it git remote add origin https://github.com/sbgraves237/Ecdat ### push to your GitHub repo git push -u origin master Then you need to run similar commands for Ecfun. Best, Josh > > > >Thanks, > >Spencer > > > > > > On 2019-06-29 14:09, Henrik Bengtsson wrote: > > > On Sat, Jun 29, 2019 at 9:43 AM Spencer Graves > > > wrote: > > >> Hi, Ott et al.: > > >> > > >> > > >> What's the best way to get "Travis CI" to build and test the two > > >> packages, Ecdat and Ecfun, that have long been combined in the Ecdat > > >> project? > > >> > > >> > > >> Following Ott's advice and studying studying Wickham's "R > > >> Packages" (http://r-pkgs.had.co.nz/), I was able to configure RStudio so > > >> it would sync using git with "GitHub.com/sbgraves237/Ecdat". However, > > >> when I tried to configure "Travis CI", it said, "No DESCRIPTION file > > >> found, user must supply their own install and script steps". > > >> > > >> > > >> Earlier in this thread, I think someone suggested I make the > > >> Ecdat and Ecfun packages separate projects on GitHub (though I can't > > >> find that suggestion now). This would not be an issue if it were all > > >> local without version control. With RStudio managing my interface with > > >> GitHub, it now seems quite tricky. > > > I'm 99.999% confident that your life will be much much easier if you > > > keep one R package per repository. If you don't, you'll probably be > > > very lonely when it comes to tools etc. There are built-in 'git' > > > commands, but also git utility tools, for extracting a subset of > > > folders/files from git repository into new git repositories. You'll > > > still preserve the commit history. I would deal with this in the > >
Re: [Rd] R-Forge > GitHub?
> >> > >> > >> Spencer > >> > >> > >> On 2019-06-28 22:25, Ott Toomet wrote: > >>> Apparently your username/password are wrong. Can you clone/push from > >>> other repos? > >>> > >>> You do not need authorization when cloning a public repo, so even > >>> incorrect credentials may work (haven't tested this though). But for > >>> push you have to have that in order. > >>> > >>> I suggest you create ssh keys, upload those to GH, and use ssh > >>> authorization instead of https. > >>> > >>> Cheers, > >>> Ott > >>> > >>> On Fri, Jun 28, 2019 at 8:18 PM Spencer Graves > >>> mailto:spencer.gra...@prodsyse.com>> wrote: > >>> > >>> Thanks to Duncan, Henrik and Henrik, Brian, and Gábor: > >>> > >>> > >>> I created a local copy of the new GitHub version using the > >>> following: > >>> > >>> git clone > >>> https://sbgraves237:mypassw...@github.com/sbgraves237/Ecdat.git > >>> > >>> > >>> > >>> That worked in the sense that I got a local copy. However, > >>> after > >>> I rolled the version number and did "git commit" on the DESCRIPTION > >>> files, my "git push" command generated the following: > >>> > >>> > >>> remote: Invalid username or password. > >>> fatal: Authentication failed for > >>> 'https://sbgraves237:mypassw...@github.com/sbgraves237/Ecdat.git/' > >>> > >>> > >>> What am I missing? [Note: I used my actual GitHub > >>> password in > >>> place of "mypassword" here, and this "Authentication failed" message > >>> reported the GitHub password I used here.] > >>> > >>> > >>> Thanks, > >>> Spencer > >>> > >>> > >>> p.s. I'm doing this under macOS Mojave 10.14.5. Also, I added > >>> ".onAttach" functions to the R-Forge versions as Brian G. Peterson > >>> suggested. That seemed to work fine. > >>> > >>> > >>> On 2019-06-28 07:13, Duncan Murdoch wrote: > >>> > On 28/06/2019 6:26 a.m., Gábor Csárdi wrote: > >>> > > >>> >> Instead, you can do as Duncan suggested, and put a README in your > >>> >> R-Forge > >>> >> repository, that points to *your* GitHub repositor(y/ies). Then > >>> the > >>> >> https://github.com/rforge/ecdat read only mirror will pick this up > >>> >> and will > >>> >> point there as well. > >>> > > >>> > Just for the record: that was Henrik Singmann's suggestion, I just > >>> > agreed with it. > >>> > > >>> > Duncan Murdoch > >>> > > >>> > >>> > >>> [[alternative HTML version deleted]] > >>> > >>> __ > >>> R-devel@r-project.org <mailto:R-devel@r-project.org> mailing list > >>> https://stat.ethz.ch/mailman/listinfo/r-devel > >>> > >> > >> [[alternative HTML version deleted]] > >> > >> __ > >> R-devel@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
I thought it would be good to summarize my thoughts, since I made a few hypotheses that turned out to be false. This isn't a bug in base R, in either rbind() or `[<-.Date`. To summarize the root cause: base::rbind.data.frame() calls `[<-` for each column of the data.frame, and there is no `[<-.IDate` method to ensure the replacement value is converted to integer. And, in fact, `[<-.Date` calls as.Date() and data.table::as.Date.IDate() calls as.numeric() on the IDate object. So the problem exists, and can be fixed, in data.table. Best, Josh On Mon, May 27, 2019 at 9:34 AM Joshua Ulrich wrote: > > Follow-up (inline) on my comment about a potential issue in `[<-.Date`. > > On Mon, May 27, 2019 at 9:31 AM Michael Chirico > wrote: > > > > Yes, thanks for following up on thread here. And thanks again for clearing > > things up, your email was a finger snap of clarity on the whole issue. > > > > I'll add that actually it was data.table's code at fault on the storage > > conversion -- note that if you use an arbitrary sub-class 'foo' with no > > methods defined, it'll stay integer. > > > > That's because [<- calls as.Date and then as.Date.IDate, and that method > > (ours) has as.numeric(); earlier I had recognized that if we commented that > > line, the issue was "fixed" but I still wasn't understanding the root cause. > > > > My last curiosity on this issue will be in my follow-up thread. > > > > Mike C > > > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich > > wrote: > >> > >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich > >> wrote: > >> > > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > >> > wrote: > >> > > > >> > > Have finally managed to come up with a fix after checking out > >> > > sys.calls() > >> > > from within the as.Date.IDate debugger, which shows something like: > >> > > > >> > > [[1]] rbind(DF, DF) > >> > > [[2]] rbind(deparse.level, ...) > >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > >> > > [[5]] as.Date(value) > >> > > [[6]] as.Date.IDate(value) > >> > > > >> > > I'm not sure why [<- is called, I guess the implementation is to > >> > > assign to > >> > > the output block by block? Anyway, we didn't have a [<- method. And > >> > > [<-.Date looks like: > >> > > > >> > > value <- unclass(as.Date(value)) # <- converts to double > >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class > >> > > > >> > > So we can fix our bug by defining a [<- class; the question that I > >> > > still > >> > > don't see answered in documentation or source code is, why/where is [<- > >> > > called, exactly? > >> > > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > >> > `[<-` call is this line: > >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > > >> > That's where the storage.mode changes from integer to double. > >> > > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > >> > Browse[2]> > >> > debug: xij > >> > Browse[2]> storage.mode(xij) > >> > [1] "integer" > >> > Browse[2]> value[[jj]][ri] > >> > [1] "2019-05-26" > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "integer" > >> > Browse[2]> > >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > >> > Browse[2]> storage.mode(value[[jj]][ri]) > >> > [1] "double" > >> > > >> To be clear, I don't think this is a bug in rbind() or > >> rbind.data.frame(). The confusion is that rbind.data.frame() calls > >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` > >> method. So the parent class method is dispatched, which converts the > >> storage mode to double. > >> > >> Someone may argue that this is an issue with `[<-.Date`, and that it > >> shouldn't convert the storage.mode from integer to double. > > I don't think this is an issue. The storage mode isn't converted if > the replacement is the same stora
Re: [Rd] survival changes
On Sat, Jun 1, 2019 at 5:22 AM Therneau, Terry M., Ph.D. via R-devel wrote: > > In the next version of the survival package I intend to make a non-upwardly > compatable > change to the survfit object. With over 600 dependent packages this is not > something to > take lightly, and I am currently undecided about the best way to go about it. > I'm looking > for advice. > I encountered several issues like this while making a change to xts. I encountered several buggy and inconsistent behaviors while moving attributes from the xts object to the xts object's index attribute. I don't want to "fix" some of these in the next version, because downstream packages may rely on the old behavior. Though it's specific to Python, "API Evolution the Right Way" has many good strategies. https://emptysqua.re/blog/api-evolution-the-right-way/ The "Deleting Features" and "Changing Behavior" sections are most relevant to your situation. > The change: 20+ years ago I had decided not to include the initial x=0,y=1 > data point in > the survfit object itself. It was not formally an estimand and the > plot/points/lines etc > routines could add this on themselves. That turns out to have been a > mistake, and has led > to a steady proliferation of extra bits as I realized that the time axis > doesn't always > start at 0, and later (with multi state) that y does not always start at 1 > (though the > states sum to 1), and later the the error doesn't always start at 0, and > another > realization with cumulative hazard, and ... > The new survfit method for multi-state coxph models was going to add yet > another special > case. Basically every component is turning into a duplicate of "row 1" vs > "all the > others". (And inconsistently named.) > > Three possible solutions > 1. Current working draft of survival_3.0.3: Add a 'version' element to the > survfit object > and a 'survfit2.3' function that converts old to new. All my downstream > functions (print, > plot,...) start with an "if (old) update to new" line. This has allowed me > to stage > updates to the functions that create survfit objects -- I expect it to happen > slowly. > There will also be a survfit3.2 function to go backwards. Both the forward > and backwards > functions leave objects alone if they are currently in the desired format. > This seems reasonable. It would also give you opportunity to warn users if functions that expect a new object receive an old object. That would help them convert any of their functions that rely on the old structure/behavior. You could also add a global option to enable warnings with the default set to FALSE for the first release. Let users know they can set that option to TRUE to identify places where they may need to modify their use cases. > 2. Make a new class "survfit3" and the necessary 'as' functions. The package > would contain > plot.survfit and plot.survfit3 methods, the former a two line "convert and > call the > second" function. > Rather than an entirely new class, I wonder if you could instead make the new class a subclass. I'm not sure whether this has advantages over adding an element, but it's another possibility. > 3. Something I haven't thought of. > I don't have any other ideas, but I would be happy to discuss offline if that would be helpful. > Number 2 has a cleanness about it, but there is a long term nuisance about it > wrt > documentation. Users, not unreasonably, expect the survfit function to > produce a survfit > object, and that is what they look for in the help pages. > > I plan to have 3.0-x on github before userR so that users can begin to play > with it (and > to get feeback before pushing to CRAN), so need to make a decision. > > Terry T. > > > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2019 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
Follow-up (inline) on my comment about a potential issue in `[<-.Date`. On Mon, May 27, 2019 at 9:31 AM Michael Chirico wrote: > > Yes, thanks for following up on thread here. And thanks again for clearing > things up, your email was a finger snap of clarity on the whole issue. > > I'll add that actually it was data.table's code at fault on the storage > conversion -- note that if you use an arbitrary sub-class 'foo' with no > methods defined, it'll stay integer. > > That's because [<- calls as.Date and then as.Date.IDate, and that method > (ours) has as.numeric(); earlier I had recognized that if we commented that > line, the issue was "fixed" but I still wasn't understanding the root cause. > > My last curiosity on this issue will be in my follow-up thread. > > Mike C > > On Mon, May 27, 2019, 10:25 PM Joshua Ulrich wrote: >> >> On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich >> wrote: >> > >> > On Sun, May 26, 2019 at 4:06 AM Michael Chirico >> > wrote: >> > > >> > > Have finally managed to come up with a fix after checking out sys.calls() >> > > from within the as.Date.IDate debugger, which shows something like: >> > > >> > > [[1]] rbind(DF, DF) >> > > [[2]] rbind(deparse.level, ...) >> > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) >> > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) >> > > [[5]] as.Date(value) >> > > [[6]] as.Date.IDate(value) >> > > >> > > I'm not sure why [<- is called, I guess the implementation is to assign >> > > to >> > > the output block by block? Anyway, we didn't have a [<- method. And >> > > [<-.Date looks like: >> > > >> > > value <- unclass(as.Date(value)) # <- converts to double >> > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class >> > > >> > > So we can fix our bug by defining a [<- class; the question that I still >> > > don't see answered in documentation or source code is, why/where is [<- >> > > called, exactly? >> > > >> > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The >> > `[<-` call is this line: >> > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij >> > >> > That's where the storage.mode changes from integer to double. >> > >> > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij >> > Browse[2]> >> > debug: xij >> > Browse[2]> storage.mode(xij) >> > [1] "integer" >> > Browse[2]> value[[jj]][ri] >> > [1] "2019-05-26" >> > Browse[2]> storage.mode(value[[jj]][ri]) >> > [1] "integer" >> > Browse[2]> >> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm >> > Browse[2]> storage.mode(value[[jj]][ri]) >> > [1] "double" >> > >> To be clear, I don't think this is a bug in rbind() or >> rbind.data.frame(). The confusion is that rbind.data.frame() calls >> `[<-` for each column of the data.frame, and there is no `[<-.IDate` >> method. So the parent class method is dispatched, which converts the >> storage mode to double. >> >> Someone may argue that this is an issue with `[<-.Date`, and that it >> shouldn't convert the storage.mode from integer to double. I don't think this is an issue. The storage mode isn't converted if the replacement is the same storage mode. For example: R> x <- .Date(1:5) R> storage.mode(x) [1] "integer" R> x[1L] <- .Date(0L) R> storage.mode(x) [1] "integer" R> x[1L] <- .Date(0) R> storage.mode(x) [1] "double" >> > >> > > Mike C >> > > >> > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico >> > > >> > > wrote: >> > > >> > > > Debugging this issue: >> > > > >> > > > https://github.com/Rdatatable/data.table/issues/2008 >> > > > >> > > > We have custom class 'IDate' which inherits from 'Date' (it just forces >> > > > integer storage for efficiency, hence, I). >> > > > >> > > > The concatenation done by rbind, however, breaks this and returns a >> > > > double: >> > > > >> > > > library(data.table) >> > > > DF = data.frame(date = as.IDate(Sys.Date())) >> > > > storage.mode(rbind(
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
On Sun, May 26, 2019 at 6:47 AM Joshua Ulrich wrote: > > On Sun, May 26, 2019 at 4:06 AM Michael Chirico > wrote: > > > > Have finally managed to come up with a fix after checking out sys.calls() > > from within the as.Date.IDate debugger, which shows something like: > > > > [[1]] rbind(DF, DF) > > [[2]] rbind(deparse.level, ...) > > [[3]] `[<-`(`*tmp*`, ri, value = 18042L) > > [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L) > > [[5]] as.Date(value) > > [[6]] as.Date.IDate(value) > > > > I'm not sure why [<- is called, I guess the implementation is to assign to > > the output block by block? Anyway, we didn't have a [<- method. And > > [<-.Date looks like: > > > > value <- unclass(as.Date(value)) # <- converts to double > > .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class > > > > So we can fix our bug by defining a [<- class; the question that I still > > don't see answered in documentation or source code is, why/where is [<- > > called, exactly? > > > Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The > `[<-` call is this line: > value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > > That's where the storage.mode changes from integer to double. > > debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij > Browse[2]> > debug: xij > Browse[2]> storage.mode(xij) > [1] "integer" > Browse[2]> value[[jj]][ri] > [1] "2019-05-26" > Browse[2]> storage.mode(value[[jj]][ri]) > [1] "integer" > Browse[2]> > debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm > Browse[2]> storage.mode(value[[jj]][ri]) > [1] "double" > To be clear, I don't think this is a bug in rbind() or rbind.data.frame(). The confusion is that rbind.data.frame() calls `[<-` for each column of the data.frame, and there is no `[<-.IDate` method. So the parent class method is dispatched, which converts the storage mode to double. Someone may argue that this is an issue with `[<-.Date`, and that it shouldn't convert the storage.mode from integer to double. > > > Mike C > > > > On Sun, May 26, 2019 at 1:16 PM Michael Chirico > > wrote: > > > > > Debugging this issue: > > > > > > https://github.com/Rdatatable/data.table/issues/2008 > > > > > > We have custom class 'IDate' which inherits from 'Date' (it just forces > > > integer storage for efficiency, hence, I). > > > > > > The concatenation done by rbind, however, breaks this and returns a > > > double: > > > > > > library(data.table) > > > DF = data.frame(date = as.IDate(Sys.Date())) > > > storage.mode(rbind(DF, DF)$date) > > > # [1] "double" > > > > > > This is specific to base::rbind (data.table's rbind returns an integer as > > > expected); in ?rbind we see: > > > > > > The method dispatching is not done via UseMethod(), but by C-internal > > > dispatching. Therefore there is no need for, e.g., rbind.default. > > > The dispatch algorithm is described in the source file > > > (‘.../src/main/bind.c’) as > > > 1. For each argument we get the list of possible class memberships from > > > the class attribute. > > > 2. *We inspect each class in turn to see if there is an applicable > > > method.* > > > 3. If we find an applicable method we make sure that it is identical to > > > any method determined for prior arguments. If it is identical, we proceed, > > > otherwise we immediately drop through to the default code. > > > > > > It's not clear what #2 means -- an applicable method *for what*? Glancing > > > at the source code would suggest it's looking for rbind.IDate: > > > > > > https://github.com/wch/r-source/blob/trunk/src/main/bind.c#L1051-L1063 > > > > > > const char *generic = ((PRIMVAL(op) == 1) ? "cbind" : "rbind"); // should > > > be rbind here > > > const char *s = translateChar(STRING_ELT(classlist, i)); // iterating over > > > the classes, should get to IDate first > > > sprintf(buf, "%s.%s", generic, s); // should be rbind.IDate > > > > > > but adding this method (or even exporting it) is no help [ simply defining > > > rbind.IDate = function(...) as.IDate(NextMethod()) ] > > > > > > Lastly, it appears that as.Date.IDate is called, which is causing the type > > > conversion: > > > > > > debug(data.table
Re: [Rd] rbind has confusing result for custom sub-class (possible bug?)
On Sun, May 26, 2019 at 4:06 AM Michael Chirico
wrote:
>
> Have finally managed to come up with a fix after checking out sys.calls()
> from within the as.Date.IDate debugger, which shows something like:
>
> [[1]] rbind(DF, DF)
> [[2]] rbind(deparse.level, ...)
> [[3]] `[<-`(`*tmp*`, ri, value = 18042L)
> [[4]] `[<-.Date`(`*tmp*`, ri, value = 18042L)
> [[5]] as.Date(value)
> [[6]] as.Date.IDate(value)
>
> I'm not sure why [<- is called, I guess the implementation is to assign to
> the output block by block? Anyway, we didn't have a [<- method. And
> [<-.Date looks like:
>
> value <- unclass(as.Date(value)) # <- converts to double
> .Date(NextMethod(.Generic), oldClass(x)) # <- restores 'IDate' class
>
> So we can fix our bug by defining a [<- class; the question that I still
> don't see answered in documentation or source code is, why/where is [<-
> called, exactly?
>
Your rbind(DF, DF) call dispatches to base::rbind.data.frame(). The
`[<-` call is this line:
value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij
That's where the storage.mode changes from integer to double.
debug: value[[jj]][ri] <- if (is.factor(xij)) as.vector(xij) else xij
Browse[2]>
debug: xij
Browse[2]> storage.mode(xij)
[1] "integer"
Browse[2]> value[[jj]][ri]
[1] "2019-05-26"
Browse[2]> storage.mode(value[[jj]][ri])
[1] "integer"
Browse[2]>
debug: if (!is.null(nm <- names(xij))) names(value[[jj]])[ri] <- nm
Browse[2]> storage.mode(value[[jj]][ri])
[1] "double"
> Mike C
>
> On Sun, May 26, 2019 at 1:16 PM Michael Chirico
> wrote:
>
> > Debugging this issue:
> >
> > https://github.com/Rdatatable/data.table/issues/2008
> >
> > We have custom class 'IDate' which inherits from 'Date' (it just forces
> > integer storage for efficiency, hence, I).
> >
> > The concatenation done by rbind, however, breaks this and returns a double:
> >
> > library(data.table)
> > DF = data.frame(date = as.IDate(Sys.Date()))
> > storage.mode(rbind(DF, DF)$date)
> > # [1] "double"
> >
> > This is specific to base::rbind (data.table's rbind returns an integer as
> > expected); in ?rbind we see:
> >
> > The method dispatching is not done via UseMethod(), but by C-internal
> > dispatching. Therefore there is no need for, e.g., rbind.default.
> > The dispatch algorithm is described in the source file
> > (‘.../src/main/bind.c’) as
> > 1. For each argument we get the list of possible class memberships from
> > the class attribute.
> > 2. *We inspect each class in turn to see if there is an applicable
> > method.*
> > 3. If we find an applicable method we make sure that it is identical to
> > any method determined for prior arguments. If it is identical, we proceed,
> > otherwise we immediately drop through to the default code.
> >
> > It's not clear what #2 means -- an applicable method *for what*? Glancing
> > at the source code would suggest it's looking for rbind.IDate:
> >
> > https://github.com/wch/r-source/blob/trunk/src/main/bind.c#L1051-L1063
> >
> > const char *generic = ((PRIMVAL(op) == 1) ? "cbind" : "rbind"); // should
> > be rbind here
> > const char *s = translateChar(STRING_ELT(classlist, i)); // iterating over
> > the classes, should get to IDate first
> > sprintf(buf, "%s.%s", generic, s); // should be rbind.IDate
> >
> > but adding this method (or even exporting it) is no help [ simply defining
> > rbind.IDate = function(...) as.IDate(NextMethod()) ]
> >
> > Lastly, it appears that as.Date.IDate is called, which is causing the type
> > conversion:
> >
> > debug(data.table:::as.Date.IDate)
> > rbind(DF, DF) # launches debugger
> > x
> > # [1] "2019-05-26" <-- singleton, so apparently applied to DF$date, not
> > c(DF$date, DF$date)
> > undebug(data.table:::as.Date.IDate)
> >
> > I can't really wrap my head around why as.Date is being called here, and
> > even allowing that, why the end result is still the original class [
> > class(rbind(DF, DF)$date) == c('IDate', 'Date') ]
> >
> > So, I'm beginning to think this might be a bug. Am I missing something?
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2019 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] anyNA() performance on vectors of POSIXct
On Wed, May 1, 2019 at 7:45 AM Harvey Smith wrote:
>
> Inside of the anyNA() function, it will use the legacy any(is.na()) code if
> x is an OBJECT(). If x is a vector of POSIXct, it will be an OBJECT(), but
> it is also TYPEOF(x) == REALSXP. Therefore, it will skip the faster
> ITERATE_BY_REGION, which is typically 5x faster in my testing.
>
> Is the OBJECT() condition really necessary, or could it be moved after the
> switch() for the individual TYPEOF(x) ITERATE_BY_REGION calls?
>
> # script to demonstrate performance difference if x is an OBJECT or not by
> using unclass()
> x.posixct = Sys.time() + 1:1e6
> microbenchmark::microbenchmark(
> any(is.na( x.posixct )),
> anyNA( x.posixct ),
> anyNA( unclass(x.posixct) ),
> unit='ms')
>
>
>
> static Rboolean anyNA(SEXP call, SEXP op, SEXP args, SEXP env)
> {
> SEXP x = CAR(args);
> SEXPTYPE xT = TYPEOF(x);
> Rboolean isList = (xT == VECSXP || xT == LISTSXP), recursive = FALSE;
>
> if (isList && length(args) > 1) recursive = asLogical(CADR(args));
> *if (OBJECT(x) || (isList && !recursive)) {*
> SEXP e0 = PROTECT(lang2(install("is.na"), x));
> SEXP e = PROTECT(lang2(install("any"), e0));
> SEXP res = PROTECT(eval(e, env));
> int ans = asLogical(res);
> UNPROTECT(3);
> return ans == 1; // so NA answer is false.
> }
>
> R_xlen_t i, n = xlength(x);
> switch (xT) {
> case REALSXP:
> {
> if(REAL_NO_NA(x))
> return FALSE;
> ITERATE_BY_REGION(x, xD, i, nbatch, double, REAL, {
> for (int k = 0; k < nbatch; k++)
> if (ISNAN(xD[k]))
> return TRUE;
> });
> break;
> }
>
I'm interested in this as well, because it causes performance
degradation in xts subsetting:
https://github.com/joshuaulrich/xts/issues/296
Would it be possible to special-case POSIXct, and perhaps other types
defined in base+recommended packages?
Best,
Josh
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2019 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] NOTE for a suggested package
You can remove the `library("doParallel")`. Your call to
requireNamespace() plus calling all doParallel functions with `::` is
sufficient. That should resolve the NOTE.
On Sat, Feb 23, 2019 at 5:49 PM Mohammad Ali Nilforooshan
wrote:
>
> I'm struggling to stop receiving a NOTE about a suggested package, and it
> seems to be the problem for many people. Your help is really appreciated.
>
> In the function, I have:
>
> if(requireNamespace("doParallel", quietly=TRUE))
> {
>library("doParallel")
>cl = parallel::makeCluster(ncl)
>doParallel::registerDoParallel(cl)
>Q = foreach(i=ggID, .combine='cbind') %dopar% ## do something
> } else {
>## do something else
> }
>
> In DESCRIPTION, I have:
>
> Suggests:
>doParallel (>= 1.0.14)
>
> This is the NOTE that I receive:
>
> 'library' or 'require' call to 'doParallel' in package code.
> Please use :: or requireNamespace() instead.
> See section 'Suggested packages' in the 'Writing R Extensions' manual.
>
> [[alternative HTML version deleted]]
>
> ______
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] nativeRoutines error when using roxygen2 in RStudio
ves me: > > >>> > > >>> Updating secsse documentation Loading secsse Registered S3 method > > >>> overwritten by 'dplyr': method from as.data.frame.tbl_df tibble > > >>> Registered S3 method overwritten by 'geiger': method from > > >>> unique.multiPhylo ape Error in nativeRoutines[[lib]] <- routines : > > >>> object 'nativeRoutines' not found > > >> > > >> Do you get that error if you run "R CMD check" on the tarball > > of your > > >> package? > > >> > > >> If so, how comfortable are you with debugging R code? I can > > tell you > > >> how to debug the check process, but it's a little tricky. > > >> > > >> Duncan Murdoch > > >> > > >>> > > >>> > > >>> Any other suggestions? > > >>> > > >>> Cheers, Rampal > > >>> > > >>> On 02-Nov-18 08:33, Georgi Boshnakov wrote: > > >>>> Try installing the latest development version of roxygen2, if you > > >>>> are not using it already. > > >>>> There was a bug in the released version preventing > > installation in > > >>>> some cases. > > >>>> If you don't want to use development version of roxygen2, > > >>>> try putting all filenames in the Collate field in DESCRIPTION > > on one > > >>>> line and make sure that there is a single space > > >>>> between them. This should work if your problem is what I think. > > >>>> > > >>>> By the way, in such cases you will get more informative > > messages if > > >>>> you run devtools::check(). > > >>>> > > >>>> -- > > >>>> Georgi Boshnakov > > >>>> > > >>>> > > >>>> > > >>>> From: R-package-devel [r-package-devel-boun...@r-project.org > > <mailto:r-package-devel-boun...@r-project.org>] on > > >>>> behalf of Rampal Etienne [rampaletie...@gmail.com > > <mailto:rampaletie...@gmail.com>] > > >>>> Sent: 01 November 2018 22:40 > > >>>> To: r-package-devel@r-project.org > > <mailto:r-package-devel@r-project.org> > > >>>> Subject: [R-pkg-devel] nativeRoutines error when using > > roxygen2 in > > >>>> RStudio > > >>>> > > >>>> Since a few weeks (after updating R-devel and Rtools) I get the > > >>>> following error when trying to build a package or document > > it, when > > >>>> using roxygen2 in RStudio: > > >>>> > > >>>> In R CMD INSTALL Error in nativeRoutines[[lib]] <- routines : > > object > > >>>> 'nativeRoutines' not found Calls: > > suppressPackageStartupMessages ... > > >>>> withCallingHandlers -> -> load_all -> load_dll > > Execution > > >>>> halted > > >>>> > > >>>> When I disable roxygen2, I do not get this error, but of > > course the > > >>>> documentation is not created. > > >>>> > > >>>> I have installed the latest versions of RStudio, Rtools, R-devel, > > >>>> roxygen2, pkgload, but the problem persists. > > >>>> > > >>>> Does anybody have a clue what is causing this? I am using > > Windows 10, > > >>>> and the package contains Fortran code. > > >>>> > > >>>> Kind regards, > > >>>> Rampal Etienne > > >>>> > > >>>>[[alternative HTML version deleted]] > > >>>> > > >>>> __ > > >>>> R-package-devel@r-project.org > > <mailto:R-package-devel@r-project.org> mailing list > > >>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel > > <https://stat.ethz.ch/mailman/listinfo/r-package-devel> > > >>> > > >>> > > >>> [[alternative HTML version deleted]] > > >>> > > >>> __ > > >>> R-package-devel@r-project.org > > <mailto:R-package-devel@r-project.org> mailing list > > >>> https://stat.ethz.ch/mailman/listinfo/r-package-devel > > <https://stat.ethz.ch/mailman/listinfo/r-package-devel> > > >>> > > >> > > > > > > > __ > > R-package-devel@r-project.org > > <mailto:R-package-devel@r-project.org> mailing list > > https://stat.ethz.ch/mailman/listinfo/r-package-devel > > <https://stat.ethz.ch/mailman/listinfo/r-package-devel> > > > > > > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[Rd] Proposal: Internal as.Date.POSIXct()
I would like to propose a .Internal() POSIXct2Date() function, similar to do_POSIXlt2D() in src/main/datetime.c. I created a working prototype, which is now included in Dirk Eddelbuettel's RApiDatetime package: https://github.com/eddelbuettel/rapidatetime This prototype is a combination of asPOSIXlt and POSIXlt2D from R Core. The functions are essentially concatenated, with the extraneous parts removed. Directly converting from POSIXct to Date uses less memory and is roughly twice as fast. You can find the relevant bits in the commit on GitHub: https://github.com/eddelbuettel/rapidatetime/commit/6bc9320451fb1e62d53b5012d33acf538c0f50c6#diff-7df5c1a674e930a4d93ba45f3ce0aaaf Would someone on R Core be interested in adding this function? If so, I can create a patch that includes all the necessary code additions needed to make it a .Internal() function. Thanks for your consideration! Best, Josh -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Submission to CRAN when package needs personal data (API key)
Hi Rainer, On Wed, Sep 5, 2018 at 2:28 AM, Rainer Krug wrote: > Hi > > I have a package at GitHub (https://github.com/rkrug/ROriginStamp) which I am > pre[paring for CRAN. > > It creates a trusted timestamp using the API fro OriginStamp > (https://originstamp.org/home) which requires an API key. Now this API should > not be made public, as to much traffic through one API key will lead to it’s > blocking. > > I have stored the key encrypted in the travis.yml, and the package passes all > tests. > > But if I send it to CRAN, it would fail the tests, as the api key is not in > the package itself. > > I could disable all tests for CRAN which need the API key, but I think it > would be better tu run the tests there as well (as an additional check to > travis). > > My question: > > Is there a way of storing the API key encrypted, so that only the CRAN test > servers can decrypt it, or is there another way can steal with this? > I have a similar issue with quantmod. I need API keys to test some functionality and I would like the tests run regularly, so I can know when something breaks without having to wait for a user to report the change. I store the API keys in encrypted environment variable in TravisCI, and I check for those environment variables before running the tests that require them. Then I added a cron job on TravisCI to run the build if there hasn't been a build in the past 24 hours. That solves the problem adequately for my purposes without adding any burden to CRAN. Hopefully it works for your purposes too. Best, Josh > Thanks, > > Rainer > > > > -- > Rainer M. Krug, PhD (Conservation Ecology, SUN), > MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) > > University of Zürich > > Cell: +41 (0)78 630 66 57 > email: rai...@krugs.de > Skype: RMkrug > > PGP: 0x0F52F982 > > > > > ______ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Bug in POSIXct string representation?
Hi Andreas,
On Thu, Aug 9, 2018 at 2:26 AM, Festl, Andreas wrote:
> Dear all,
>
> I just have identified the following issue which I believe could be a bug in
> R:
>
> Let me illustrate:
>
> First, enable the display of fractional seconds and check that it works:
>> options(digits.secs = 6, digits = 6)
>> as.character(as.POSIXct("2018-08-31 14:15:16.123456"))
> [1] "2018-08-31 14:15:16.123456"
>
> Now create a sequence of POSIXct with stepwidth 0.1sec:
>> test <- as.POSIXct("2018-08-31 14:15:16.00")
>> test_seq <- seq(test, test + 1, by = 1/10)
>
> Calling format with the millisecond conversion specification gives the
> intended result (even though there is a small representation error):
>> format(test_seq, "%F %T.%OS")
> [1] "2018-08-31 14:15:16.16.00" "2018-08-31 14:15:16.16.09"
> "2018-08-31 14:15:16.16.20" "2018-08-31 14:15:16.16.29"
> [5] "2018-08-31 14:15:16.16.40" "2018-08-31 14:15:16.16.50"
> "2018-08-31 14:15:16.16.59" "2018-08-31 14:15:16.16.70"
> [9] "2018-08-31 14:15:16.16.79" "2018-08-31 14:15:16.16.90"
> "2018-08-31 14:15:17.17.00"
>
> However, if I use as.character, the milliseconds seemingly just get cut-off
> after one digit, resulting in incorrect representations:
>> as.character(test_seq)
> [1] "2018-08-31 14:15:16.0" "2018-08-31 14:15:16.0" "2018-08-31 14:15:16.2"
> "2018-08-31 14:15:16.2" "2018-08-31 14:15:16.4" "2018-08-31 14:15:16.5"
> [7] "2018-08-31 14:15:16.5" "2018-08-31 14:15:16.7" "2018-08-31 14:15:16.7"
> "2018-08-31 14:15:16.9" "2018-08-31 14:15:17.0"
>
> It seems to me, that R correctly decides that there is only one significant
> digit after the decimal point, but then incorrectly (due to representation
> error) just cuts off after the first digit.
>
This is known behavior with how POSIXt objects are printed, and has
been discussed before on R-help:
https://stat.ethz.ch/pipermail/r-help/2015-June/429600.html
Basically, the behavior is a combination of truncating fractional
seconds rather than rounding combined with the floating point
representation error you noticed. And truncation is the behavior for
printing whole seconds:
format(as.POSIXct("2018-08-31 14:15:16.9")) # 16s, not 17s
[1] "2018-08-31 14:15:16"
So it would not be consistent to round fractional seconds, unless you
kept track of the rounding error relative to the desired resolution.
There are more details in the R-help thread and the StackOverflow Q
it references.
Best,
Josh
> BR,
> Andreas
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Memory leakage from large lists
This looks like a case of FAQ 7.42:
https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-is-R-apparently-not-releasing-memory_003f
On Mon, Jul 16, 2018 at 2:32 PM, Daniel Raduta wrote:
> Hello,
>
> I am experiencing a very noticeable memory leak when using large lists of
> large data. The code below creates a list of matrices, yet the memory does
> not free up after removing that item and performing a garbage collection.
> Is there a something I can do to prevent this memory leak? Any help would
> be greatly appreciated. By the way, if you execute the code, please run it
> in a new R session.
>
> # Start of code
>
>
> # Function that returns memory being used
> MemoryUsed <- function(){
> pid <- Sys.getpid()
> system(paste0("top -n 1 -p ", pid, " -b"), intern = TRUE)[c(7,8)]
> }
>
> # Initial memory (VIRT memory equals about 400,000 bytes on my machine)
> MemoryUsed()
>
> # Create a large list of large data, remove it, and perform garbarge
> collection
> ncols <- 100
> nrows <- 1
> mat <- matrix(seq(nrows * ncols), nrow = nrows, ncol = ncols)
> ls <- lapply(1:1000, function(x) as.data.frame(mat))
> rm(list = setdiff(ls(), 'MemoryUsed'))
> invisible(gc())
>
> # Final memory (now, VIRT memory equals about 4,600,000 bytes on my machine)
> MemoryUsed()
>
> # End of code
> ==
>
> My session info is:
>
> R version 3.4.4 (2018-03-15)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 16.04.3 LTS
>
> Matrix products: default
> BLAS: /usr/lib/libblas/libblas.so.3.6.0
> LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
> LC_PAPER=en_US.UTF-8 LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.4.4 tools_3.4.4yaml_2.1.19
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Warnings and error message in CRAN Package Check Results
On Sat, Mar 24, 2018 at 6:42 PM, Wenchao Ma <wenchao2...@gmail.com> wrote:
> Dear Duncan,
>
> Thank you. I really appreciate your help!
>
> Yes, the Fedora warning is self explanatory, but I'm still quite confused.
> Below are part of my Mstep.cpp file (both functions are in the same file).
> The fedora warned me that, among others,
> Mstep.cpp:62:69: warning: explicitly assigning value of variable of type
> 'int' to itself [-Wself-assign]
> arma::vec Pj = Calc_Pj(par = par, designMj = designMj, linkfunc =
> linkfunc, boundary = boundary, eps = eps);
> ^
>
>
> So, basically, I called Calc_Pj function from Calc_Pj_jac function, but not
> sure why this is not allowed.
>
The compiler isn't complaining about the function call. It's warning
about all the assignments you make in the function call.
Like Duncan said, the compiler thinks you have something like "i = i"
in your code. Which you do, for every argument to the function.
You seem to confuse calling functions in R with calling functions in
C++. C++ function arguments do not have names. They are matched by
position. Note that none of the other function calls in the file use
"named arguments".
>
>
> // [[Rcpp::depends(RcppArmadillo)]]
> #include
> using namespace Rcpp;
> using namespace arma;
> // [[Rcpp::export]]
> arma::vec Calc_Pj(const arma::vec par, const arma::mat designMj,
> const int & linkfunc,int boundary = 0,const double eps = 1e-16){
> arma::vec Pj;
> if(linkfunc==1){ //identity
> Pj = designMj*par;
> }else if(linkfunc==2){//logit
> Pj = exp(designMj*par)/(1 + exp(designMj*par));
> }else if(linkfunc==3){//log
> Pj = exp(designMj*par);
> }
> if(boundary==1){
> Pj.elem(find(Pj<eps)).fill(eps);
> Pj.elem(find(Pj>1-eps)).fill(1-eps);
> }
> return Pj;
> }
>
> // [[Rcpp::export]]
> arma::mat Calc_Pj_jac(arma::vec par,
> arma::mat designMj,
> int & linkfunc,
> int boundary = 0,
> double eps = 1e-16){
> arma::mat ret = designMj;
> if(linkfunc>1){
> arma::vec Pj = Calc_Pj(par = par, designMj = designMj, linkfunc =
> linkfunc, boundary = boundary, eps = eps);
> if(linkfunc==2){//logit
> ret.each_col() %= Pj%(1-Pj);
> }else if(linkfunc==3){//log
> ret.each_col() %= Pj;
> }
> }
> return ret;
>
> }
>
> On 3/24/2018 6:18 PM, Duncan Murdoch wrote:
>>
>> On 24/03/2018 9:28 AM, Wenchao Ma wrote:
>>>
>>> Dear all,
>>>
>>> I just submitted a package to CRAN. It went well on my computer and
>>> winbuilder, but produced warning and error messages on some linux and
>>> solaris systems, as shown here:
>>> https://cran.r-project.org/web/checks/check_results_GDINA.html
>>>
>>> Does anyone know what is going on here?
>>
>>
>> The compilers on those systems are set up to be more sensitive to coding
>> problems than yours.
>>
>> The Solaris error comes from your code
>>
>> arma::umat B = arma::ones(K,pow(2,K)-K-1);
>>
>> The library you're using has no function definition for pow() that takes 2
>> integers. If a float or double result is okay, you could use pow(2.0, K).
>> If you really need an int in that argument, you'll need to typecast or write
>> your own pow() function.
>>
>> The Fedora warning seems self explanatory:
>>
>> Mstep.cpp:62:69: warning: explicitly assigning value of variable of type
>> 'int' to itself [-Wself-assign]
>>
>> Take a look at that line (and the other similar ones) and see why the
>> compiler thinks you've got something like
>>
>> i = i;
>>
>> in your code.
>>
>> Duncan Murdoch
>
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] install.packages doesn't produce warnings unless qualified with utils::
On Sat, Mar 3, 2018 at 1:33 PM, David Hugh-Jones
<davidhughjo...@gmail.com> wrote:
> Hi all,
>
> Assuming this is an R core issue:
>
> tryCatch(install.packages("clipr", repos = "bullshit"), warning = function
> (w) cat("got a warning"))
> Warning in install.packages :
> unable to access index for repository bullshit/src/contrib:
> cannot open URL 'bullshit/src/contrib/PACKAGES'
> Warning in install.packages :
> package ‘clipr’ is not available (for R version 3.4.3)
> Warning in install.packages :
> unable to access index for repository
> bullshit/bin/macosx/el-capitan/contrib/3.4:
> cannot open URL 'bullshit/bin/macosx/el-capitan/contrib/3.4/PACKAGES'
>
> In other words, lots of warnings, but none are caught.
>
Using R-3.4.3 on 64-bit Ubuntu throws and catches the warning.
R> tryCatch(install.packages("clipr", repos="w"), warning=function(w)
cat("got a warning!\n"))
Installing package into '/home/josh/R/library'
(as 'lib' is unspecified)
got a warning!
> It works if you use the fully qualified version in utils:
>
> tryCatch(utils::install.packages("clipr", repos = "bullshit"), warning =
> function (w) cat("got a warning"))
> got a warning
>
My guess is that something (a package, console, etc) is masking
utils::install.packages().
> Any ideas?
> Cheers,
> David
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Help with CRAN error
On Sun, Feb 4, 2018 at 12:47 PM, Dev Chakraborty <dpc10s...@gmail.com> wrote: > Dear All, > > I get the following message on running R CMD check -- as - cran: > > Package has a VignetteBuilder field but no prebuilt vignette index. > > I do not know how to fix this. Any help will be appreciated. > I don't know how to fix it either. That said, often a great first step is to search for the error. A Google search for, "Package has a VignetteBuilder field but no prebuilt vignette index.", which points to: https://stackoverflow.com/q/30976308. And a Google search "VignetteBuilder" has several promising "how to" results, and there is a "Non-Sweave Vignettes" section of Writing R Extensions: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Non_002dSweave-vignettes. It will be easier for others to help you if you can do some investigation on your own and come back with more specifics. > Sincerely, > > Dev > > > -- > Dev Chakraborty, PhD > Professor of Radiology retired > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Help with correcting error in package
On Sat, Feb 3, 2018 at 11:44 AM, Dev Chakraborty <dpc10s...@gmail.com> wrote:
> Dear All,
>
> I tried unsuccessfully to upload a package named RJafroc to CRAN. On my OSX
> machine it passes R CMD Check with no errors, warnings or notes. However, I
> get the following output from CRAN, from which I cannot tell which function
> is causing the error (I am truncating the intermediate lines, which appear
> to be the opening screen upon starting R, with ellipses; the full output
> produced by the CRAN pre-test is attached):
>
Submit your package to http://win-builder.r-project.org. That will
provide you access to all the output in the Rcheck.RJafroc directory.
Also note that the "Confirmation" step of the submission process
suggests, "You could [run R CMD check] using the win-builder service
at http://win-builder.r-project.org;. And the CRAN Repository Policy
says, "It should be normal for those without Windows machines of their
own to use the winbuilder service to check a package before
submission." You could also use https://builder.r-hub.io/. Both are
great resources for the community.
>
>
> *** running examples for arch 'x64' ... ERROR*
>
>
> *Running examples in 'RJafroc-Ex.R' failed*
>
>
> *The error occurred in:*
>
>
> *R Under development (unstable) (2018-02-01 r74194) -- "Unsuffered
> Consequences"*
>
> *Copyright (C) 2018 The R Foundation for Statistical Computing*
>
> *Platform: x86_64-w64-mingw32/x64 (64-bit)*
>
>
> *R is free software and comes with ABSOLUTELY NO WARRANTY.*
>
> *You are welcome to redistribute it under certain conditions*
>
>
> *...*
>
> *...*
>
> *Type 'q()' to quit R.*
>
>
> *> pkgname <- "RJafroc"*
>
> *> source(file.path(R.home("share"), "R", "examples-header.R"))*
>
>
> Thanking you in advance for any help rendered,
>
> Sincerely,
>
> Dev
>
> --
> Dev Chakraborty, PhD
> Professor of Radiology, retired
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Conditionally register method with generic in other package
For posterity, the solution I employed did not register the method as a formal S3 method. Functionality in package B was unchanged as long as package B was loaded and attached... which was good enough for my purposes. On Wed, Dec 6, 2017 at 6:45 AM, Joshua Ulrich <josh.m.ulr...@gmail.com> wrote: > I have a feeling I know the answer to this question, but I'm asking > with the hope that I learn something new. > > Package A defines a S3 generic function, "foo()". Package B defines a > S3 class "bar", and a foo.bar() method. This method is not strictly > necessary for the package to function, and package A has many > dependencies. > > To avoid excessive dependencies, I would like to only register > foo.bar() if package A is installed at the time package B is > installed. If package A is installed after package B, then warn the > user when package B is loaded and/or attached, so they can re-install > A and have foo.bar() registered correctly. > > I could figure out a way to do this manually, but doubt it would be > CRAN-approved. What is the recommended way to handle this situation? > I've received one suggestion to split B into 2 packages: one without > the method, and one with. Are there other alternatives? > > Thanks, > Josh > > -- > Joshua Ulrich | about.me/joshuaulrich > FOSS Trading | www.fosstrading.com > R/Finance 2018 | www.rinfinance.com -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2018 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Conditionally register method with generic in other package
Bill, Duncan,
Thanks for the replies.
On Wed, Dec 6, 2017 at 12:51 PM, Duncan Murdoch
<murdoch.dun...@gmail.com> wrote:
> On 06/12/2017 8:44 AM, Bill Denney wrote:
>>
>>
>>> On Dec 6, 2017, at 07:45, Joshua Ulrich <josh.m.ulr...@gmail.com> wrote:
>>>
>>> To avoid excessive dependencies, I would like to only register
>>> foo.bar() if package A is installed at the time package B is
>>> installed. If package A is installed after package B, then warn the
>>> user when package B is loaded and/or attached, so they can re-install
>>> A and have foo.bar() registered correctly.
>>
>>
>> One simple solution would be to wrap the instantiation of foo.bar in a
>> require test:
>>
>> if (require(A)) {
>>foo.bar <- function(...) {
>> print("To be or not to be")
>>}
>> } else {
>>message("To use the foo.bar function, please install package A.")
>> }
>>
>
> It's usually better to use requireNamespace("A") instead of require("A")
> (and the quotes are needed in requireNamespace(), but optional in
> require()). That tries to load the package, but doesn't mess with the search
> list.
>
> But that'll likely cause warnings, unless the same condition is used in the
> NAMESPACE file where there should be an entry
>
> S3method(foo, bar)
>
> The thing is, I'm not sure if
>
> if (requireNamespace("A"))
> S3method(foo, bar)
>
> is legal in a NAMESPACE file.
>
While perhaps not technically illegal, that doesn't work because you need
if (requireNamespace("A", quietly = TRUE)) {
importFrom(A, foo)
S3method(foo, bar)
}
to prevent an error about being able to find object 'foo' when loading
B's namespace. But the importFrom() causes R CMD check to expect a
hard dependency on package A (and all its dependencies), which is what
I'm trying to avoid...
> Duncan Murdoch
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Installing Tests with R Build
On Thu, Nov 23, 2017 at 10:05 AM, Bill Denney <b...@denney.ws> wrote:
> Hi Joshua and Dirk,
>
>> On Nov 23, 2017, at 10:17, Dirk Eddelbuettel <e...@debian.org> wrote:
>>
>>
>> On 23 November 2017 at 08:07, Joshua Ulrich wrote:
>> | On Wed, Nov 22, 2017 at 4:21 PM, Bill Denney <b...@denney.ws> wrote:
>> | > When running R BUILD now (via Travis CI), I get an error that the build
>> failed trying to build the vignettes. For R CMD install, I can add
>> --install-tests. Is there a way to control R BUILD so that the installation
>> includes test installation?
>> | >
>> | Put the tests you want to install into a directory inside the inst/
>> | directory (e.g. inst/tests/).
>>
>> Which happens to be the default mode of operations with RUnit.
>>
>> And it is a feature that I happen to _like_ a great deal because it permits
>> testing of the _installed_ package which is something you need in real life
>> when eg some of you system libraries may have change.
>
> Putting the tests there seems like a very reasonable solution to the
> immediate issue. It contravenes the options to install (or not install)
> tests ("R CMD INSTALL --install-tests" loses meaning). I'm going to update
> the structure of my vignette so that it detects if the tests are there and
> builds the vignette based on the existence of the tests (making it say
> "Please install the tests following these instructions...").
>
> I'm immediately ok with reworking the directory structure and tests, but I
> think it would be preferable long run to:
> Have R CMD BUILD gain an --install-args option so that installation arguments
> can be controlled (for situations like this and I'm sure others).
>
I just took a closer look at this, and I don't understand why any
change is necessary. With a package created by package.skeleton():
josh@thinkpad: ~/git
> R CMD build anRpackage
* checking for file 'anRpackage/DESCRIPTION' ... OK
* preparing 'anRpackage':
* checking DESCRIPTION meta-information ... OK
* installing the package to process help pages
* saving partial Rd database
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building 'anRpackage_1.0.tar.gz'
josh@thinkpad: ~/git
> tar -ztf anRpackage_1.0.tar.gz
anRpackage/DESCRIPTION
anRpackage/NAMESPACE
anRpackage/R/
anRpackage/R/file.R
anRpackage/build/
anRpackage/build/partial.rdb
anRpackage/build/vignette.rds
anRpackage/inst/
anRpackage/inst/doc/
anRpackage/inst/doc/file.Rnw
anRpackage/inst/doc/file.pdf
anRpackage/man/
anRpackage/man/anRpackage-package.Rd
anRpackage/tests/
anRpackage/tests/unittests.R
anRpackage/vignettes/
anRpackage/vignettes/file.Rnw
The package builds successfully, and the tests are still in their
tests/ directory. Now if I install without the --install-tests flag:
josh@thinkpad: ~/git
> R CMD INSTALL anRpackage_1.0.tar.gz 2>/dev/null
josh@thinkpad: ~/git
> ls -l ~/R/library/anRpackage/
total 32
-rw-r--r-- 1 josh josh 420 Nov 25 08:16 DESCRIPTION
drwxr-xr-x 2 josh josh 4096 Nov 25 08:16 doc
drwxr-xr-x 2 josh josh 4096 Nov 25 08:16 help
drwxr-xr-x 2 josh josh 4096 Nov 25 08:16 html
-rw-r--r-- 1 josh josh 59 Nov 25 08:16 INDEX
drwxr-xr-x 2 josh josh 4096 Nov 25 08:16 Meta
-rw-r--r-- 1 josh josh 72 Nov 25 08:16 NAMESPACE
drwxr-xr-x 2 josh josh 4096 Nov 25 08:16 R
You can see there's no tests/ directory. Now with the --install-tests flag:
josh@thinkpad: ~/git
> R CMD INSTALL anRpackage_1.0.tar.gz --install-tests 2>/dev/null
josh@thinkpad: ~/git
> ls -l ~/R/library/anRpackage/
total 36
-rw-r--r-- 1 josh josh 420 Nov 25 08:17 DESCRIPTION
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 doc
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 help
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 html
-rw-r--r-- 1 josh josh 59 Nov 25 08:17 INDEX
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 Meta
-rw-r--r-- 1 josh josh 72 Nov 25 08:17 NAMESPACE
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 R
drwxr-xr-x 2 josh josh 4096 Nov 25 08:17 tests
The tests/ directory is present, and I can run the tests:
josh@thinkpad: ~/git
> Rscript -e 'tools::testInstalledPackage("anRpackage")'
Testing examples for package 'anRpackage'
Running specific tests for package 'anRpackage'
Running 'unittests.R'
Running vignettes for package 'anRpackage'
Running 'file.Rnw'
Someone could probably give you specific advice if you share a link to
the code and/or TravisCI log(s).
> Thank you (and Happy Thanksgiving to those celebrating it),
>
> Bill
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Installing Tests with R Build
On Wed, Nov 22, 2017 at 4:21 PM, Bill Denney <b...@denney.ws> wrote: > Hi, > > I have a package that I'm trying to make a validation vignette for. The > validation vignette is intended to assist users with documentation that the > tests work. > > When running R BUILD now (via Travis CI), I get an error that the build > failed trying to build the vignettes. For R CMD install, I can add > --install-tests. Is there a way to control R BUILD so that the installation > includes test installation? > Put the tests you want to install into a directory inside the inst/ directory (e.g. inst/tests/). That directory will be copied to the top-level of the installed package directory. > Thanks, > > Bill > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] spurious rchk warning
On Mon, Nov 20, 2017 at 8:34 PM, Patrick Perry <ppe...@stern.nyu.edu> wrote: > One of my packages has what I believe to be spurious rchk warnings: > > From > https://raw.githubusercontent.com/kalibera/cran-checks/master/rchk/results/utf8.out > > Package utf8 version 1.1.0 > Package built using 73757/R 3.5.0; x86_64-pc-linux-gnu; 2017-11-20 22:02:23 > UTC; unix > Checked with rchk version 63f79d910f5835174fcaa5a0a7d2409348f7d2ac > More information at > https://github.com/kalibera/cran-checks/blob/master/rchk/PROTECT.md > > Function rutf8_as_utf8 > [UP] unprotected variable ans while calling allocating function > R_CheckUserInterrupt utf8/src/as_utf8.c:68 > [UP] unprotected variable ans while calling allocating function > rutf8_translate_utf8 utf8/src/as_utf8.c:83 > [UP] unprotected variable ans while calling allocating function > Rf_mkCharLenCE utf8/src/as_utf8.c:117 > > The offending function is at > https://github.com/patperry/r-utf8/blob/master/src/as_utf8.c#L46 > > I tried to fix the warning in line 64 with > > PROTECT(ans = sx); nprot++; > > This did not work. Any suggestions? > My hypothesis is that your PROTECT and UNPROTECT calls inside the for loop are leaving 'ans' unprotected at some point. Consider using PROTECT_WITH_INDEX() and REPROTECT() instead. > Thanks, > > > Patrick > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[Rd] SIGSEGV during startup
Hi,
This problem started as of r73472 ("Merged in the rest of the basic
ALTREP framework."); I tested r73471 and it did not exist.
I'm building R devel on Ubuntu 16.04.3 LTS (64-bit), with the following flags:
CC="gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer"
CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic -mtune=native"
and using
configure --with-valgrind-instrumentation=2
Running 'make' builds the R binary, but R fails to start. So 'make'
ultimately fails when trying to install sysdata.rda during the tools
package build. Here is the traceback:
make[5]: Leaving directory '/home/josh/R/R-build/src/library/tools/src'
make[4]: Leaving directory '/home/josh/R/R-build/src/library/tools'
make[4]: Entering directory '/home/josh/R/R-build/src/library/tools'
installing 'sysdata.rda'
ASAN:SIGSEGV
=
==3543==ERROR: AddressSanitizer: SEGV on unknown address
0x (pc 0x0043b2f2 bp 0x7ffeb5fdc670 sp 0x7ffeb5fdc660
T0)
#0 0x43b2f1 in ALTVEC_DATAPTR ../../../R-svn/src/main/altrep.c:305
#1 0x6b1068 in DATAPTR ../../../R-svn/src/include/Rinlinedfuns.h:105
#2 0x6b1068 in GetNewPage ../../../R-svn/src/main/memory.c:908
#3 0x6b5284 in Rf_allocVector3 ../../../R-svn/src/main/memory.c:2514
#4 0x654a49 in Rf_allocVector ../../../R-svn/src/include/Rinlinedfuns.h:514
#5 0x654a49 in Rf_mkTrue ../../../R-svn/src/main/gram.c:4164
#6 0x6b616c in Rf_InitMemory ../../../R-svn/src/main/memory.c:2137
#7 0x69186a in setup_Rmainloop ../../../R-svn/src/main/main.c:842
#8 0x694488 in Rf_mainloop ../../../R-svn/src/main/main.c:1088
#9 0x41e398 in main ../../../R-svn/src/main/Rmain.c:29
#10 0x7f9d71f4182f in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#11 0x41fbe8 in _start (/home/josh/R/R-build/bin/exec/R+0x41fbe8)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ../../../R-svn/src/main/altrep.c:305
ALTVEC_DATAPTR
==3543==ABORTING
../../../../R-svn/share/make/basepkg.mk:150: recipe for target 'sysdata' failed
make[4]: *** [sysdata] Error 1
make[4]: Leaving directory '/home/josh/R/R-build/src/library/tools'
Makefile:30: recipe for target 'all' failed
make[3]: *** [all] Error 2
make[3]: Leaving directory '/home/josh/R/R-build/src/library/tools'
Makefile:36: recipe for target 'R' failed
make[2]: *** [R] Error 1
make[2]: Leaving directory '/home/josh/R/R-build/src/library'
Makefile:28: recipe for target 'R' failed
make[1]: *** [R] Error 1
make[1]: Leaving directory '/home/josh/R/R-build/src'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1
The context around memory.c:908 is below, and suggests the issue
manifests with the combination of the ALTREP framework and valgrind
instrumentation level >1.
#if VALGRIND_LEVEL > 1
if (NodeClassSize[node_class] > 0)
VALGRIND_MAKE_MEM_NOACCESS(DATAPTR(s),
NodeClassSize[node_class]*sizeof(VECREC));
#endif
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Storing Data Used in an Example
On Wed, Sep 20, 2017 at 11:32 AM, Roy Mendelssohn - NOAA Federal <roy.mendelss...@noaa.gov> wrote: > Hi All: > > I have a package whose main function is to download environmental data using > a service. I have added some plot routines, As it is, my examples of just > the downloads push the CRAN limits on time, so I want the plot examples to > use data that has already been downloaded and stored as part of the package. > > Questions: > > 1. Most appropriate place to store the example data (they will be .RData > files). > https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Data-in-packages > 2. What size of data set is considered to be getting too large to be > appropriate to include in a package. > The CRAN Repository Policy says: "As a general rule, neither data nor documentation should exceed 5MB (which covers several books). A CRAN package is not an appropriate way to distribute course notes, and authors will be asked to trim their documentation to a maximum of 5MB." > Thanks, > > -Roy > > > > ** > "The contents of this message do not reflect any position of the U.S. > Government or NOAA." > ** > Roy Mendelssohn > Supervisory Operations Research Analyst > NOAA/NMFS > Environmental Research Division > Southwest Fisheries Science Center > ***Note new street address*** > 110 McAllister Way > Santa Cruz, CA 95060 > Phone: (831)-420-3666 > Fax: (831) 420-3980 > e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/ > > "Old age and treachery will overcome youth and skill." > "From those who have been given much, much will be expected" > "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] loadMethod() not available with Rscript
Hi Thierry, On Wed, Sep 20, 2017 at 7:45 AM, Thierry Onkelinx <thierry.onkel...@inbo.be> wrote: > Dear all, > > Could someone explain this to me? I have a simple script (see below). > my_function() is an S4 method. The package imports all functions from > methods that it uses. > > When I run the script in a vanilla R session it works. When using > Rscript I get the error: Error in loadMethod(function (x, ...) : > could not find function "loadMethod" > > loadMethod() is NOT used in mypackage and not imported. > Rscript does not load/attach the "methods" package like is done in an interactive R session. Perhaps your package uses a package that needs methods::loadMethod(), but does not import it correctly? > library(mypackage) > my_function() > > Best regards, > > ir. Thierry Onkelinx > Statisticus/ Statiscian > > Vlaamse Overheid / Government of Flanders > INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE > AND FOREST > Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance > thierry.onkel...@inbo.be > Kliniekstraat 25, B-1070 Brussel > www.inbo.be > > /// > To call in the statistician after the experiment is done may be no > more than asking him to perform a post-mortem examination: he may be > able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher > The plural of anecdote is not data. ~ Roger Brinner > The combination of some data and an aching desire for an answer does > not ensure that a reasonable answer can be extracted from a given body > of data. ~ John Tukey > /// > > > Van 14 tot en met 19 december 2017 verhuizen we uit onze vestiging in > Brussel naar het Herman Teirlinckgebouw op de site Thurn & Taxis. > Vanaf dan ben je welkom op het nieuwe adres: Havenlaan 88 bus 73, 1000 > Brussel. > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Errors in Package Check Results due to package in Suggestions
Hi Daniel, See section 1.1.3.1 of Writing R Extensions: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages On Tue, Sep 19, 2017 at 2:53 AM, Daniel Lüdecke <d.luede...@uke.de> wrote: > Hi, > > I have a package on CRAN where most check results are OK except two with > errors (one of those two I'm already addressing). > > However, the one of the errors > (https://www.r-project.org/nosvn/R.check/r-release-osx-x86_64/sjstats-00chec > k.html) is because I use the package "rstanarm" for one of the examples, and > running this example fails. What would be an appropriate way to deal with > this issue? My package does not depend on rstanarm, it's "just" a suggestion > because I needed it for the examples. I'm a bit unsure about the status of > these check results. When submitting a package, the maintainer is asked to > address issues from these checks, so I'd like to fix this error. I could, of > course, wrap the example into a "don't run"-section, but this seems to me > like a sloppy hack rather than a proper solution... > > Best > Daniel > > > -- > > _ > > Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen Rechts; > Gerichtsstand: Hamburg | www.uke.de > Vorstandsmitglieder: Prof. Dr. Burkhard Göke (Vorsitzender), Prof. Dr. Dr. > Uwe Koch-Gromus, Joachim Prölß, Martina Saurin (komm.) > _ > > SAVE PAPER - THINK BEFORE PRINTING > > ______ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Why is as.function() slower than eval(call("function"())?
On Thu, Aug 3, 2017 at 11:32 PM, Gregory Werbin
<outth...@me.gregwerbin.com> wrote:
> (Apologies if this is better suited for R-help.)
>
> On my system (macOS Sierra, late 2014 MacBook Pro; R 3.4.1, Homebrew build),
> I found that it is faster to construct a function using eval(call("function",
> ...)) than using as.function(list(...)). Example:
>
> make_fn_1 <- function(a, b) eval(call("function", a, b), env =
> parent.frame())
> make_fn_2 <- function(a, b) as.function(c(a, list(b)), env =
> parent.frame())
>
> a <- as.pairlist(alist(x = , y = ))
> b <- quote(x + y)
>
> library("microbenchmark")
> microbenchmark(make_fn_1(a, b), make_fn_2(a, b))
>
> # Unit: microseconds
> # expr min lqmean median uqmax neval cld
> # make_fn_1(a, b) 1.671 1.8855 2.13297 2.039 2.1950 9.852 100 a
> # make_fn_2(a, b) 3.541 3.7230 4.13400 3.906 4.1055 23.153 100 b
>
> At first I thought the gap was due to the overhead of calling c(a, list(b)).
> But this turns out not to be the case:
>
> make_fn_weird <- function(a, b) as.function(c(a, b), env = parent.frame())
> b_wrapped <- list(b)
>
> make_fn_weirder <- function(a_b) as.function(a_b, env = parent.frame())
> a_b <- c(a, b_wrapped)
>
> microbenchmark(make_fn_1(a, b), make_fn_2(a, b),
>make_fn_weird(a, b_wrapped), make_fn_weirder(a_b))
>
> # Unit: microseconds
> # expr min lqmean median uqmax
> neval cld
> # make_fn_1(a, b) 1.718 1.8990 2.12119 1.9860 2.1605 8.057
> 100 a
> # make_fn_2(a, b) 3.393 3.5865 4.03029 3.6655 3.9615 27.499
> 100 c
> # make_fn_weird(a, b_wrapped) 3.354 3.5005 3.77190 3.6405 3.9425 6.839
> 100 c
> # make_fn_weirder(a_b) 2.488 2.6290 2.83352 2.7215 2.8800 7.007
> 100 b
>
> One IRC user pointed out that as.function() takes its own path through the
> code, namely do_asfunction() (in src/main/coerce.c). What is it about this
> code path that's 50% slower than whatever happens during
> eval(call("function", a, b))?
>
> Obviously this is a trivial micro-optimization and it doesn't matter to 99%
> of users. Mostly asking out of curiosity, but also wondering if there's a
> more general lesson to be learned here.
>
Agreed that this is minor (~2us), but the majority of the difference
seems to be from S3 method dispatch. as.function() is generic and has
to dispatch to as.function.default(). The times are very similar if
you call the method directly.
R> make_fn_3 <- function(a, b) as.function.default(c(a, list(b)), env
= parent.frame())
R> microbenchmark(make_fn_1(a, b), make_fn_2(a, b), make_fn_3(a, b))
Unit: microseconds
expr min lq mean medianuq max neval
make_fn_1(a, b) 1.615 1.7595 12.78339 1.9115 2.145 1077.657 100
make_fn_2(a, b) 3.077 3.3390 19.89423 3.5215 3.862 1589.505 100
make_fn_3(a, b) 1.629 1.7975 15.40389 1.9505 2.227 1335.306 100
Now the difference is <100ns, which is much harder to investigate.
> Thanks!
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[R-pkg-devel] Need to call strptime() with specific LC_TIME
Hi,
I'm looking for any guidance/best-practices about how to deal with an
issue a quantmod user ran into an issue when calling
getSymbols.google():
https://github.com/joshuaulrich/quantmod/issues/140
Google provides a CSV where the dates are specified using English
month abbreviations. Converting the characters to Date does not work
in any locale that doesn't use those abbreviations.
So I need to call strptime() in a locale that uses English month
abbreviations. Is there a better solution than this?
lc_time <- Sys.getlocale("LC_TIME")
Sys.setlocale(category = "LC_TIME", locale = "C")
on.exit(Sys.setlocale(category = "LC_TIME", locale = lc_time)
Thanks!
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2017 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] bug in rbind?
I'm not sure whether or not this is a bug, but I did isolate the line where the error is thrown: src/library/base/R/dataframe.R:1395. https://github.com/wch/r-source/blob/01374c3c367fa12f555fd354f735a6e16e5bd98e/src/library/base/R/dataframe.R#L1395 The error is thrown because the line attempts to set a subset of the rownames to NULL, which fails. R> options(error = recover) R> rbind(dfm.names, dfm) Error in rownames(value[[jj]])[ri] <- rownames(xij) : replacement has length zero Enter a frame number, or 0 to exit 1: rbind(dfm.names, dfm) 2: rbind(deparse.level, ...) Selection: 2 Called from: top level Browse[1]> rownames(value[[jj]]) [1] "a" "b" "c" NA NA NA Browse[1]> rownames(xij) NULL Browse[1]> ri [1] 4 5 6 Browse[1]> rownames(value[[jj]])[ri] [1] NA NA NA On Mon, Jan 16, 2017 at 7:50 PM, Krzysztof Banas <krzysz...@nus.edu.sg> wrote: > I suspect there may be a bug in base::rbind.data.frame > > Below there is minimal example of the problem: > > m <- matrix (1:12, 3) > dfm <- data.frame (c = 1 : 3, m = I (m)) > str (dfm) > > m.names <- m > rownames (m.names) <- letters [1:3] > dfm.names <- data.frame (c = 1 : 3, m = I (m.names)) > str (dfm.names) > > rbind (m, m.names) > rbind (m.names, m) > rbind (dfm, dfm.names) > > #not working > rbind (dfm.names, dfm) > > Error in rbind(deparse.level, ...) : replacement has length zero > > rbind (dfm, dfm.names)$m > > > [,1] [,2] [,3] [,4] > > 147 10 > > 258 11 > > 369 12 > > a 147 10 > > b 258 11 > > c 369 12 > > > > > > Important: This email is confidential and may be privileged. If you are not > the intended recipient, please delete it and notify us immediately; you > should not copy or use it for any purpose, nor disclose its contents to any > other person. Thank you. > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] wish list: generalized apply
On Thu, Dec 8, 2016 at 3:59 PM, David Winsemius <dwinsem...@comcast.net> wrote:
>
>> On Dec 8, 2016, at 12:09 PM, John P. Nolan <jpno...@american.edu> wrote:
>>
>> Dear All,
>>
>> I regularly want to "apply" some function to an array in a way that the
>> arguments to the user function depend on the index on which the apply is
>> working. A simple example is:
>>
>> A <- array( runif(160), dim=c(5,4,8) )
>> x <- matrix( runif(32), nrow=4, ncol=8 )
>> b <- runif(8)
>> f1 <- function( A, x, b ) { sum( A %*% x ) + b }
>> result <- rep(0.0,8)
>> for (i in 1:8) {
>> result[i] <- f1( A[,,i], x[,i] , b[i] )
>> }
>>
>> This works, but is slow. I'd like to be able to do something like:
>>generalized.apply( A, MARGIN=3, FUN=f1, list(x=x,MARGIN=2),
>> list(b=b,MARGIN=1) ), where the lists tell generalized.apply to pass x[,i]
>> and b[i] to FUN in addition to A[,,i].
>>
>> Does such a generalized.apply already exist somewhere? While I can write a
>> C function to do a particular case, it would be nice if there was a fast,
>> general way to do this.
>
> I would have thought that this would achieve the same result:
>
> result <- sapply( seq_along(b) , function(i) { f1( A[,,i], x[,i] , b[i] )} )
>
> Or:
>
> result <- sapply( seq.int( dim(A)[3] ) , function(i) { f1( A[,,i], x[,i] ,
> b[i] )} )
>
> (I doubt it will be any faster, but if 'i' is large, parallelism might help.
> The inner function appears to be fairly efficient.)
You're right, it's slower. Despite how often it's repeated that
"loops in R are slow", they're not *that* slow. They're often faster
than the *apply functions, especially if they have been "compiled" by
compiler::cmpfun().
You really need to know *why* code is slow before trying to make it
faster. I profiled an example that would have a loop with 1e6
iterations and 80%+ of the time was still spent inside f1().
set.seed(21)
nc <- 1e6
nr <- 10
A <- array( runif(5*nr*nc), dim=c(5,nr,nc) )
x <- matrix( runif(nr*nc), nrow=nr, ncol=nc )
b <- runif(nc)
f1 <- compiler::cmpfun(function( A, x, b ) { sum( A %*% x ) + b })
f2 <- compiler::cmpfun({
function(A, x, b, FUN) {
result <- numeric(length(b))
for (i in seq_along(b)) {
result[i] <- FUN( A[,,i], x[,i] , b[i] )
}
return(result)
}
})
Rprof(interval=0.01)
result <- f2(A,x,b,f1)
Rprof(NULL)
summaryRprof()
$by.self
self.time self.pct total.time total.pct
"FUN" 4.2984.28 4.76 93.52
"%*%" 0.47 9.23 0.47 9.23
"f2" 0.33 6.48 5.09100.00
$by.total
total.time total.pct self.time self.pct
"f2"5.09100.00 0.33 6.48
"FUN" 4.76 93.52 4.2984.28
"%*%" 0.47 9.23 0.47 9.23
$sample.interval
[1] 0.01
$sampling.time
[1] 5.09
In this case, almost all the time is spent evaluating f1() ("FUN"),
even after calling compiler::cmpfun on f1() and on a function
containing the loop. Making the looping construct faster is not going
to improve the performance of this code by a significant amount.
I.e., dropping to compiled code will only help if you avoid the R
function call, but then that's not a general solution...
> --
>
> David Winsemius
> Alameda, CA, USA
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] segfault with POSIXlt zone=NULL zone=""
On Tue, Dec 6, 2016 at 10:30 AM, <frede...@ofb.net> wrote: > Hi Joshua, > > Thank you for minimizing my test case. > >> > Hope I'm not doing something illegal... >> > >> You are. You're changing the internal structure of a POSIXlt object >> by re-ordering the list elements. You should not expect a malformed >> POSIXlt object to behave as if it's correctly formed. You can see >> it's malformed by comparing it's unclass()'d output. >> >> d <- as.POSIXlt(Sys.time()) >> unclass(d) # valid POSIXlt object >> d$zone <- NULL >> d$zone <- "" >> unclass(d) # your malformed POSIXlt object > > I don't know if these questions are not already obvious, but: > > 1. Is there a reasonable way to fail more elegantly when a user makes > this mistake? > It's not just "this mistake". See below. > 2. Should we update the documentation for POSIXlt to warn people that > the optional "zone" list element must precede the optional "gmtoff" > list element, in cases where both are present? > No, because that's not the only way to create a malformed POSIXlt object. Reordering *any* of the elements results in a segfault, and there are probably other things you could do to the internal structure of POSIXlt objects to cause segfaults. Maybe update the documentation to say, "If you update the internal structure of a POSIXlt object, you deserve whatever happens."? ;-) > Thanks, > > Frederick -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] segfault with POSIXlt zone=NULL zone=""
On Tue, Dec 6, 2016 at 6:37 AM, <frede...@ofb.net> wrote:
> Hi all,
>
> I ran into a segfault while playing with dates.
>
> $ R --no-init-file
> ...
> > library(lubridate); d=as.POSIXlt(floor_date(Sys.time(),"year"));
> d$zone=NULL; d$zone=""; d
>
If you're asking about a bug in R, you should provide a *minimal*
reproducible example (i.e. one without any package dependencies).
This has nothing to do with lubridate, so you can reproduce the
behavior with:
d <- as.POSIXlt(Sys.time())
d$zone <- NULL
d$zone <- ""
d
> Attaching package: ‘lubridate’
>
> The following object is masked from ‘package:base’:
>
> date
>
> Warning message:
> package ‘lubridate’ was built under R version 3.4.0
>
> *** caught segfault ***
> address (nil), cause 'unknown'
>
> Traceback:
> 1: format.POSIXlt(x, usetz = TRUE)
> 2: format(x, usetz = TRUE)
> 3: print(format(x, usetz = TRUE), ...)
> 4: print.POSIXlt(x)
> 5: function (x, ...) UseMethod("print")(x)
>
> Possible actions:
> ...
>
> Hope I'm not doing something illegal...
>
You are. You're changing the internal structure of a POSIXlt object
by re-ordering the list elements. You should not expect a malformed
POSIXlt object to behave as if it's correctly formed. You can see
it's malformed by comparing it's unclass()'d output.
d <- as.POSIXlt(Sys.time())
unclass(d) # valid POSIXlt object
d$zone <- NULL
d$zone <- ""
unclass(d) # your malformed POSIXlt object
> Thanks,
>
> Frederick
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] robust download function in R (similar to wget)?
Try downloader: https://cran.r-project.org/package=downloader On Thu, Aug 25, 2016 at 7:47 AM, Lucas Ferreira Mation <lucasmat...@gmail.com> wrote: > I am creating this package <https://github.com/lucasmation/microdadosBrasil> > that has to download data from some sources that are a bit > unstable/unreliable. The problem is described in this issue > <https://github.com/lucasmation/microdadosBrasil/issues/56>. > > Is there a more "robust" download fucntion in R? > Something similar to wget, that would: > > - where there are errors, repeat the download a few times before giving up, > preferably, using partial data on a file being downloaded from previous > attempts > - check the integrity of the downloaded file > > As this has to be embedded in an R package, the solution needs to be OS > independent and require no installation of external software by the user. > > regards > Lucas > > [[alternative HTML version deleted]] > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?
On Tue, Aug 2, 2016 at 11:26 AM, Thomas J. Leeper <thosjlee...@gmail.com> wrote:
> I have a fairly open-ended question about the handling of package
> dependencies that is inspired by the precise distinction between
> "Depends", "Imports", and "Suggests" listed in DESCRIPTION.
>
> Some background, as I understand it: Prior to requiring package
> namespaces, we listed package dependencies in "Depends" because the
> required package had to be (1) installed and (2) on the search path in
> order to access its functions. After introducing and requiring
> namespaces, there are very few circumstances in which "Depends" is
> needed because functions from the required package can either be
> imported (via "Imports" and NAMESPACE) and/or referred to using the
> `pkg::fun()` style. "Suggests" packages are maybe needed for examples,
> vignettes, etc. but are not installed by default via
> `install.packages()`, so are not guaranteed to be available.
>
> Some observations:
>
> 1. "Depends" seems to be less useful than before, except in rare cases
> where a package needs to be (a) installed, (b) loaded, and (c) on the
> search path. Imports covers most package dependency use cases.
>
> 2. There is a gap in functionality between "Imports" and "Suggests".
> Sometimes there is a need for functions that should be available
> (i.e., installed) but do not need to be loaded or imported because
> they are rarely used (e.g., graphing functions). Importing the
> functions adds bloat but only putting their package in "Suggests" does
> not guarantee availability when, for example, calling
> `requireNamespace()` or `require()`.
>
Maybe I'm missing something, but isn't that the point of calling
requireNamespace()? For example:
if (requireNamespace("suggestedPackage"))
stop("suggestedPackage required but not installed")
That doesn't seem like a heavy burden for package writers when writing
infrequently used functions in their package. You could even add the
install.packages command to the error message to instruct users to
install the required package.
> Suggestion:
>
> Might it be useful to have a category between "Imports" and "Suggests"
> that covers packages that should be installed but not imported?
>
> This could be done by changing `install.packages()` to cover
> "Suggests" by default, changing the meaning of "Depends" and
> "Imports", or adding a new category to DESCRIPTION.
>
I personally would not want install.packages() to install packages I'm
unlikely to actually use.
It's also not clear to me how importing rarely used functions causes
bloat, but installing all packages for all rarely-used functions does
not cause bloat.
> I am interested in hearing your thoughts on this issue.
>
> Best,
> -Thomas
>
> Thomas J. Leeper
> http://www.thomasleeper.com
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Calling C implementations of rnorm and friends
orm) >>> >>> >>> Doesn't help me at all in understanding how it works. It should create a >>> function random2(sn, sa, sb, norm, REALSXP); I understand that is a >>> version >>> of the random2 function that returns a real S expression taking sn, sa and >>> sb as parameters. But how does find the actual functional form for the >>> normal distribution? >>> >>> I am asking because I would like to rewrite some of the other functions, >>> such as parameterizing rbeta by the mean and sample size rather than by >>> the >>> number of successes and failures and rgamma by the mean and total time >>> elapsed instead of the number of events. Once I understand how the C >>> source >>> code works, it would be hopefully not very difficult to reparameterize >>> them. >>> >>> Thanks, >>> >>> Luis Usier >>> >>> [[alternative HTML version deleted]] >>> >>> __ >>> R-devel@r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> >> > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2016 | www.rinfinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] RODBC on Mac & _R_CHECK_FORCE_SUGGESTS_
On Mon, Jun 20, 2016 at 9:19 AM, Spencer Graves
<spencer.gra...@prodsyse.com> wrote:
>"R CMD check sos" with R 3.3.0 under Mac OS X 10.11.5 ends as
> follows:
>
>
> >* checking package dependencies ... ERROR
> >Package suggested but not available: ‘RODBC’
> >
> >The suggested packages are required for a complete check.
> >Checking can be attempted without them by setting the environment
> >variable _R_CHECK_FORCE_SUGGESTS_ to a false value.
>
>
>Unfortunately, "install.packages('RODBC')" says it's only available
> in source form. When I attempt to install from sources, it fails as
> follows:
>
>
> >checking for sqlext.h... no
> >configure: error: "ODBC headers sql.h and sqlext.h not found"
> >ERROR: configuration failed for package ‘RODBC’
> >* removing
> ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/RODBC’
> >Warning in install.packages :
> > installation of package ‘RODBC’ had non-zero exit status
>
>
>The CRAN checks for RODBC also failed for the same reason
> (https://cloud.r-project.org/web/checks/check_results_RODBC.html).
>
>
>
>I'm not sure how to set "the environment variable
> _R_CHECK_FORCE_SUGGESTS_ to a false value" on my Mac, and I'd rather not do
> it permanently. I tried various versions of "R CMD check
> _R_CHECK_FORCE_SUGGESTS_=FALSE sos_1.3-9.tar.gz" and "R CMD check
> _R_CHECK_FORCE_SUGGESTS_=0 sos_1.3-9.tar.gz", none of which worked.
> Somewhere I found a suggestion to try, "R CMD check --as-cran
> sos_1.3-9.tar.gz". That also failed for me.
>
_R_CHECK_FORCE_SUGGESTS_=FALSE R CMD check sos_1.3-9.tar.gz
>
>Suggestions?
>Thanks,
>Spencer Graves
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] [R-SIG-Finance] [VC++ calling R] How to create a real-time interactive ticking time-series chart using dygraph via RInside?
Please choose *one* _relevant_ mailing list. Spamming 5 (!)
mailing lists fragments the conversation and makes things difficult
for everyone involved.
On Mon, Apr 11, 2016 at 10:03 AM, Mike Deanza <mikedea...@outlook.com> wrote:
> Hi all,
>
>
> I am trying to figure out how to do this in R and I need your help.
>
>
> My journey started from something like the following:
>
>
> http://stackoverflow.com/questions/11365857/real-time-auto-updating-incremental-plot-in-r/1#1
>
>
> n=1000
> df=data.frame(time=1:n,y=runif(n))
> window=100
> for(i in 1:(n©\window)) {
> flush.console()
> plot(df$time,df$y,type='l',xlim=c(i,i+window))
> Sys.sleep(.09)
> }
>
>
> Then I wanted to make it nicer looking so I went to dygraph.
>
>
> And then, I would like to be able to live send tick data from within Visual
> C++ so I started to investigate RInside.
>
>
> Following the example code here:
>
>
> http://dirk.eddelbuettel.com/papers/useR2009RcppRInside.pdf
>
>
> I can open an RInside object in VC++, and then send some data to it, and then
> execute some command in it, and then get data back.
>
>
> It is really great.
>
>
> However, is there a way to have the real-time updating ticking plots to be
> drawn on dygraph inside RInside?
>
>
> It turns out the dygraph package tends to draw onto a browser. That makes the
> real-time updating pretty slow.
>
>
> Is there a way to set the dygraph to plot to a GUI window in VC++? For
> example, a QT or MFC GUI window?
>
>
> My working environment is Win7 64bit ,with VS 2013 and VS2015, QT 5.3 32bit.
>
>
> Could anybody please shed some lights on me?
>
>
> Thanks a lot!
>
>
>
>
>
>
>
> [[alternative HTML version deleted]]
>
>
> ___
> r-sig-fina...@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] [R-SIG-Finance] [VC++ calling R] How to create a real-time interactive ticking time-series chart using dygraph via RInside?
Please choose *one* _relevant_ mailing list. Spamming 5 (!)
mailing lists fragments the conversation and makes things difficult
for everyone involved.
On Mon, Apr 11, 2016 at 10:03 AM, Mike Deanza <mikedea...@outlook.com> wrote:
> Hi all,
>
>
> I am trying to figure out how to do this in R and I need your help.
>
>
> My journey started from something like the following:
>
>
> http://stackoverflow.com/questions/11365857/real-time-auto-updating-incremental-plot-in-r/1#1
>
>
> n=1000
> df=data.frame(time=1:n,y=runif(n))
> window=100
> for(i in 1:(n©\window)) {
> flush.console()
> plot(df$time,df$y,type='l',xlim=c(i,i+window))
> Sys.sleep(.09)
> }
>
>
> Then I wanted to make it nicer looking so I went to dygraph.
>
>
> And then, I would like to be able to live send tick data from within Visual
> C++ so I started to investigate RInside.
>
>
> Following the example code here:
>
>
> http://dirk.eddelbuettel.com/papers/useR2009RcppRInside.pdf
>
>
> I can open an RInside object in VC++, and then send some data to it, and then
> execute some command in it, and then get data back.
>
>
> It is really great.
>
>
> However, is there a way to have the real-time updating ticking plots to be
> drawn on dygraph inside RInside?
>
>
> It turns out the dygraph package tends to draw onto a browser. That makes the
> real-time updating pretty slow.
>
>
> Is there a way to set the dygraph to plot to a GUI window in VC++? For
> example, a QT or MFC GUI window?
>
>
> My working environment is Win7 64bit ,with VS 2013 and VS2015, QT 5.3 32bit.
>
>
> Could anybody please shed some lights on me?
>
>
> Thanks a lot!
>
>
>
>
>
>
>
> [[alternative HTML version deleted]]
>
>
> ___
> r-sig-fina...@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Is it possible to reverse engineer a package from the installed library directory ?
On Wed, Mar 23, 2016 at 9:25 PM, Ben Bolker <bbol...@gmail.com> wrote:
>
> You could take your old install and do this:
>
> library("your_package")
> funs <- ls(pos="package:your_package")
> dump(funs,"your_package.R")
>
> that wouldn't recreate documentation, NAMESPACE files, etc., but it would
> be a start.
>
You might be able to recreate the Rd files using the answers to this
StackOverflow question: http://stackoverflow.com/q/7493843/271616
Using the example in my answer as a starting point:
sink(file="survey.Rd")
utils:::.getHelpFile(as.character(help(survey,package="MASS")))
sink()
It might still need some formatting clean-up, but hopefully that gets
you 80-90% of the solution.
>
> On 16-03-23 04:43 PM, Paul Hurley wrote:
>>
>> Several years ago I assembled some of my own most used code into a
>> package,
>> and had that installed on my machine. Since then I have lost the source
>> files, but still have the package installed in an old v2.x R installation.
>>
>> I now need to use the package in my current R install, which is v3.1, and
>> won't let me just copy the directory across, as it complains it was build
>> pre v3. Is it possible to reverse engineer a package back to R source
>> files so I can then rebuild it in a new package ?
>>
>> Thanks
>>
>> Paul.
>>
>> --http://www.paulhurley.co.uk
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-package-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] %OS on output
On Wed, Feb 24, 2016 at 10:40 AM, Suharto Anggono Suharto Anggono via
R-devel <r-devel@r-project.org> wrote:
> R help on 'strptime' has the following in "Details" section.
> Specific to R is ‘%OSn’, which for output gives the seconds truncated to ‘0
> <= n <= 6’ decimal places (and if ‘%OS’ is not followed by a digit, it uses
> the setting of ‘getOption("digits.secs")’, or if that is unset, ‘n = 3’).
>
> In reality, for output, if '%OS' is not followed by a digit and
> getOption("digits.secs") is unset, the output has no fractional part, as if n
> = 0 is used.
>
That's because n = 0 _is_ used, and appears to have always been the
default (since the logic was added in r37395). So this appears to be
a typo in the documentation (the comment about "n=3" when digits.secs
is unset was added in r37439).
>> getOption("digits.secs")
> NULL
>> z <- strptime("20/2/06 11:16:16.683", "%d/%m/%y %H:%M:%OS")
>> format(z, "%OS")
> [1] "16"
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] should `data` respect default.stringsAsFactors()?
On Thu, Feb 18, 2016 at 6:03 PM, Cook, Malcolm <m...@stowers.org> wrote:
> Hi Peter,
>
> Sorry if I was not clear. Perhaps an example will make my point:
>
>> data(iris)
>> class(iris$Species)
> [1] "factor"
>> write.table(iris,'data/myiris.tab')
>> data(myiris)
>> class(myiris$Species)
> [1] "factor"
>> rm(myiris)
>> options(stringsAsFactors = FALSE)
>> data(myiris)
>> class(myiris$Species)
> [1] "factor"
>> myiris<-read.table("data/myiris.tab",header=TRUE)
>> class(myiris$Species)
> [1] "character"
>
> I am surprised to find that in the above
> setting the global option stringsAsFactors = FALSE does NOT effect
> how Species is being read in by the `data` function
> whereas
> setting the global option stringsAsFactors = FALSE DOES effect how
> Species is being read in by read.table
>
> especially since data is documented as calling read.table.
>
To be explicit, it's documented as calling read.table(..., header =
TRUE) in this case, but it actually calls read.table(..., header =
TRUE, as.is = FALSE), which results in class(myiris$Species) of
"factor".
R> myiris<-read.table("data/myiris.tab",header=TRUE,as.is=FALSE)
R> class(myiris$Species)
[1] "factor"
So it seems like adding as.is = FALSE to the call in the documentation
would clear this up.
> In my opinion, one or the other should change (the behavior of data, or the
> documentation).
>
> ,
>
> ~ Malcolm
>
>
> > -Original Message-
> > From: peter dalgaard [mailto:pda...@gmail.com]
> > Sent: Thursday, February 18, 2016 3:32 PM
> > To: Cook, Malcolm <m...@stowers.org>
> > Cc: r-de...@stat.math.ethz.ch
> > Subject: Re: [Rd] should `data` respect default.stringsAsFactors()?
> >
> > What the are you on about? data() does many things, only some of
> > which call read.table() et al., and the ones that do have no special
> treatment
> > of stringsAsFactors.
> >
> > -pd
> >
> > > On 18 Feb 2016, at 21:25 , Cook, Malcolm <m...@stowers.org> wrote:
> > >
> > > Hiya,
> > >
> > > Probably been debated elsewhere
> > >
> > > I note that R's `data` function does not respect default.stringsAsFactors
> > >
> > > By my lights, it should, especially as it is documented to call
> read.table,
> > which DOES respect.
> > >
> > > Oh, but: http://r.789695.n4.nabble.com/stringsAsFactors-FALSE-
> > tp921891p921893.html
> > >
> > > Compelling. I have to agree.
> > >
> > > So, I change my mind.
> > >
> > > By my lights, `data` should then be documented to NOT respect
> > default.stringsAsFactors.
> > >
> > > Else?
> > >
> > > ~Malcolm Cook
> > >
> > > ______
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> > --
> > Peter Dalgaard, Professor,
> > Center for Statistics, Copenhagen Business School
> > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> > Phone: (+45)38153501
> > Office: A 4.23
> > Email: pd@cbs.dk Priv: pda...@gmail.com
> >
> >
> >
> >
> >
> >
> >
> >
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2016 | www.rinfinance.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Class with functions as slots
On Wed, Nov 18, 2015 at 7:35 PM, Glenn Schultz <glennmschu...@me.com> wrote:
> All,
>
> I have the following class with slots as functions. For the analysis of MBS
> this is good set-up as it allows for a very clean and modular way of
> handling things. My question is really geared to understanding what is
> happening in R.
>
> Below is the function without sato = NULL R CMD throws a warning global
> variable not defined. With SATO = NULL I do not get the warning. Why only
> sato and not two and ten which are also inputs?
>
You use sato in the yr15 function even though it is not an argument to
the function. Did you forget to add it as an argument to the yr15
function?
> Glenn
>
> # Bond Lab is a software application for the analysis of # fixed income
> securities it provides a suite of applications
> # in addition to standard fixed income analysis bond lab provides # for the
> specific analysis of structured products residential mortgage backed
> securities, # asset backed securities, and commerical mortgage backed
> securities
> # License GPL3 + File License
> # Copyright (C) 2014 Glenn M Schultz, CFA
>
> #' An S4 class whose slots are functions used to propogate
> #' the mortgage rate used to motivate the prepayment model
> #' @slot yr30 A function defining the 30-year mortgage rate as a function
> #' of the 2- and 10-year swap rate
> #' @slot yr15 A function defining the 15-year mortgage rate as a function
> #' of the 2- and 10-year swap rate #' @export MortgageRate
>
> setClass("MortgageRate",
> representation(
> yr30 = "function",
> yr15 = "function"))
>
> #' A constructor function for the class Mortgage Rate
> MortgageRate <- function(){
> sato = NULL
> new("MortgageRate",
> yr30 = function(two = numeric(), ten = numeric(), sato = numeric()) {
> 2.25 + (.06 * two) + (.75 * ten) + sato},
>
> yr15 = function(two = numeric(), ten = numeric()){
> 1.75 + (.06 * two) + (.75 * ten) + sato}
> )}
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Confusing print method for Inf dates
On Wed, Oct 21, 2015 at 4:57 PM, Hadley Wickham <h.wick...@gmail.com> wrote: > x <- as.Date(Inf, origin = "1970-01-01") > x > #> [1] NA > str(x) > #> Date[1:1], format: NA > unclass(x) > #> [1] Inf > > It's not clear what the correct behaviour is. The documentation for > ?Date has: "It is intended that the date should be an integer,", which > suggests that -Inf and Inf are not valid dates. But if that's true the You omitted the second half of the sentence, which contains important information. The entire sentence is, "It is intended that the date should be an integer, but this is not enforced in the internal representation." Since it's not enforced internally, it doesn't necessarily follow that non-integer values are invalid dates. The rest of the paragraph describes how fractional (internal) dates can be created. Both ?format.Date and ?strptime say that 'NA' dates/times are printed as NA_character_. It might be clearer to say that "invalid" dates/times are printed as NA_character_. > behaviour for max.Date() needs some thought: > > max(as.Date(NA), na.rm = TRUE) > #> Warning in max.default(structure(NA_real_, class = "Date"), na.rm = TRUE): > #> no non-missing arguments to max; returning -Inf > #> [1] NA > > If dates are integers, then there is no date that is smaller than all > other dates, so it's not clear what max() should return - NA? > But they're not integers in the strict sense. > Hadley > > -- > http://had.co.nz/ > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Weird issue when iterating through dates
On Wed, Aug 12, 2015 at 10:55 AM, Gábor Csárdi csardi.ga...@gmail.com wrote:
I am not sure if this is a bug or not.
I would argue that this isn't a bug, not even in the documentation of
for (even though it might be clearer). ?for says that `seq` is
[A]n expression evaluating to a vector (including a list and an
expression) or to a pairlist or 'NULL'. Date objects aren't strictly
vectors, so they're treated as integer/numeric.
This answer on StackOverflow said that for does not copy any of the
iterators attributes (including class), which causes this behavior.
http://stackoverflow.com/a/23278464/271616
To respond to the original question regarding why the code below,
prints the dates as a string. Quite simply, you convert seq(d1,d2,
by=1) to character, so it's no longer a Date. The fact that
Sys.Date() and as.character(Sys.Date()) both *print* the same thing
does not mean they are the same.
for ( dt in as.character(seq(d1,d2, by=1)) ) {
print(dt)
}
Best,
Josh
Gabor
On Wed, Aug 12, 2015 at 11:51 AM, Luca Cerone luca.cer...@gmail.com wrote:
Following up on this, should I report a bug? can you drive me through
the process?
Cheers,
Luca
On Thu, Aug 6, 2015 at 4:55 PM, William Dunlap wdun...@tibco.com wrote:
Just a quick question: what's the difference between `[.Date` and
`[[.Date`?
Is it supposed to be the method for accessing the value right?
For Dates and atomic vectors in general they are the same, but ...
Even for atomic vectors with names they are not quite the same
c(One=1, Two=2)[[2]]
[1] 2
c(One=1, Two=2)[2]
Two
2
(and [[ will only return 1 item, unlike [).
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Thu, Aug 6, 2015 at 5:36 AM, Gábor Csárdi csardi.ga...@gmail.com wrote:
On Thu, Aug 6, 2015 at 6:30 AM, Luca Cerone luca.cer...@gmail.com wrote:
[...]
Just a quick question: what's the difference between `[.Date` and
`[[.Date`?
Is it supposed to be the method for accessing the value right?
For Dates and atomic vectors in general they are the same, but in
general they are two different operators that behave differently on
some data types. E.g. on lists [ selects a sub-list and [[ selects a
single element.
Gabor
[...]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Working with connections
On Sun, Aug 9, 2015 at 8:59 AM, Glenn Schultz glennmschu...@me.com wrote:
Hi All,
I use connections to open and close data folders needed by my package.
After each function closes I get the following warnings (depending on the
connection that has been opened).
10: closing unused connection 3
(/Library/Frameworks/R.framework/Versions/3.2/Resources/library/BondLab/BondData/bondlabMBS4.rds)
Below is the connection function that is related to the above warning:
#
#' A connection function to BondData calling MBS cusps
#'
#' Opens a connection to the BondData folder to call MBS cusip data
#' @param MBS.id A character string the MBS.id or cusip number current
MBS.id is supported
#' @export
MBS - function(MBS.id = character){
MBS.Conn - gzfile(description = paste(system.file(package
= BondLab),
/BondData/, MBS.id, .rds, sep = ), open
= rb)
MBS - readRDS(MBS.Conn)
return(MBS)
close.connection(MBS.Conn)
}
I have googled this warning and it seems to be triggered when a function
terminates and the connection is open. But, I think the connection function
closes the connection once the object is returned. What am I doing wrong?
Your call to return() exits the function, so the close.connection()
call is never evaluated. Considering using on.exit() to close the
connection, since it will close the connection regardless of how the
function exits (e.g. because of an error).
-Glenn
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] R CMD build failure
On Thu, Jul 9, 2015 at 12:26 PM, Therneau, Terry M., Ph.D. thern...@mayo.edu wrote: I have a local library 'dart' that imports httr. It has routines that access central patient data such as birth date, so it is heavily used locally but of no interest to anyone else. The httr library (and 300 others) are in a shared directory, referenced by everyone in the biostatistics group via adding this location to the .libPaths in their default .Rprofile. (Actually, their .Rprofile starts by running material from a central one, the libPaths is there). When I run R (3.2.0) all is fine, but R CMD build fails with the text below * creating vignettes ... ERROR Error: processing vignette 'dart.Rnw' failed with diagnostics: chunk 2 (label = auth1) Error : package ‘httr’ required by ‘dart’ could not be found Execution halted If I add the requiste directory to my R_LIBS_USER environment variable then all is fine. However, that's a nuisance since the location changes over time (e.g. R releases). The system admins have a whole process that keeps .bashrc, .Rprofle and etc dot references up to date. Plugging into this is why we use .Rprofile. They are quite willing to make select changes in the central file, but with 1000 users any suggested changes in the overall process do not get a warm welcome. Any ideas? There is no mention in the Writing R Extentions manual that it ignores the Rprofile file. If suck it up and use R_LIBS_USER is the answer, well, there are only a few who build packages. It is mentioned in ?Startup: 'R CMD check' and 'R CMD build' do not always read the standard startup files, but they do always read specific 'Renviron' files. The location of these can be controlled by the environment variables 'R_CHECK_ENVIRON' and 'R_BUILD_ENVIRON'. If these are set their value is used as the path for the 'Renviron' file; otherwise, files '~/.R/check.Renviron' or '~/.R/build.Renviron' or sub-architecture-specific versions are employed. Maybe one of those options could work for you? PS, I can't use RHOME:/etc/Rprofile.site since the biostat group is not the only set of R users. Some other groups, for instance, cannot even see our central area. Terry T. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] download.file and https
Hi Paul,
On Thu, Jul 2, 2015 at 9:46 PM, Paul Gilbert pgilbert...@gmail.com wrote:
(This problem with download.file() affects quantmod, and possibly several
other packages. e.g. getSymbols('M2',src='FRED') fails.)
Thanks for the note. I'm aware of this, and actually wanted to ask
this list what the best practice was for downloading files from
https URLs.
I think the St Louis Fed has moved to using https for connections, and I
believe all the US government web sites are doing this. An http request is
automatically switched to https. The default download.file method does not
seem to handle this, but method=wget does:
IIUC, only if your system has wget installed and it is on the search
path. Another user has suggested setting method=curl, but that has
the same limitations. I could use RCurl to access these files from
FRED, but I'd prefer a solution that doesn't require an additional
package dependency, if possible.
I don't have a lot of experience with downloading from https URLs in R
(especially on Windows) so I would really appreciate some pointers on
how best to handle this.
tmp - tempfile()
download.file(http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv;,
destfile = tmp)
trying URL
'http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv'
Error in
download.file(http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv;,
:
cannot open URL
'http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv'
download.file(http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv;,
destfile = tmp, method=wget)
snip wget output
[34519]
Paul
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] R CMD check: Uses the superseded package: ‘doSNOW’
On Tue, Feb 10, 2015 at 5:10 AM, Xavier Robin ro...@lindinglab.org wrote:
Oh, I completely missed that one.
It's very neat as it seems to work both on Windows and Unix.
It works on both Windows and *nix because it combines functionality
from snow (Windows) and multicore (*nix).
Thanks!
Xavier
On 10/02/15 10:52, Martyn Plummer wrote:
The CRAN package snow is superseded by the parallel package which is
distributed with R since version 2.14.0. Here are the release notes
\item There is a new package \pkg{parallel}.
It incorporates (slightly revised) copies of packages
\CRANpkg{multicore} and \CRANpkg{snow} (excluding MPI, PVM and NWS
clusters). Code written to use the higher-level API functions in
those packages should work unchanged (apart from changing any
references to their namespaces to a reference to \pkg{parallel},
and links explicitly to \CRANpkg{multicore} or \CRANpkg{snow} on help
pages).
So you should replace your dependency on doSNOW with doParallel, which
is the equivalent foreach adapter for the parallel package.
Martyn
On Mon, 2015-02-09 at 23:08 +0100, Xavier Robin wrote:
Dear list,
When I run an R CMD check --as-cran on my package (pROC) I get the
following note:
Uses the superseded package: ‘doSNOW’
The fact that it uses the doSNOW package is correct as I have the
following example in an .Rd file:
#ifdef windows
if (require(doSNOW)) {
registerDoSNOW(cl - makeCluster(2, type = SOCK))
ci(roc2, method=bootstrap, parallel=TRUE)
\dontrun{ci(roc2, method=bootstrap, parallel=TRUE)}
\dontshow{ci(roc2, method=bootstrap, parallel=TRUE, boot.n=20)}
stopCluster(cl)
}
#endif
#ifdef unix
if (require(doMC)) {
registerDoMC(2)
\dontrun{ci(roc2, method=bootstrap, parallel=TRUE)}
\dontshow{ci(roc2, method=bootstrap, parallel=TRUE, boot.n=20)}
}
#endif
The superseded part is more confusing to me, though. The doSNOW
package seems to be still available on CRAN with no special notice,
listed in the HighPerformanceComputing view likewise, and under active
development (last change a couple of days ago on R-Forge). I could find
no mention of what it has been superseded with. Surprisingly, Google was
no help on this.
I could see the note is triggered in QC.R file of the tools package.
However this finding is not much help and leaves me just as confused as
before.
I recall spending quite some time to setup this example to run both
under Windows and Unix. doSNOW was the only way I could get it to work
there. doMC is apparently still available for Unix only. I couldn't get
doRNG to work on either platforms.
So what is R CMD check noticing me about?
Should I ignore the notice, or take an action? If so, which one?
Best wishes,
Xavier
--
Xavier Robin, PhD
Cellular Signal Integration Group (C-SIG) - Linding Lab
Biotech Research and Innovation Center (BRIC) - University of Copenhagen
Anker Engelundsvej, DTU Campus, Building 301, DK-2800 Lyngby, DENMARK
Mobile: +45 42 799 833
www.lindinglab.org - www.bric.ku.dk
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [PATCH] Makefile: add support for git svn clones
On Mon, Jan 19, 2015 at 3:13 PM, Nathan Kurz n...@verse.com wrote: On Mon, Jan 19, 2015 at 1:00 PM, Felipe Balbi ba...@kernel.org wrote: I just thought that such a small patch which causes no visible change to SVN users and allow for git users to build R would be acceptable, but if it isn't, that's fine too. Felipe --- It would appear that you are unaware that you are walking a minefield of entrenched positions and personality conflicts. For those like myself who are mystified by the positions taken in this thread, a partial back story may be helpful. In 2012, Han-Tak Leung reported a problem compiling the development version of R that he had checked out using git's svn compability feature: https://stat.ethz.ch/pipermail/r-devel/2012-October/065133.html You forgot to mention that, in February of 2013, Hin-Tak reported that Matrix did not build with the R trunk since October, 2012; but it turned out not to build because he didn't have a subversion checkout. https://mailman.stat.ethz.ch/pipermail/r-devel/2013-February/065858.html In 2013, Brian Ripley applied a patch with the comment trap HK Leung misuse explicitly to prevent users from being able to do this: https://github.com/wch/r-source/commit/4f13e5325dfbcb9fc8f55fc6027af9ae9c7750a3 Shortly thereafter, Han-Tak tried to start discussion on this list about that patch, suggesting that preventing the use of non-SVN mirrors reduced the frequency with which development versions would be tested: https://stat.ethz.ch/pipermail/r-devel/2013-March/066128.html The opinions expressed on the thread were universally against Leung. Peter Dalgaard summarized as: The generic point is that you are given access to a working tool that is internal to the core R developers. We are not putting restrictions on what you do with that access, but if you want to play the game by other rules than we do, you need to take the consequences. If things don't work and you start complaining about them being broken, steps may be taken to make it clearer who broke them. https://stat.ethz.ch/pipermail/r-devel/2013-March/066131.html As a newcomer hoping to contribute to R who had already encountered this same compilation issue and considered it was a bug, I am astounded to learn that it is instead desired and intentional behavior. --nate __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Problem with build and check
On Wed, Nov 12, 2014 at 1:28 PM, Duncan Murdoch
murdoch.dun...@gmail.com wrote:
On 12/11/2014 2:11 PM, Therneau, Terry M., Ph.D. wrote:
I am getting failure of build and check, for an Rd file that has a long
argument list.
Guess diagnosis: a quoted string beyond a certain point in the argument
list is fatal.
No, the problem is that % is a comment marker in .Rd. You need to escape
those in the dateformat default. Apparently
prompt() doesn't know this...
Duncan Murdoch
prompt.default does add the escapes for the example section, but not
for the usage section.
Example: Use the function below, create an Rd file for it with prompt().
Move the .Rd
file to the man directory (no need to edit it) and try building
dart.control - function(server=c(production, integration,
development,
http),
out.poll.duration = 5,
out.poll.increase = 1.1,
out.poll.max = 30,
out.poll.timeout = 3600,
netrc.path,
netrc.server = ldap,
rtype = c(xml, json),
dateformat= %Y-%m-%d) {
server - match.arg(server)
server
}
I created a package dummy with only this function, and get the following
on my Linux box.
tmt-local2021% R CMD build dummy
* checking for file ‘dummy/DESCRIPTION’ ... OK
* preparing ‘dummy’:
* checking DESCRIPTION meta-information ... OK
Warning: newline within quoted string at dart.control.Rd:11
Warning: /tmp/RtmpjPjz9V/Rbuild398d6e382572/dummy/man/dart.control.Rd:46:
unexpected
section header '\value'
Warning: newline within quoted string at dart.control.Rd:11
Error in
parse_Rd(/tmp/RtmpjPjz9V/Rbuild398d6e382572/dummy/man/dart.control.Rd, :
Unexpected end of input (in quoted string opened at
dart.control.Rd:88:16)
Execution halted
Session info for my version
sessionInfo()
R Under development (unstable) (2014-10-30 r66907)
Platform: i686-pc-linux-gnu (32-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8LC_COLLATE=C
[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
Terry T.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] subscripting a data.frame (without changing row order) changes internal row.names
On Mon, Nov 10, 2014 at 12:35 PM, Dr Gregory Jefferis jeffe...@mrc-lmb.cam.ac.uk wrote: Dear R-devel, Can anyone help me to understand this? It seems that subscripting the rows of a data.frame without actually changing their order, somehow changes an internal representation of row.names that is revealed by e.g. dput/dump/serialize I have read the docs and inspected the (R) code for data.frame, rownames, row.names and dput without enlightenment. Look at ?.row_names_info (which is mentioned in the See Also section of ?row.names) and its type argument. Also see the discussion here: http://stackoverflow.com/q/26468746/271616 df=data.frame(a=1:10, b=1) dput(df) df2=df[1:nrow(df), ] # R thinks they are equal (so do I!) all.equal(df, df2) dput(df2) Looking at the output of the dputs dput(df) structure(list(a = 1:10, b = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)), .Names = c(a, b), row.names = c(NA, -10L), class = data.frame) dput(df2) structure(list(a = 1:10, b = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)), .Names = c(a, b), row.names = c(NA, 10L), class = data.frame) we have row.names = c(NA, -10L) in the first case and row.names = c(NA, 10L) in the second, so somehow these objects have a different representation Can anyone explain why? This has come up because The first are automatic. The second are a compact form of 1:10, as mentioned in ?row.names. I'm not certain of the root cause/reason, but the second object will not have automatic rownames because you have subset it with a non-missing 'i'. library(digest) digest(df)==digest(df2) [1] FALSE digest uses serialize under the hood, but serialize, dput and dump all show the same effect (I've pasted an example below using dump, md5sum from base R). Many thanks for any enlightenment! More generally is there any way to calculate a digest of a data.frame that could get round this issue or is that not possible? Best wishes, Greg. A digest using base R: library(tools) td=tempfile() dir.create(td) tempfiles=file.path(td,c(df, df2)) dump(df,tempfiles[1]) dump(df2,tempfiles[2]) md5sum(tempfiles) # different md5sum sessionInfo() # for my laptop but also observed on R 3.1.2 R version 3.1.1 (2014-07-10) Platform: x86_64-apple-darwin13.1.0 (64-bit) locale: [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 attached base packages: [1] tools stats graphics grDevices utils datasets methods base other attached packages: [1] nat_1.5.14 nat.utils_0.4.2 digest_0.6.4Rvcg_0.9 devtools_1.6.1 igraph_0.7.1 [7] testthat_0.9.1 rgl_0.93.1098 loaded via a namespace (and not attached): [1] codetools_0.2-9 filehash_2.2-2nabor_0.4.3 parallel_3.1.1 plyr_1.8.1 [6] Rcpp_0.11.3 rstudio_0.98.1062 rstudioapi_0.1XML_3.98-1.1 yaml_2.1.13 -- Gregory Jefferis, PhD Division of Neurobiology MRC Laboratory of Molecular Biology Francis Crick Avenue Cambridge Biomedical Campus Cambridge, CB2 OQH, UK http://www2.mrc-lmb.cam.ac.uk/group-leaders/h-to-m/g-jefferis http://jefferislab.org http://flybrain.stanford.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Unexpected behavior of identical() with language objects
On Wed, Oct 29, 2014 at 3:26 PM, Winston Chang winstoncha...@gmail.com wrote:
I ran into this and found the result very surprising:
identical( quote({ a }), quote({ a }) )
# FALSE
It seems related to curly braces. For example, parens work fine:
identical( quote(( a )), quote(( a )) )
# TRUE
Is this expected behavior? I can't seem to find anything in the help
for identical that relates to this.
It's not in ?identical, but ?Paren gives you some pointers.
str(quote((a))) and str(quote({a})) are also informative.
-Winston
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Fastest way to merge 300+ .5MB dataframes?
The same comment Jeroen Ooms made about your last email also applies to this one: it is better suited to R-help. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sun, Aug 10, 2014 at 1:18 PM, Grant Rettke g...@wisdomandwonder.com wrote: Good afternoon, Today I was working on a practice problem. It was simple, and perhaps even realistic. It looked like this: • Get a list of all the data files in a directory • Load each file into a dataframe • Merge them into a single data frame Because all of the columns were the same, the simplest solution in my mind was to `Reduce' the vector of dataframes with a call to `merge'. That worked fine, I got what was expected. That is key actually. It is literally a one-liner, and there will never be index or scoping errors with it. Now with that in mind, what is the idiomatic way? Do people usually do something else because it is /faster/ (by some definition)? Kind regards, Grant Rettke | ACM, ASA, FSF, IEEE, SIAM g...@wisdomandwonder.com | http://www.wisdomandwonder.com/ “Wisdom begins in wonder.” --Socrates ((λ (x) (x x)) (λ (x) (x x))) “Life has become immeasurably better since I have been forced to stop taking it seriously.” --Thompson __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Typeof for character vector in dataframe returns integer
On Apr 1, 2014 7:48 PM, Sandip Nandi sanna...@umail.iu.edu wrote: Hi , I want to know is this behavior expected and why is that ? Need some help gender - c(F, M, M, F, F, M, F, F) age- c(23, 25, 27, 29, 31, 33, 35, 37) df- data.frame(gender,age) typeof(df[[1]]) [1] integer Why is this integer . *Should not it be character ?* typeof(df[[2]]) [1] double typeof(gender) [1] character typeof(age) [1] double In my code i am trying to do some thing based on typeof and the type for character column is strange. The first column is coerced to factor when stringsAsFactors=TRUE, which is the default for data.frame. See ?data.frame. Thanks, Sandip [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Typeof for character vector in dataframe returns integer
On Apr 1, 2014 8:04 PM, Sandip Nandi sanna...@umail.iu.edu wrote: Hi , I am concerned about the return value of typeof in character column in a dataframe. I expect it to be of character ,it returns integer instead . Of course typeof returns integer; typeof returns the internal storage type of an object. Factors are integer vectors with a levels attribute. See ?typeof and ?factor. As I said, it's not a character column. data.frame coerced it to factor. Thanks On Tue, Apr 1, 2014 at 5:56 PM, Joshua Ulrich josh.m.ulr...@gmail.com wrote: On Apr 1, 2014 7:48 PM, Sandip Nandi sanna...@umail.iu.edu wrote: Hi , I want to know is this behavior expected and why is that ? Need some help gender - c(F, M, M, F, F, M, F, F) age- c(23, 25, 27, 29, 31, 33, 35, 37) df- data.frame(gender,age) typeof(df[[1]]) [1] integer Why is this integer . *Should not it be character ?* typeof(df[[2]]) [1] double typeof(gender) [1] character typeof(age) [1] double In my code i am trying to do some thing based on typeof and the type for character column is strange. The first column is coerced to factor when stringsAsFactors=TRUE, which is the default for data.frame. See ?data.frame. Thanks, Sandip [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Tue, Mar 18, 2014 at 3:24 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: snip ## Summary Extending the r-release cycle to CRAN seems like a solution that would be easy to implement. Package updates simply only get pushed to the r-devel branches of cran, rather than r-release and r-release-old. This separates development from production/use in a way that is common sense in most open source communities. Benefits for R include: Nothing is ever as simple as it seems (especially from the perspective of one who won't be doing the work). There is nothing preventing you (or anyone else) from creating repositories that do what you suggest. Create a CRAN mirror (or more than one) that only include the package versions you think they should. Then have your production servers use it (them) instead of CRAN. Better yet, make those repositories public. If many people like your idea, they will use your new repositories instead of CRAN. There is no reason to impose this change on all world-wide CRAN users. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 12:59 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Mar 19, 2014 at 5:52 AM, Duncan Murdoch murdoch.dun...@gmail.comwrote: I don't see why CRAN needs to be involved in this effort at all. A third party could take snapshots of CRAN at R release dates, and make those available to package users in a separate repository. It is not hard to set a different repository than CRAN as the default location from which to obtain packages. I am happy to see many people giving this some thought and engage in the discussion. Several have suggested that staging freezing can be simply done by a third party. This solution and its limitations is also described in the paper [1] in the section titled R: downstream staging and repackaging. If this would solve the problem without affecting CRAN, we would have been done this obviously. In fact, as described in the paper and pointed out by some people, initiatives such as Debian or Revolution Enterprise already include a frozen library of R packages. Also companies like Google maintain their own internal repository with packages that are used throughout the company. The suggested solution is not described in the referenced article. It was not suggested that it be the operating system's responsibility to distribute snapshots, nor was it suggested to create binary repositories for specific operating systems, nor was it suggested to freeze only a subset of CRAN packages. The problem with this approach is that when you using some 3rd party package snapshot, your r/sweave scripts will still only be reliable/reproducible for other users of that specific snapshot. E.g. for the examples above, a script that is written in R 3.0 by a Debian user is not guaranteed to work on R 3.0 in Google, or R 3.0 on some other 3rd party cran snapshot. Hence this solution merely redefines the problem from this script depends on pkgA 1.1 and pkgB 0.2.3 to this script depends on repository foo 2.0. And given that most users would still be pulling packages straight from CRAN, it would still be terribly difficult to reproduce a 5 year old sweave script from e.g. JSS. This can be solved by the third party making the repository public. For this reason I believe the only effective place to organize this staging is all the way upstream, on CRAN. Imagine a world where your r/sweave script would be reliable/reproducible, out of the box, on any system, any platform in any company using on R 3.0. No need to investigate which specific packages or cran snapshot the author was using at the time of writing the script, and trying to reconstruct such libraries for each script you want to reproduce. No ambiguity about which package versions are used by R 3.0. However for better or worse, I think this could only be accomplished with a cran release cycle (i.e. universal snapshots) accompanying the already existing r releases. This could be done by a public third-party repository, independent of CRAN. However, you would need to find a way to actively _prevent_ people from installing newer versions of packages with the stable R releases. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 4:28 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Mar 19, 2014 at 11:50 AM, Joshua Ulrich josh.m.ulr...@gmail.com wrote: The suggested solution is not described in the referenced article. It was not suggested that it be the operating system's responsibility to distribute snapshots, nor was it suggested to create binary repositories for specific operating systems, nor was it suggested to freeze only a subset of CRAN packages. IMO this is an implementation detail. If we could all agree on a particular set of cran packages to be used with a certain release of R, then it doesn't matter how the 'snapshotting' gets implemented. It could be a separate repository, or a directory on cran with symbolic links, or a page somewhere with hyperlinks to the respective source packages. Or you can put all packages in a big zip file, or include it in your OS distribution. You can even distribute your entire repo on cdroms (debian style!) or do all of the above. The hard problem is not implementation. The hard part is that for reproducibility to work, we need community wide conventions on which versions of cran packages are used by a particular release of R. Local downstream solutions are impractical, because this results in scripts/packages that only work within your niche using this particular snapshot. I expect that requiring every script be executed in the context of dependencies from some particular third party repository will make reproducibility even less common. Therefore I am trying to make a case for a solution that would naturally improve reliability/reproducibility of R code without any effort by the end-user. So implementation isn't a problem. The problem is that you need a way to force people not to be able to use different package versions than what existed at the time of each R release. I said this in my previous email, but you removed and did not address it: However, you would need to find a way to actively _prevent_ people from installing newer versions of packages with the stable R releases. Frankly, I would stop using CRAN if this policy were adopted. I suggest you go build this yourself. You have all the code available on CRAN, and the dates at which each package was published. If others who care about reproducible research find what you've built useful, you will create the very community you want. And you won't have to force one single person to change their workflow. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [RFC] A case for freezing CRAN
On Wed, Mar 19, 2014 at 5:16 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Mar 19, 2014 at 2:59 PM, Joshua Ulrich josh.m.ulr...@gmail.com wrote: So implementation isn't a problem. The problem is that you need a way to force people not to be able to use different package versions than what existed at the time of each R release. I said this in my previous email, but you removed and did not address it: However, you would need to find a way to actively _prevent_ people from installing newer versions of packages with the stable R releases. Frankly, I would stop using CRAN if this policy were adopted. I am not proposing to force anything to anyone, those are your words. Please read the proposal more carefully before derailing the discussion. Below *verbatim* a section from the paper: snip Yes force is too strong a word. You want a barrier (however small) to prevent people from installing newer (or older) versions of packages than those that correspond to a given R release. I still think you're going to have a very hard time convincing CRAN maintainers to take up your cause, even if you were to build support for it. Especially because there's nothing stopping anyone else from doing it. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Regarding: stat.math.ethz.ch mailing list memberships reminder
Please read the instructions/warnings before subscribing. You may enter a privacy password below. This provides only mild security, but should prevent others from messing with your subscription. Do not use a valuable password as it will occasionally be emailed back to you in cleartext. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Fri, Mar 7, 2014 at 11:17 AM, Sachko Honda sachko.ho...@mountainpacificgroup.com wrote: Please never ever send the password in clear text, never!!! Sachko Honda Phone: 425.284.7200 Fax: 425.284.7201 sachko.ho...@mountainpacificgroup.com Mountain Pacific Group 11820 Northup Way, Suite E210 Bellevue, WA 98005-1926 [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Possible POSIXlt / wday glitch bugs.r-project.org status
In an effort to redeem myself, I have found and submitted a patch for what seems to be causing this issue. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sat, Oct 5, 2013 at 1:18 PM, Simon Urbanek simon.urba...@r-project.org wrote: On Oct 5, 2013, at 4:51 PM, Sean O'Riordain sean...@acm.org wrote: Some people (luckily not me anymore!) working with mortgages and pensions need to calculate up to 40 years into the future for the payment schedule. Just to clarify since the Joshua's comment was ambiguous (and in part plain wrong) - R's POSIXct has no such limit since it doesn't use integers, so that is not really the issue here. As the original post suggested there may be a bug in handing some cases where conversions hit the system libraries (that may truncate to integers) and some cases may be worked around - and that remains to be investigated. Cheers, Simon On 5 October 2013 02:37, Joshua Ulrich josh.m.ulr...@gmail.com wrote: On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan i.costi...@me.com wrote: Thanks for the responses and quoting the timezone help file. I am assuming that in order to determine the wday element of POSIXlt, R does the necessary calculations in Julian time (via POSIXct). Based on this excerpt from ?DateTimeClasses, it looks like R is responsible for determining time zones post 2037 (the example I gave was in 2038). So it could be an R issue. It's an issue with size of the largest number you can store in a signed integer, which is not specific to R. .POSIXct(.Machine$integer.max, tz=UTC) [1] 2038-01-19 03:14:07 UTC Dates larger than that cannot be represented by a signed integer. It could be worked around, but it's not trivial because R would have to use something other than the tm C struct. Luckily, there's a decade or two before it starts to become a pressing issue. :) ‘POSIXct’ objects may also have an attribute ‘tzone’, a character vector of length one. If set to a non-empty value, it will determine how the object is converted to class ‘POSIXlt’ and in particular how it is printed. This is usually desirable, but if you want to specify an object in a particular timezone but to be printed in the current timezone you may want to remove the ‘tzone’ attribute (e.g. by ‘c(x)’). Unfortunately, the conversion is complicated by the operation of time zones and leap seconds (24 days have been 86401 seconds long so far: the times of the extra seconds are in the object ‘.leap.seconds’). **The details of this are entrusted to the OS services where possible. This always covers the period 1970-2037, and on most machines back to 1902 (when time zones were in their infancy). Outside the platform limits we use our own C code. On 05/10/2013, at 12:59 AM, Scott Kostyshak skost...@princeton.edu wrote: On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan i.costi...@me.com wrote: Wanted to raise two questions: 1. Is bugs.r-project.org down? I haven't been able to reach it for two or three days: Yes. Quote from Duncan: ... the server is currently down. The volunteer who runs the server is currently away from his office, so I expect it won't get fixed until he gets back in a few days. https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html Scott ``` ping bugs.r-project.org PING rbugs.research.att.com (207.140.168.137): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 Request timeout for icmp_seq 3 Request timeout for icmp_seq 4 Request timeout for icmp_seq 5 Request timeout for icmp_seq 6 ``` 2. Is wday element of POSIXlt meant to be timezone invariant? You would expect the wday element to be invariant to the timezone of a date. That is, the same date/time instant of 5th October 2013 in both Australia/Sydney and UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 min past midnight on 5 October 2013: ``` library(lubridate) d_utc - ymd_hms(2013100501, tz='UTC') d_local - ymd_hms(2013100501, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6 ``` But this isn't always the case. For example, ``` d_utc - ymd_hms(2038100201, tz='UTC') d_local - ymd_hms(2038100201, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5 ``` Is this expected behaviour? I would have expected a properly encoded date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone. Obligatory system dump: ``` sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin12.4.0 (64-bit) locale: [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other
Re: [Rd] read.table() with quoted integers
On Tue, Oct 1, 2013 at 11:29 AM, David Winsemius dwinsem...@comcast.net wrote: On Sep 30, 2013, at 6:38 AM, Joshua Ulrich wrote: On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat nalimi...@club.fr wrote: Hi! It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider quoted integers as an acceptable value for columns for which colClasses=integer. But when colClasses is omitted, these columns are read as integer anyway. For example, let's consider a file named file.dat, containing: 1 2 read.table(file.dat, colClasses=integer) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : scan() expected 'an integer' and got '1' But: str(read.table(file.dat)) 'data.frame': 2 obs. of 1 variable: $ V1: int 1 2 The latter result is indeed documented in ?read.table: Unless ‘colClasses’ is specified, all columns are read as character columns and then converted using ‘type.convert’ to logical, integer, numeric, complex or (depending on ‘as.is’) factor as appropriate. Quotes are (by default) interpreted in all fields, so a column of values like ‘42’ will result in an integer column. Should the former behavior be considered a bug? No. If you tell read.table the column is integer and it's actually character on disk, it should be an error. My reading of the `read.table` help page is that one should expect that when there is an 'integer'-class and an `as.integer` function and integer is the argument to colClasses, that `as.integer` will be applied to the values in the column. Should I be reading elsewhere? I assume you're referring to the paragraph below. Possible values are ‘NA’ (the default, when ‘type.convert’ is used), ‘NULL’ (when the column is skipped), one of the atomic vector classes (logical, integer, numeric, complex, character, raw), or ‘factor’, ‘Date’ or ‘POSIXct’. Otherwise there needs to be an ‘as’ method (from package ‘methods’) for conversion from ‘character’ to the specified formal class. I read that as meaning that an as method is required for classes not already listed in the prior sentence. It doesn't say an as method will be applied if colClasses is one of the atomic, factor, Date, or POSIXct classes; but I can see how you might assume that, since all the atomic, factor, Date, and POSIXct classes already have as methods... Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Thu, Oct 3, 2013 at 9:44 AM, Jens Oehlschlägel
jens.oehlschlae...@truecluster.com wrote:
I agree that quoted integer columns are not the most efficient way of
delivering csv-files. However, the sad reality is that one receives such
formats and still needs to read the data. Therefore it is not helpful to
state that one should 'consider character to be the correct colClass in
case an integer is surrounded by quotes'.
The philosophy of read.table.ffdf is delegating the actual csv-parsing to a
parse engine 'similarly' parametrized like 'read.table'. It is not 'bad
coding practice' - but a conscious design decision - to assume that the
parse engine behaves consistently, which read.table does not yet: it
automatically recognizes a quoted integer column as 'integer', but when
asked to explicitly interpret the column as 'integer' it does refuse to do
read.table() does not automatically recognize a quoted integer column
as 'integer'. If colClasses is not specified, it reads the entire
column into a 'character' vector and then calls type.convert() on it.
type.convert() does all the necessary work to determine what class the
'character' vector should be converted to. If colClasses is
specified, quotes are not interpreted in non-'character' columns.
You want scan() to allocate an 'integer' vector, and then ensure (on
each read from the column in the file) that the value read is a valid
'integer' type, while interpreting quotes (which strtol does not do,
so someone would have to write and test this new functionality).
So your complaint is more with scan() than read.table(). And more
with Strtoi() (and therefore strtol) than scan().
so. So there is nothing wrong with read.table.ffdf (but something can be
improved about read.table). It is *not* the 'best solution [...] to rewrite
read.table.ffdf()' given that it nicely imports such data, see 4+1 ways to
do so below.
Jens Oehlschlägel
# --- first create a csv file for demonstration
---
require(ff)
file - test.csv
path - c:/tmp
n - 1e2
d - data.frame(x=1:n, y=shQuote(1:n))
write.csv(d, file=file.path(path,file), row.names=FALSE, quote=FALSE)
# --- how to do it with read.table.ffdf
---
# 1 let the parse engine ignore colClasses and hope for the best
fixedengine - function(file, ..., colClasses=NA){
read.csv(file, ...)
}
df - read.table.ffdf(file=file.path(path,file), first.rows = 10,
FUN=fixedengine)
df
# 2 Suspend colClasses(=NA) for the quoted integer column only
df - read.csv.ffdf(file=file.path(path,file), first.rows = 10,
colClasses=c(integer, NA))
df
# 3 do your own type conversion using transFUN
# after reading the problematic column as character
# Being able to inject regexps is quite powerful isn't it?
# Or error handlinig in case of varying column format!
custominterp - function(d){
d[[2]] - as.integer(gsub('', '', d[[2]]))
d
}
df - read.table.ffdf(file=file.path(path,file), first.rows = 10,
colClasses=c(integer, character), FUN=read.csv, transFUN=custominterp)
df
# 4 do your own line parsing and type conversion
# Here you can even handle non-standard formats
# such as varying number of columns
customengine - function(file, header=TRUE, col.names, colClasses=NA,
nrows=0, skip=0, fileEncoding=, comment.char = ){
l - scan(file, what=character, nlines=nrows+header, skip=skip,
fileEncoding=fileEncoding, comment.char = comment.char)
s - do.call(rbind, strsplit(l, ,))
if (header){
d - data.frame(as.integer(s[-1,1]),
as.integer(gsub('','',s[-1,2])))
names(d) - s[1,]
}else{
d - data.frame(as.integer(s[,1]),
as.integer(gsub('','',s[,2])))
}
if (!missing(col.names))
names(d) - col.names
d
}
df - read.table.ffdf(file=file.path(path,file), first.rows = 10,
FUN=customengine)
df
# 5 use a parsing engine that can apply colClasses to quoted integers
# Unfortunately Henry Bengtson's readDataFrame does not work as a
# parse engine for read.table.ffdf because read.table.ffdf expects
# the parse engine to read successive chunks from a file connection
# while readDataFrame only accepts a filename as input file spec.
# Yes it has 'skip', but using that would reread the file from scratch
# for each chunk (O(N^2) costs)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Possible POSIXlt / wday glitch bugs.r-project.org status
Quoting from ?timezone: Note that except on Windows, the operation of time zones is an OS service, and even on Windows a third-party database is used and can be updated (see the section on ‘Time zone names’). Incorrect results will never be an R issue, so please ensure that you have the courtesy not to blame R for them. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Fri, Oct 4, 2013 at 5:11 AM, Imanuel Costigan i.costi...@me.com wrote: Wanted to raise two questions: 1. Is bugs.r-project.org down? I haven't been able to reach it for two or three days: ``` ping bugs.r-project.org PING rbugs.research.att.com (207.140.168.137): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 Request timeout for icmp_seq 3 Request timeout for icmp_seq 4 Request timeout for icmp_seq 5 Request timeout for icmp_seq 6 ``` 2. Is wday element of POSIXlt meant to be timezone invariant? You would expect the wday element to be invariant to the timezone of a date. That is, the same date/time instant of 5th October 2013 in both Australia/Sydney and UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 min past midnight on 5 October 2013: ``` library(lubridate) d_utc - ymd_hms(2013100501, tz='UTC') d_local - ymd_hms(2013100501, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6 ``` But this isn't always the case. For example, ``` d_utc - ymd_hms(2038100201, tz='UTC') d_local - ymd_hms(2038100201, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5 ``` Is this expected behaviour? I would have expected a properly encoded date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone. Obligatory system dump: ``` sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin12.4.0 (64-bit) locale: [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] lubridate_1.3.0 testthat_0.7.1 devtools_1.3 loaded via a namespace (and not attached): [1] colorspace_1.2-4 dichromat_2.0-0digest_0.6.3 evaluate_0.5.1 [5] ggplot2_0.9.3.1grid_3.0.1 gtable_0.1.2 httr_0.2 [9] labeling_0.2 MASS_7.3-29memoise_0.1munsell_0.4.2 [13] parallel_3.0.1 plyr_1.8 proto_0.3-10 RColorBrewer_1.0-5 [17] RCurl_1.95-4.1 reshape2_1.2.2 scales_0.2.3 stringr_0.6.2 [21] tools_3.0.1whisker_0.3-2 ``` Using R compiled by homebrew [1]. But also experiencing the same bug using R installed on Windows 7 from the CRAN binaries. For those interested, I've also noted this on the `lubridate` Github issues page [2], even though this doesn't appear to be a lubridate issue. Thanks for any help. [1] http://brew.sh [2] https://github.com/hadley/lubridate/issues/209 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Possible POSIXlt / wday glitch bugs.r-project.org status
On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan i.costi...@me.com wrote: Thanks for the responses and quoting the timezone help file. I am assuming that in order to determine the wday element of POSIXlt, R does the necessary calculations in Julian time (via POSIXct). Based on this excerpt from ?DateTimeClasses, it looks like R is responsible for determining time zones post 2037 (the example I gave was in 2038). So it could be an R issue. It's an issue with size of the largest number you can store in a signed integer, which is not specific to R. .POSIXct(.Machine$integer.max, tz=UTC) [1] 2038-01-19 03:14:07 UTC Dates larger than that cannot be represented by a signed integer. It could be worked around, but it's not trivial because R would have to use something other than the tm C struct. Luckily, there's a decade or two before it starts to become a pressing issue. :) ‘POSIXct’ objects may also have an attribute ‘tzone’, a character vector of length one. If set to a non-empty value, it will determine how the object is converted to class ‘POSIXlt’ and in particular how it is printed. This is usually desirable, but if you want to specify an object in a particular timezone but to be printed in the current timezone you may want to remove the ‘tzone’ attribute (e.g. by ‘c(x)’). Unfortunately, the conversion is complicated by the operation of time zones and leap seconds (24 days have been 86401 seconds long so far: the times of the extra seconds are in the object ‘.leap.seconds’). **The details of this are entrusted to the OS services where possible. This always covers the period 1970-2037, and on most machines back to 1902 (when time zones were in their infancy). Outside the platform limits we use our own C code. On 05/10/2013, at 12:59 AM, Scott Kostyshak skost...@princeton.edu wrote: On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan i.costi...@me.com wrote: Wanted to raise two questions: 1. Is bugs.r-project.org down? I haven't been able to reach it for two or three days: Yes. Quote from Duncan: ... the server is currently down. The volunteer who runs the server is currently away from his office, so I expect it won't get fixed until he gets back in a few days. https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html Scott ``` ping bugs.r-project.org PING rbugs.research.att.com (207.140.168.137): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 Request timeout for icmp_seq 3 Request timeout for icmp_seq 4 Request timeout for icmp_seq 5 Request timeout for icmp_seq 6 ``` 2. Is wday element of POSIXlt meant to be timezone invariant? You would expect the wday element to be invariant to the timezone of a date. That is, the same date/time instant of 5th October 2013 in both Australia/Sydney and UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 min past midnight on 5 October 2013: ``` library(lubridate) d_utc - ymd_hms(2013100501, tz='UTC') d_local - ymd_hms(2013100501, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6 ``` But this isn't always the case. For example, ``` d_utc - ymd_hms(2038100201, tz='UTC') d_local - ymd_hms(2038100201, tz='Australia/Sydney') as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6 as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5 ``` Is this expected behaviour? I would have expected a properly encoded date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone. Obligatory system dump: ``` sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin12.4.0 (64-bit) locale: [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] lubridate_1.3.0 testthat_0.7.1 devtools_1.3 loaded via a namespace (and not attached): [1] colorspace_1.2-4 dichromat_2.0-0digest_0.6.3 evaluate_0.5.1 [5] ggplot2_0.9.3.1grid_3.0.1 gtable_0.1.2 httr_0.2 [9] labeling_0.2 MASS_7.3-29memoise_0.1munsell_0.4.2 [13] parallel_3.0.1 plyr_1.8 proto_0.3-10 RColorBrewer_1.0-5 [17] RCurl_1.95-4.1 reshape2_1.2.2 scales_0.2.3 stringr_0.6.2 [21] tools_3.0.1whisker_0.3-2 ``` Using R compiled by homebrew [1]. But also experiencing the same bug using R installed on Windows 7 from the CRAN binaries. For those interested, I've also noted this on the `lubridate` Github issues page [2], even though this doesn't appear to be a lubridate issue. Thanks for any help. [1] http://brew.sh [2] https://github.com/hadley/lubridate/issues/209 __ R-devel@r-project.org mailing list
Re: [Rd] read.table() with quoted integers
On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat nalimi...@club.fr wrote: Hi! It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider quoted integers as an acceptable value for columns for which colClasses=integer. But when colClasses is omitted, these columns are read as integer anyway. For example, let's consider a file named file.dat, containing: 1 2 read.table(file.dat, colClasses=integer) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : scan() expected 'an integer' and got '1' But: str(read.table(file.dat)) 'data.frame': 2 obs. of 1 variable: $ V1: int 1 2 The latter result is indeed documented in ?read.table: Unless ‘colClasses’ is specified, all columns are read as character columns and then converted using ‘type.convert’ to logical, integer, numeric, complex or (depending on ‘as.is’) factor as appropriate. Quotes are (by default) interpreted in all fields, so a column of values like ‘42’ will result in an integer column. Should the former behavior be considered a bug? No. If you tell read.table the column is integer and it's actually character on disk, it should be an error. This creates problems when combined with read.table.ffdf from package ff, since this function tries to guess the column classes by reading the first rows of the file, and then passes colClasses to read.table to read the remaining rows by chunks. A column of quoted integers is correctly detected as integer in the first read, but read.table() fails in subsequent reads. This sounds like a issue with read.table.ffdf. The column of quoted integers is *incorrectly* detected as integer because they're actually character on disk. read.table.ffdf should rely on how the data are actually stored on disk (via as.is=TRUE), not how read.table might convert them once they're read into R. Regards __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] read.table() with quoted integers
On Mon, Sep 30, 2013 at 9:45 AM, Milan Bouchet-Valat nalimi...@club.fr wrote: Le lundi 30 septembre 2013 à 08:38 -0500, Joshua Ulrich a écrit : On Mon, Sep 30, 2013 at 7:33 AM, Milan Bouchet-Valat nalimi...@club.fr wrote: Hi! It seems that read.table() in R 3.0.1 (Linux 64-bit) does not consider quoted integers as an acceptable value for columns for which colClasses=integer. But when colClasses is omitted, these columns are read as integer anyway. For example, let's consider a file named file.dat, containing: 1 2 read.table(file.dat, colClasses=integer) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : scan() expected 'an integer' and got '1' But: str(read.table(file.dat)) 'data.frame': 2 obs. of 1 variable: $ V1: int 1 2 The latter result is indeed documented in ?read.table: Unless ‘colClasses’ is specified, all columns are read as character columns and then converted using ‘type.convert’ to logical, integer, numeric, complex or (depending on ‘as.is’) factor as appropriate. Quotes are (by default) interpreted in all fields, so a column of values like ‘42’ will result in an integer column. Should the former behavior be considered a bug? No. If you tell read.table the column is integer and it's actually character on disk, it should be an error. All values in a CSV file are stored as characters on disk, disregarding the fact that they are surrounded by quotes or not. 1 is saved as 00110001 (ASCII character #49), not 0001, nor 0001 (as would for example imply a 32 bit storage of integers). Yes, I'm aware that write.table creates a character representation of the data on disk. That's its purpose. writeBin is for writing actual binary representations. I thought you would understand that by actually character on disk I meant actually a quoted value. I assumed you would understand my intent. read.table uses scan to read the file. ?scan says: The allowed input for a numeric field is optional whitespace followed either ‘NA’ or an optional sign followed by a decimal or hexadecimal constant (see NumericConstants), or ‘NaN’, ‘Inf’ or ‘infinity’ (ignoring case). Out-of-range values are recorded as ‘Inf’, ‘-Inf’ or ‘0’. For an integer field the allowed input is optional whitespace, followed by either ‘NA’ or an optional sign and one or more digits (‘0-9’): all out-of-range values are converted to ‘NA_integer_’. There's no mention of quotes being allowed. So, with all due respect, please refrain from formulating such blatantly erroneous statements. So, with all due respect, please refrain from formulating such blatantly pedantic responses to someone trying to help you. Regards This creates problems when combined with read.table.ffdf from package ff, since this function tries to guess the column classes by reading the first rows of the file, and then passes colClasses to read.table to read the remaining rows by chunks. A column of quoted integers is correctly detected as integer in the first read, but read.table() fails in subsequent reads. This sounds like a issue with read.table.ffdf. The column of quoted integers is *incorrectly* detected as integer because they're actually character on disk. read.table.ffdf should rely on how the data are actually stored on disk (via as.is=TRUE), not how read.table might convert them once they're read into R. Regards __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Vignette problem and CRAN policies
As Berend Hasselman already told you, Then surf to the address given
at the end of each posting; go to the bottom of that page and follow
the instructions for unsubscribing.
Here's the link, so you don't have to scroll to the bottom of this
message: https://stat.ethz.ch/mailman/listinfo/r-devel
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Mon, Sep 23, 2013 at 4:57 PM, Eric Malitz eric.mal...@gmail.com wrote:
take me off here
On Mon, Sep 23, 2013 at 4:44 PM, Marc Schwartz marc_schwa...@me.com wrote:
Spencer,
FYI. I just noted in your post below the error message from WriteXLS
regarding TEXT::CSV_XS missing.
Please note that in version =3.0 of WriteXLS (current is 3.2.1), that is
no longer required and has been replaced by Text::CSV_PP, which is a Pure
Perl module and is included in the WriteXLS CRAN package to reduce the
dependencies on nonstandard external Perl modules.
Regards,
Marc Schwartz
On Sep 23, 2013, at 4:28 PM, Spencer Graves spencer.gra...@prodsyse.com
wrote:
Hello, All:
Professor Ripley is correct as usual: I misunderstood his original
statement of the problem.
He gave two possible solutions. I could not make the first
solution work, and I didn't try the second until someone else on this list
explained it in slightly more detail.
The correction passed R CMD check on my local computer. It has
been building on R-Forge since 2013-09-20 19:19:14+02. I hope this
completes soon enough for me to meet Ripley's Sept. 25 deadline for this
correction to sos.
Thanks again to Prof. Ripley and everyone else who took the time to
read my post.
Spencer
On 9/19/2013 12:00 AM, Prof Brian Ripley wrote:
This is nothing to do with CRAN policies (nor R).
The issue is that the current upquote.sty does not play with 'ae' fonts
as used by default by Sweave. The change is in TeX.
And that was what Spencer Graves was informed.
On 19/09/2013 04:35, Spencer Graves wrote:
Hello, All:
The vignette with the sos package used upquote.sty, required
for R Journal when it was published in 2009. Current CRAN policy
disallows upquote.sty, and I've so far not found a way to pass R CMD
check with sos without upquote.sty.
I changed sos.Rnw per an email exchange with Prof. Ripley without
solving the problem; see below. The key error messages (see the
results
of R CMD build below) appear to be sos.tex:16: LaTeX Error:
Environment article undefined and sos.tex:558: LaTeX Error:
\begin{document} ended by \end{article}. When the article worked, it
had bot \begin{document} and \begin{article}, with matching \end
statements for both. I've tried commenting out either without success.
The current nonworking code is available on R-Forge via anonymous
SVN checkout using svn checkout
svn://svn.r-forge.r-project.org/svnroot/rsitesearch/. Any suggestions
on how to fix this would be greatly appreciated.
Thanks,
Spencer
## COMPLETE RESULTS FROM R CMD check
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\sgravescd 2013
C:\Users\sgraves\2013cd R_pkgs
C:\Users\sgraves\2013\R_pkgscd sos
C:\Users\sgraves\2013\R_pkgs\soscd pkg
C:\Users\sgraves\2013\R_pkgs\sos\pkgR CMD build sos
* checking for file 'sos/DESCRIPTION' ... OK
* preparing 'sos':
* checking DESCRIPTION meta-information ... OK
* installing the package to re-build vignettes
* creating vignettes ... ERROR
Loading required package: brew
Attaching package: 'sos'
The following object is masked from 'package:utils':
?
Loading required package: WriteXLS
Perl found.
The following Perl modules were not found on this system:
Text::CSV_XS
If you have more than one Perl installation, be sure the correct one
was
used he
re.
Otherwise, please install the missing modules. See the package INSTALL
file for
more information.
Loading required package: RODBC
Warning in odbcUpdate(channel, query, mydata, coldata[m, ], test =
test, :
character data 'Adrian Baddeley adrian.badde...@uwa.edu.au and
Rolf Turner
r.tur...@auckland.ac.nz with substantial contributions of code by
Kasper Klitgaard Berthelsen;Abdollah Jalilian; Marie-Colette van
Liesho
ut; Ege Rubak; Dominic Schuhmacher;and Rasmus
Waagepetersen.
Additional contributionsby Q.W. Ang;S. Azaele; C. Beale;
R. Bernhardt; T. Bendtsen;A. Bevan; B. Biggerstaff; R.
Bivan
d; F. Bonneu; J. Burgos; S. Byers; Y.M. Chang;
J.B.
Che
n; I. Chernayavsky;Y.C. Chin; B. Christensen; J.-F.
Co
eurjolly; R. Corria Ainslie; M. de la Cruz; P. Dalgaard;
P.J. Dig
gle;P. Donnelly;I. Dryden; S. Eglen; O. Flores; N.
Funwi-Gabga
[Rd] Mailing Lists page, turning off HTML mail
Hello, I just noticed that the link to instructions on turning off HTML mail has been dead for quite some time. The last capture I could find on web.archive.org was in mid-2009. http://web.archive.org/web/20090625155306/http://www.expita.com/nomime.html For reference, the link is in the following sentence: For more details and instructions on turning off HTML for your e-mail software, see here. Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] What algorithm is R using to calculate mean?
This was also asked on StackOverflow:
http://stackoverflow.com/q/17866149/271616. Here is the answer I
posted:
This appears to be the updating method of West, 1979 [1] and it was
implemented in R-2.3.0 in response to PR#1228 [2].
I'm not positive this is the correct algorithm, since it was suggested
by Martin Maechler but implemented by Brian Ripley. I couldn't find a
reference in the source code or version control logs that listed the
actual algorithm used. It was implemented in cov.c in revision 37389
and in summary.c in revision 37393.
[1] http://dl.acm.org/citation.cfm?doid=359146.359153
[2] https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=1228
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
On Thu, Jul 25, 2013 at 2:44 PM, Zach Harrington
zach.harring...@gmail.com wrote:
I am curious to know what algorithm R's mean function uses. Is there some
reference to the numerical properties of this algorithm?
I found the following C code in summary.c:do_summary():
case REALSXP:
PROTECT(ans = allocVector(REALSXP, 1));
for (i = 0; i n; i++) s += REAL(x)[i];
s /= n;
if(R_FINITE((double)s)) {
for (i = 0; i n; i++) t += (REAL(x)[i] - s);
s += t/n;
}
REAL(ans)[0] = s;
break;
It seems to do a straight up mean:
for (i = 0; i n; i++) s += REAL(x)[i];
s /= n;
Then it adds what i assume is a numerical correction which seems to be the
mean difference from the mean of the data:
for (i = 0; i n; i++) t += (REAL(x)[i] - s);
s += t/n;
I haven't been able to track this algorithm down anywhere (mean is not a
great search term).
Any help would be much appreciated,
Zach Harrington
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] as.Date.POSIXct
Would it make sense for as.Date.POSIXct to not assume tz=UTC if the POSIXct object has a valid tzone attribute? Current behavior may be confusing in certain cases, for example: (d - structure(1090450800, tzone=Europe/Berlin, + class=c(POSIXct,POSIXt))) [1] 2004-07-22 01:00:00 CEST as.Date(d) [1] 2004-07-21 as.Date(as.POSIXlt(d)) [1] 2004-07-22 as.Date(d, tz=attr(d,'tzone')) [1] 2004-07-22 Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2013: Applied Finance with R | www.RinFinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Who does develop the R core and libs, and how / where is it hosted?
On Mon, Jan 14, 2013 at 2:29 PM, oliver oli...@first.in-berlin.de wrote: Hello, I saw Binaries, stable release-souzrces and daily snapshots of R, but not something like a repository, visible for the public (like on githb for example). Go to http://www.r-project.org and click the Developer Page link in the left-hand column. How is the R development handled, what repositories / source code versioning tools are used, who are the developers? Ibid, and Go to http://www.r-project.org and click the Contributors link in the left-hand column. And is there something like a plan with future goals, which are planned for the next releases? Are there areas, where help is needed? And in which way could support be done? Ciao, Oliver __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] scientific notation and comparison with character variable
It's expected. From ?: If the two arguments are atomic vectors of different types, one is coerced to the type of the other, the (decreasing) order of precedence being character, complex, numeric, integer, logical and raw. as.character(1e-2) 0.05 [1] TRUE as.character(1e-4) 0.05 [1] FALSE Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Wed, Jan 2, 2013 at 12:38 PM, Tobias Verbeke tobias.verb...@openanalytics.eu wrote: L.S. Is the following expected and/or documented? 1e-2 0.05 [1] TRUE 1e-4 0.05 [1] FALSE Many thanks in advance for any pointer. Best, Tobias sessionInfo() R Under development (unstable) (2013-01-01 r61512) Platform: i386-w64-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Quiz: How to get a named column from a data frame
On Tue, Aug 21, 2012 at 2:34 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 12-08-18 12:33 PM, Martin Maechler wrote: Joshua Ulrich josh.m.ulr...@gmail.com on Sat, 18 Aug 2012 10:16:09 -0500 writes: I don't know if this is better, but it's the most obvious/shortest I could come up with. Transpose the data.frame column to a 'row' vector and drop the dimensions. R identical(nv, drop(t(df))) [1] TRUE Yes, that's definitely shorter, congratulations! One gotta is that I'd want a solution that also works when the df has more columns than just one... Your idea to use t(.) is nice and perfect insofar as it coerces the data frame to a matrix, and that's really the clue: Where as df[,1] is losing the names, the matrix indexing is not. So your solution can be changed into t(df)[1,] which is even shorter... and slightly less efficient, at least conceptually, than mine, which has been as.matrix(df)[,1] Now, the remaining question is: Shouldn't there be something more natural to achieve that? (There is not, currently, AFAIK). I've been offline, so I'm a bit late to this game, but the examples above fail when df contains a character column as well as the desired one, because everything gets coerced to a character matrix. You need to select the column first, then convert to a matrix, e.g. drop(t(df[,1,drop=FALSE])) That's true, but I was assuming a one-column data.frame, which can be achieved via: df - data.frame(VAR=nv,CHAR=letters[1:3],stringsAsFactors=FALSE) drop(t(df[1])) That said, I prefer the setNames() solution for its efficiency. Best, Josh Duncan Murdoch Martin Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com On Sat, Aug 18, 2012 at 10:03 AM, Martin Maechler maech...@stat.math.ethz.ch wrote: Today, I was looking for an elegant (and efficient) way to get a named (atomic) vector by selecting one column of a data frame. Of course, the vector names must be the rownames of the data frame. Ok, here is the quiz, I know one quite cute/slick answer, but was wondering if there are obvious better ones, and also if this should not become more idiomatic (hence R-devel): Consider this toy example, where the dataframe already has only one column : nv - c(a=1, d=17, e=101); nv a d e 1 17 101 df - as.data.frame(cbind(VAR = nv)); df VAR a 1 d 17 e 101 Now how, can I get 'nv' back from 'df' ? I.e., how to get identical(nv, ...) [1] TRUE where .. only uses 'df' (and no non-standard R packages)? As said, I know a simple solution (*), but I'm sure it is not obvious to most R users and probably not even to the majority of R-devel readers... OTOH, people like Bill Dunlap will not take long to provide it or a better one. (*) In my solution, the above '...' consists of 17 letters. I'll post it later today (CEST time) ... or confirm that someone else has done so. Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Suggestion for Writing R Extensions
Hi, Early in Chapter 1, the manual states: A computing environment including a number of tools is assumed; the R Installation and Administration manual describes what is needed. Under a Unix-alike most of the tools are likely to be present by default, but Microsoft Windows may require careful setup. Would it make sense to add links and/or mention the relevant appendices (A and D) of R Installation and Administration? Best, -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Suggestion for Writing R Extensions
On Wed, Aug 22, 2012 at 2:58 PM, Oliver Bandel oli...@first.in-berlin.de wrote: Hello, Zitat von Joshua Ulrich josh.m.ulr...@gmail.com (Wed, 22 Aug 2012 12:35:51 -0500) [...] Would it make sense to add links and/or mention the relevant appendices (A and D) of R Installation and Administration? [...] I do not understad, what your question is all about. Maybe you should be more verbose. Do you want to make the document better, or do you have questions on how to understand what it talks about? The sentences I quoted, which you removed from the thread, point the reader to R Installation and Administration. Rather than point the reader to the entire manual, I suggest pointing them to the relevant appendices (A and D) of R Installation and Administration. Ciao, Oliver __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
