We agreed a deprecation cycle should be implemented. We are pushing up changes to BiocFileCache, AnnotationHub, and ExperimentHub so instead of an ERROR and failing out, it will give a warning with deprecation notice and use the old default cache.
After the next release however, we will reimplement as an error in devel (3.14?) with the intention of removing the rappdirs dependency completely in (3.15?) Lori Shepherd Bioconductor Core Team Roswell Park Comprehensive Cancer Center Department of Biostatistics & Bioinformatics Elm & Carlton Streets Buffalo, New York 14263 ________________________________ From: Henrik Bengtsson <henrik.bengts...@gmail.com> Sent: Wednesday, April 7, 2021 3:13 PM To: Aaron Lun <infinite.monkeys.with.keyboa...@gmail.com> Cc: Kern, Lori <lori.sheph...@roswellpark.org>; bioc-devel@r-project.org <bioc-devel@r-project.org> Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and ExperimentHub FWIW, I ran into a similar problem when I moved R.cache (https://secure-web.cisco.com/1wXExx3fIivrKLOo_pJ7VCpkJk5105Y5JXKMssE6mF31qNZZyrUDzt8mwpB9urmx7TyHTfsSInEGOEkOxvCIeuCCf2GWdupZehBtX-RAdUFq7jEMNwF2or4UjEo-g4V7u2ffM6bdK_ccByCWsWSJbAWQYz-bSMCnJYo74NejZOGeAdckB0mNurxDQsuIaa_tnCknMZhUzN4XVlcPPQyrwtBMUowL4oVv7OPQ-K3xd0FcszC8fpHaI0a89lVkqn4xrjkbBuWmpbvbr_kOrRBt9W8DzTmMeaZNeEu9LbZ3vC5ZcOWV-xJ2XJgjrEH8HHm92/https%3A%2F%2Fcran.r-project.org%2Fpackage%3DR.cache) from using ~/.Rcache to ~/.cache/R/R.cache (etc). I decided on making it 100%-backward compatible, i.e. if there's already a legacy ~/.Rcache cache folder, it'll keep using that, otherwise the new standard. That way nothing breaks, and it's not a biggie if it keeps writing to the legacy cache folder. For now, it's silent, but I'll eventually deprecate ~/.Rcache, e.g. by producing one-time warning per session and in a later release be more aggressive, and eventually make it defunct (but not rushing there). Here's what I wrote in my NEWS release: Version: 0.14.0 [2019-12-05] SIGNIFICANT CHANGES: * Now R.cache uses a default cache path that adheres to the standard cache location on the current operating system, whereas in previous versions it defaulted to ~/.Rcache. On Unix, the 'XDG Base Directory Specification' is followed, which means that the R.cache folder will typically be ~/.cache/R/R.cache/. On macOS, it will be ~/Library/Caches/R/R.cache/. On modern versions of Microsoft Windows, environment variables such as 'LOCALAPPDATA' will be used, which typically resolves to '%USERPROFILE%/AppData/Local, e.g. 'C:/Users/alice/AppData/Local'. If R.cache fails find a proper OS-specific cache folder, it will fall back to using ~/.Rcache as previously done. Importantly, if ~/.Rcache already exists, then that will be used by default. This is done in order to not lose previously cached files. Users with an existing folder who wish to move to the new standard need to move or remove the ~/.Rcache folder manually. /Henrik On Wed, Apr 7, 2021 at 9:41 AM Aaron Lun <infinite.monkeys.with.keyboa...@gmail.com> wrote: > > > There is no guarantee we would be under the right user to have permissions > > to move the cache automatically and would not want to leave it in a broken > > state. > > Well, can't you try? If people follow your 4.1 instructions and they > don't have permissions, the cache will be broken anyway. > > But let's say you can't move it, and your worst-case scenario comes to > pass. EVEN THEN: I would expect a deprecation warning, no error, and > BiocFileCache continuing to pull from the old cache for 6 months. > > Every previous non-transparent change to BioC's core infrastructure > has come with a deprecation warning. I don't see why this is any > different. An error is particularly galling given that the package was > working fine before, it's not like you're doing some kind of critical > bugfix. > > > This should not affect any cache that is explicitly stated with a different > > name in the constructor or using environment variables; only in the case > > of BiocFileCache() . Most package specific caches created their own cache > > in the constructor so it should not cause the ERROR in that case. > > If Vince's last email is any indication, and calling ExperimentHub() > or AnnotationHub() causes an error... this will be a disaster. I'm > going to get a lot of emails, unnecessary emails, from users wondering > why scRNAseq and celldex don't work anymore. It'll be like our > AWS-China problems multiplied by 10. > > Why not just make a new cache and populate it? Well, I don't really > care what you do, as long as I don't get an error. > > -A > > > ________________________________ > > From: Aaron Lun <infinite.monkeys.with.keyboa...@gmail.com> > > Sent: Wednesday, April 7, 2021 11:41 AM > > To: Kern, Lori <lori.sheph...@roswellpark.org> > > Cc: bioc-devel@r-project.org <bioc-devel@r-project.org> > > Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and > > ExperimentHub > > > > Woah, I missed the part where you said that there would be an error. > > > > This does not sound good. Users are going to flip out, especially when > > EHub and AHub are not visible dependencies (e.g., scRNAseq, celldex). > > It also sounds completely unnecessary for EHub and AHub given that the > > new cache can just be populated by fresh downloads. Similarly, > > BiocFileCache::bfcrpath should not be affected, and people using that > > shouldn't be getting an error. > > > > Why not just move the old default cache into the new location > > automatically? This seems like the simplest solution given that > > everyone accessing BFC resources should be doing so through the BFC > > API. And most files are not position-dependent, unless people are > > putting shared libraries in there. > > > > But even if you can't, an error is just too much. We use BiocFileCache > > a lot in our company infrastructure and the brown stuff will hit the > > fan if we have to find every old default cache and delete it. The > > package should handle this for us. > > > > -A > > > > On Wed, Apr 7, 2021 at 4:46 AM Kern, Lori <lori.sheph...@roswellpark.org> > > wrote: > > > > > > Mostly to lighten the dependency tree using tools that is built in with R > > > would remove one additional dependency. Also clarity; the tools > > > directory adds an R folder for distinction that they are used with R > > > packages which seemed like if a user was ever investigating, they would > > > have a better idea where those files came from. > > > > > > > > > Lori Shepherd > > > > > > Bioconductor Core Team > > > > > > Roswell Park Comprehensive Cancer Center > > > > > > Department of Biostatistics & Bioinformatics > > > > > > Elm & Carlton Streets > > > > > > Buffalo, New York 14263 > > > > > > ________________________________ > > > From: Bioc-devel <bioc-devel-boun...@r-project.org> on behalf of Aaron > > > Lun <infinite.monkeys.with.keyboa...@gmail.com> > > > Sent: Wednesday, April 7, 2021 4:10 AM > > > To: bioc-devel@r-project.org <bioc-devel@r-project.org> > > > Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and > > > ExperimentHub > > > > > > rebook and basilisk are also currently using rappdirs. I would be > > > interested in the motivation behind the switch for the Hubs and whether > > > that is applicable to those two packages as well. > > > > > > -A > > > > > > On 4/5/21 6:41 AM, Kern, Lori wrote: > > > > We are in process of making some major updates to the caching in > > > > BiocFileCache, AnnotationHub, and ExperimentHub. Namely, the default > > > > caching location will change from using rappdirs::user_cache_dir to > > > > using tools::R_user_dir eventually relieving the dependency on > > > > rappdirs. To avoid conflicting default caches, if anyone used an old > > > > default caching directory, there will be an error to decide how to deal > > > > with the old location before proceeding and documentation in the > > > > vignettes for how to resolve. Currently I have update BiocFileCache, > > > > the changes were just pushed to the devel branch and should propagate > > > > tonight. I plan on doing the same for both AnnotationHub and > > > > ExperimentHub within the next few days. We appreciate any feedback or > > > > questions with regards to these updates. > > > > > > > > This is only relevant to using the default cache location, if a user > > > > manually specified a unique location, used environment variables, or > > > > created a package specific cache the code/location is not affected. > > > > Anyone using package specific caching that utilizes rappdirs is > > > > encouraged also to consider changing package code to use the now > > > > available function in tools. > > > > > > > > Cheers, > > > > > > > > > > > > Lori Shepherd > > > > > > > > Bioconductor Core Team > > > > > > > > Roswell Park Comprehensive Cancer Center > > > > > > > > Department of Biostatistics & Bioinformatics > > > > > > > > Elm & Carlton Streets > > > > > > > > Buffalo, New York 14263 > > > > > > > > > > > > This email message may contain legally privileged and/or confidential > > > > information. If you are not the intended recipient(s), or the employee > > > > or agent responsible for the delivery of this message to the intended > > > > recipient(s), you are hereby notified that any disclosure, copying, > > > > distribution, or use of this email message is prohibited. If you have > > > > received this message in error, please notify the sender immediately by > > > > e-mail and delete this email message from your computer. Thank you. > > > > [[alternative HTML version deleted]] > > > > > > > > _______________________________________________ > > > > Bioc-devel@r-project.org mailing list > > > > https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel > > > > > > > > > > _______________________________________________ > > > Bioc-devel@r-project.org mailing list > > > https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel > > > > > > > > > This email message may contain legally privileged and/or confidential > > > information. If you are not the intended recipient(s), or the employee or > > > agent responsible for the delivery of this message to the intended > > > recipient(s), you are hereby notified that any disclosure, copying, > > > distribution, or use of this email message is prohibited. If you have > > > received this message in error, please notify the sender immediately by > > > e-mail and delete this email message from your computer. Thank you. > > > > This email message may contain legally privileged and/or confidential > > information. If you are not the intended recipient(s), or the employee or > > agent responsible for the delivery of this message to the intended > > recipient(s), you are hereby notified that any disclosure, copying, > > distribution, or use of this email message is prohibited. If you have > > received this message in error, please notify the sender immediately by > > e-mail and delete this email message from your computer. Thank you. > > _______________________________________________ > Bioc-devel@r-project.org mailing list > https://secure-web.cisco.com/1uPA4va40Fh5q5TEZpCYA5kV_043lp7TOyA545uz5r4ZJ0Z7sFUZr1yb2rW9MMuhx_u_nCUTJv1UMZj7hJ4bkdGFOZt0U7o1t09_m8VCz9M1da18ncOt_LdUTjdM8BBTfH3m22rYsMv6wBUFLy7NHpf4Ait7H7bSnIabvqXVLjc2nXre-sSkCb6672jKNooCKB7pWuRK3-4uKZwS9opsD3T-jpWvgheDkcTfQCcXKGcWrTWtWZ3Y2nZekJ9Otb66hZpjolKIOFpESh5PG3zInsGnUBaP4DZREforX_4ClLCelZyMuvI1F8_Qur9xuwDhp/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
