Hi Alan, The specs of the Linux servers have not changed.
However we've recently observed some of these random kills by the Linux kernel for a couple of other packages, and we started an application to get funding to increase the amount of RAM on these machines. These random kills are a desperate attempt by the Linux kernel to keep the machine alive when it's under extremely high load and running out of resources. Unfortunately this is not something that anybody will be able to easily reproduce as it only happens under special circumstances e.g. during the daily builds when dozens of packages are being checked simultaneously. Regardless of whether we'll manage to increase the memory on the Linux servers, it seems to me that the most memory-hungry unit tests in MungeSumstats would be a better fit for the long tests. These tests are run once a week instead of daily, and use less workers (4 instead of 28). This means that each package has access to more resources. See https://contributions.bioconductor.org/long-tests.html for how to set up the long tests in your package. Let me know if you have questions or need help with this. Best, H. On 2/2/24 09:14, alan murphy wrote: > Hi, > > I'm the maintainer of the MungeSumstats package which is currently failing on > the devel nebbiolo1 Linux platform because of RAM requirements of the unit > tests. These tests use large reference datasets (like > BSgenome.Hsapiens.1000genomes.hs37d5::BSgenome.Hsapiens.1000genomes.hs37d5) > which cause the issue, e.g. from the output of the devel linux > test<https://bioconductor.org/checkResults/3.19/bioc-LATEST/MungeSumstats/nebbiolo1-checksrc.html>: > > ``` > > Validating RSIDs of 92 SNPs using BSgenome::snpsById... > Killed > > ``` > which is from this > function:https://github.com/neurogenomics/MungeSumstats/blob/cccf77b2249f52be59fe1749f13a386ffaaae528/R/load_ref_genome_data.R#L49 > > I would like to keep these units tests as they are very important for me to > know if things have gone wrong. I can't scale down the RAM usage, it is only > using a few rows of data as it, the issue is that the reference sets are > massive rather than the actual set being tested. I currently don't have a lot > of these tests set to run on windows/mac platforms so I rely on the Linux > machines to run this. I am not sure what has changed but a few releases back, > the same tested were not causing this issue and would complete on linux - has > the machine spec changed? Is there any chance RAM could be increased for > these tests or is there a way to specify not to run the specific unit tests > on the Bioconductor server so I can at least keep these for my github actions > workflows? See below for the automated message I got about these errors. > > Any other suggestions on ways around this would be greatly appreciated! > > Thanks, > Alan. > ________________________________ > From: CoreTeam Bioconductor<bioconductorcoret...@gmail.com> > Sent: Friday 2 February 2024 16:43 > To:alanmurp...@hotmail.com <alanmurp...@hotmail.com> > Subject: MungeSumstats Bioconductor > > Hello Package Maintainer, > > We would like to bring to your attention that your package is failing in > devel on the linux platform. This is very problematic. Please investigate the > issues and fix the package to avoid deprecation. > > https://bioconductor.org/checkResults/devel/bioc-LATEST/ > <https://bioconductor.org/checkResults/3.17/workflows-LATEST/TCGAWorkflow> > > If you have further questions or concerns please reach out on > thebioc-de...@r-project.org<mailto:bioc-devel@r-project.org> > > We appreciate your quick attention to this matter > > Cheers, > On behalf of the Bioconductor Core Team > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel -- Hervé Pagès Bioconductor Core Team hpages.on.git...@gmail.com [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
