Re: [sage-devel] Re: New labels v: mimimal, v: small ... on pull requests
On Tuesday, May 28, 2024 at 12:19:42 PM UTC-7 Matthias Koeppe wrote: On Tuesday, May 28, 2024 at 1:24:17 AM UTC-7 Travis Scrimshaw wrote: Another data point: the bot is getting the size wrong too: https://github.com/sagemath/sage/pull/38105 Thanks, I've let the author know in https://github.com/sagemath/sage/pull/37262#issuecomment-2135699139 His PR https://github.com/sagemath/sage/pull/38114 fixes the size computation for PRs that are based on an out-of-date base branch. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c7192b5e-0bde-44df-8f83-2d531b775bcen%40googlegroups.com.
Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages
On Fri, May 31, 2024 at 01:04:38PM -0400, David Roe wrote: > On Fri, May 31, 2024 at 12:38 PM Dima Pasechnik wrote: > > > On Thu, May 30, 2024 at 11:25 PM Matthias Koeppe > > wrote: > > > > > > We added the packages as optional "pip" packages (see > > https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#package-types > > for the terminology), each more than 1 year ago. > > > > > > - > > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest#spkg-pytest > > (added in 2020) > > > - > > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_mock#spkg-pytest-mock > > (added in 2022) > > > - > > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_xdist#spkg-pytest-xdist > > (added in 2022) > > > > > > "pytest" is the current gold standard for running tests of Python > > packages. Various standard packages in the Sage distribution already > > declare pytest in "dependencies_check" as a conditional dependency for use > > when SAGE_CHECK=yes is set. The support for testing parts of the Sage > > library with pytest was from an effort largely stalled after 2022 -- and > > which has been completely broken since early 2024 with the arrival of > > pytest 8.x, which I have just fixed in > > https://github.com/sagemath/sage/pull/37999 (to be merged). I'll note > > that recent versions of pytest have added support for PEP-420 implicit > > namespace package, which the Sage library uses as part of the > > modularization effort. > > > > > > By making pytest a standard package, I would hope to help revive this > > effort, and thus the larger effort to "adopt mainstream Python > > testing/linting infrastructure" (see > > https://github.com/sagemath/sage/issues/28936). > > > > > > I'm proposing to make the packages (and their dependencies) standard > > (wheel) packages according to the procedures in our developer guide. > > > - Reference to previous such proposals following our project's > > procedures: > > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/KyOOQzkzAAAJ > > > - Link to our packaging documentation and explanation how making it a > > standard package compares to version pinning done for example using > > conda-lock: > > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/N2vuKb-TAAAJ > > > > > > The other pytest_* packages are related technical packages. > > > The PR that implements it: https://github.com/sagemath/sage/pull/37301 > > > > > > This is a redo of the 2nd part of my proposal > > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/NsyUa7iXAgAJ from > > Feb 10, which was stalled. (The 1st part (on python_build) was redone in > > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/9_zh6usoAAAJ and > > approved.) > > > > My objections (and not only my) to this still stand - and, frankly, I > > don't see anything new here. What's the point of bringing the stalled > > matters up again in this original unchanged way ? > > > > We don't need to have all the dependencies of packages like pytest in > > Sage, it's just pure bloat, give their peripheral role in particular. > > That is, it's fine to declare them standard, and keep them pip > > packages - what do we lose this way? Nothing, and we don't bloat Sage > > with even more packages nobody knows anything about - besides them > > being dependencies of something in Sage. > > > > But there is more trouble ahead if the currect proposal gets in over > > my objections. Say, the next version of pytest might get a part which > > needs Rust, and pytest is a wheel package, with all its buildable from > > source dependencies in Sage, and Sage is fully committed to using > > pytest for testing. > > Are you going to include a Rust toolchain in Sage ? No, obviously not. > > Are you going to demote pytest back to optional, > > and throw away work done on using pytest more? No. Have another fight > > over the ways Sage should be packaged? Yes, sure. > > > > I don't find it plausible that a package whose purpose is to test Python > code would add Rust as a dependency. We already are using a Rust-based tool to improve Sage code, namely, ruff (https://pypi.org/project/ruff/) https://github.com/sagemath/sage/pulls?q=is%3Apr+is%3Aopen+ruff > Sure, dependencies can change as > projects evolve and add features, but I don't think it's reasonable to base > decisions on what to add to the sage distribution on potential future > changes. Well, Rust or no Rust: if you act myopically, you're doomed to do extra needless work/pay extra money. There are other reasons not to add extra wheel packages if it can be avoided: distribution bandwidth might stop being free, disk space might stop being free, other "too big to be sustainable" issues are popping up - we're having so many packages that they already scream for a proper package manager, and not a slow kludge we have instead. Dima > David > > I think the only feasible way forward here is as I proposed (standard > > pip packages),
Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages
On Friday, May 31, 2024 at 9:38:30 AM UTC-7 Dima Pasechnik wrote: frankly, I don't see anything new here. There does not have to be anything new. The proposal stands on its own merit. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/6014f157-a90e-4e6d-af94-05aa62b8e6fcn%40googlegroups.com.
Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages
On Fri, May 31, 2024 at 12:38 PM Dima Pasechnik wrote: > On Thu, May 30, 2024 at 11:25 PM Matthias Koeppe > wrote: > > > > We added the packages as optional "pip" packages (see > https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#package-types > for the terminology), each more than 1 year ago. > > > > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest#spkg-pytest > (added in 2020) > > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_mock#spkg-pytest-mock > (added in 2022) > > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_xdist#spkg-pytest-xdist > (added in 2022) > > > > "pytest" is the current gold standard for running tests of Python > packages. Various standard packages in the Sage distribution already > declare pytest in "dependencies_check" as a conditional dependency for use > when SAGE_CHECK=yes is set. The support for testing parts of the Sage > library with pytest was from an effort largely stalled after 2022 -- and > which has been completely broken since early 2024 with the arrival of > pytest 8.x, which I have just fixed in > https://github.com/sagemath/sage/pull/37999 (to be merged). I'll note > that recent versions of pytest have added support for PEP-420 implicit > namespace package, which the Sage library uses as part of the > modularization effort. > > > > By making pytest a standard package, I would hope to help revive this > effort, and thus the larger effort to "adopt mainstream Python > testing/linting infrastructure" (see > https://github.com/sagemath/sage/issues/28936). > > > > I'm proposing to make the packages (and their dependencies) standard > (wheel) packages according to the procedures in our developer guide. > > - Reference to previous such proposals following our project's > procedures: > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/KyOOQzkzAAAJ > > - Link to our packaging documentation and explanation how making it a > standard package compares to version pinning done for example using > conda-lock: > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/N2vuKb-TAAAJ > > > > The other pytest_* packages are related technical packages. > > The PR that implements it: https://github.com/sagemath/sage/pull/37301 > > > > This is a redo of the 2nd part of my proposal > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/NsyUa7iXAgAJ from > Feb 10, which was stalled. (The 1st part (on python_build) was redone in > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/9_zh6usoAAAJ and > approved.) > > My objections (and not only my) to this still stand - and, frankly, I > don't see anything new here. What's the point of bringing the stalled > matters up again in this original unchanged way ? > > We don't need to have all the dependencies of packages like pytest in > Sage, it's just pure bloat, give their peripheral role in particular. > That is, it's fine to declare them standard, and keep them pip > packages - what do we lose this way? Nothing, and we don't bloat Sage > with even more packages nobody knows anything about - besides them > being dependencies of something in Sage. > > But there is more trouble ahead if the currect proposal gets in over > my objections. Say, the next version of pytest might get a part which > needs Rust, and pytest is a wheel package, with all its buildable from > source dependencies in Sage, and Sage is fully committed to using > pytest for testing. > Are you going to include a Rust toolchain in Sage ? No, obviously not. > Are you going to demote pytest back to optional, > and throw away work done on using pytest more? No. Have another fight > over the ways Sage should be packaged? Yes, sure. > I don't find it plausible that a package whose purpose is to test Python code would add Rust as a dependency. Sure, dependencies can change as projects evolve and add features, but I don't think it's reasonable to base decisions on what to add to the sage distribution on potential future changes. David I think the only feasible way forward here is as I proposed (standard > pip packages), and I propose it again. > > Dima > > > > > > > I ask everyone to focus on the specifics of this proposal. > > > > -- > > You received this message because you are subscribed to the Google > Groups "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sage-devel+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/2c42bd41-24a3-467e-857f-aedc3966c107n%40googlegroups.com > . > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit >
[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size
Before looking at https://groups.google.com/g/sage-devel/c/lPLoA7zaoyg/m/dGE1B1jQEQAJ we should look at this proposal again, as pytest is a very suitable candidate for the kinds of packages (standard pip packages) proposed here. Indeed, nothing (save for a very marginal case of a complete offline install - and this can be helped if there is a will to allow such packages) is gained by mechanically adding the pytest dependencies into Sage the distro. And doing this an extra code bloat, with stuff we don't patch, and don't even know what's there. E.g. we can add a backdoored,or otherwise broken, version of one of these into the distro, giving it extra legitimacy for no reason. Dima On Sunday, February 11, 2024 at 7:23:42 PM UTC Dima Pasechnik wrote: > Currently the standard packages cannot be pip packages, i.e. we must, in > effect, vendor them. This entails an extra effort which is often not > needed, in particular as we patch only very few Python packages. > Pip packages are on the other hand installed straight from PyPI. > > Good examples of standard packages which can become pip ones are tox, > pytest (not yet standard). > > > The other difference is that by default these packages are not included in > the Sage releases source tarball. > > Rather than adding them there I propose to split the upstream/* part of > the tarball into something optional - which is represented by a list of > files to download, and which is just not needed if you build while > connected to the internet. > > This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 > tarball is 1.3Gb, and without it is smaller than 0.25Gb. > > Note that as William writes, the desire to have Sage buildable without an > internet connection was a requirement by a past Sage funder, gone about 10 > years ago. Thus there's no longer an obligation to have this option. > I am not aware of a similar to Sage which provides tarballs allowing for > an offline build. > > Thus, I would like to call a vote on these two topics: > > 1) allow standard packages to be pip packages > > 2) drop the contents of upstream/ from the Sage source tarballs. > > > --- > Dima > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/77996190-28c2-4a88-a49c-3b4f28edff01n%40googlegroups.com.
Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages
On Thu, May 30, 2024 at 11:25 PM Matthias Koeppe wrote: > > We added the packages as optional "pip" packages (see > https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#package-types > for the terminology), each more than 1 year ago. > > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest#spkg-pytest > (added in 2020) > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_mock#spkg-pytest-mock > (added in 2022) > - > https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_xdist#spkg-pytest-xdist > (added in 2022) > > "pytest" is the current gold standard for running tests of Python packages. > Various standard packages in the Sage distribution already declare pytest in > "dependencies_check" as a conditional dependency for use when SAGE_CHECK=yes > is set. The support for testing parts of the Sage library with pytest was > from an effort largely stalled after 2022 -- and which has been completely > broken since early 2024 with the arrival of pytest 8.x, which I have just > fixed in https://github.com/sagemath/sage/pull/37999 (to be merged). I'll > note that recent versions of pytest have added support for PEP-420 implicit > namespace package, which the Sage library uses as part of the modularization > effort. > > By making pytest a standard package, I would hope to help revive this effort, > and thus the larger effort to "adopt mainstream Python testing/linting > infrastructure" (see https://github.com/sagemath/sage/issues/28936). > > I'm proposing to make the packages (and their dependencies) standard (wheel) > packages according to the procedures in our developer guide. > - Reference to previous such proposals following our project's procedures: > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/KyOOQzkzAAAJ > - Link to our packaging documentation and explanation how making it a > standard package compares to version pinning done for example using > conda-lock: > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/N2vuKb-TAAAJ > > The other pytest_* packages are related technical packages. > The PR that implements it: https://github.com/sagemath/sage/pull/37301 > > This is a redo of the 2nd part of my proposal > https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/NsyUa7iXAgAJ from Feb > 10, which was stalled. (The 1st part (on python_build) was redone in > https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/9_zh6usoAAAJ and > approved.) My objections (and not only my) to this still stand - and, frankly, I don't see anything new here. What's the point of bringing the stalled matters up again in this original unchanged way ? We don't need to have all the dependencies of packages like pytest in Sage, it's just pure bloat, give their peripheral role in particular. That is, it's fine to declare them standard, and keep them pip packages - what do we lose this way? Nothing, and we don't bloat Sage with even more packages nobody knows anything about - besides them being dependencies of something in Sage. But there is more trouble ahead if the currect proposal gets in over my objections. Say, the next version of pytest might get a part which needs Rust, and pytest is a wheel package, with all its buildable from source dependencies in Sage, and Sage is fully committed to using pytest for testing. Are you going to include a Rust toolchain in Sage ? No, obviously not. Are you going to demote pytest back to optional, and throw away work done on using pytest more? No. Have another fight over the ways Sage should be packaged? Yes, sure. I think the only feasible way forward here is as I proposed (standard pip packages), and I propose it again. Dima > > > I ask everyone to focus on the specifics of this proposal. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/2c42bd41-24a3-467e-857f-aedc3966c107n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq2fubG32GKjKroEVJ_LKB2WUY%2BqJSXCZTgd0ROC%3DYHczA%40mail.gmail.com.
