On Monday, February 19, 2024 at 12:10:49 PM UTC-8 Dima Pasechnik wrote:
On Mon, Feb 19, 2024 at 7:42 PM John H Palmieri wrote: This (A and B below) has the advantage of being quite explicit. The original proposal 1) allow standard packages to be pip packages 2) drop the contents of upstream/ from the Sage source tarballs. sounds explicit, but the more the discussion goes on, the more I feel that there are hidden pieces. Does the proposal also mean removing version restrictions and all of the other claimed maintenance burden for various components of Sage? No, this is simply FUD spread by certain parties here. You said: "The difference between wheel packages vs pip packages is that the latter don't require pre-fetched wheels, and absence of the need for package (micro)management." The implication is that changing the package management system is, maybe not part of this proposal, but a next step. In other words, I'm getting this impression from your words, not by other "certain parties." You said: "My proposal is in fact aimed at reducing the number of pinned Sage dependecies, drastically." (You have made similar comments elsewhere in this thread.) How does (1) accomplish this? Either I'm missing something or you have not spelled everything out in your proposal. You said '"Allow" does not mean "Make all of the", it should be obvious.' "Allow" does not cause any changes to happen drastically. So what exactly are you proposing to accomplish these drastic changes? If you have a roadmap in mind, it would be helpful if you described it. pytest is good example of package which can be elevated to standard, but kept pip. In no place my proposal 1) demands anything done for all packages. Regarding item (2): if I clone the github repository, there is no upstream directory at the start, but after building Sage, it ends up being almost as large as in the current tarballs. (This is on OS X with a lot of homebrew packages installed.) So how much savings are we actually talking about? (Maybe it's not savings for the end user that are important, so *what* are we saving? Disk space on the mirrors?) Dima, can you please provide data? If we convert (according to (1)) to pip packages, those still need to be downloaded, and while they may not end up in "upstream" — I don't actually know how they work — don't they still take up disk space? So again, how much savings are we talking about? Please provide data. I am talking about saving space on mirrors, and on bandwidth, by not packaging "upstream/". (as I wrote: This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 tarball is 1.3Gb, and without it is smaller than 0.25Gb.) Local disk space nowadays is cheap, but space on mirrors, and bandwidth, don't come free. (not everyone is on an unlimited internet) As well, if you don't wipe up your local upstream/, its contents can be reused. (typically not so many packages are updated with each release after all) However, as far as I can tell, by default pip package wheels are not stored in upstream/. Perhaps there is an easy way to change this, I don't know. By the way, the git clone yields a package that is 616M on my machine. A good chunk of that is the .git directory. Are you proposing that we do not distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.) Regarding item (1): can you provide a list of packages that would become pip packages? Or describe how you would come up with a list? packages not involved in sagelib directly are good candidates, e.g. pytest, tox. sphinx and jupyterlab are good candidates too, in my limited testing experiments. Dima On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote: I'll now offer: *Opinion 1. Nobody needs to care in the slightest what the size of that release tarball is. * In any use cases with internet connectivity, people will be better off by just cloning the git repo, not use the release tarball. If there are relevant use cases without internet connectivity (I have no opinion to offer on this), then the release tarball has exactly the right contents. *Proposed action items: * *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so that "git clone" is described as the primary way to obtain the Sage sources. That the big release tarball is available can be a footnote in the Installation Guide ( https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps) for the limited no-internet connectivity use case. *B. *Likewise, get rid of all of these "Download Sage source code" pages ( https://www.sagemath.org/download-source.html, https://www.sagemath.org/download-latest.html), mirror selection, etc. from the Sage website. On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote: Currently the standard packages cannot be pip packages, i.e. we must, in effect, vendor them. This entails an extra effort which is often not needed, in particular as we patch only very few Python packages. Pip packages are on the other hand installed straight from PyPI. Good examples of standard packages which can become pip ones are tox, pytest (not yet standard). The other difference is that by default these packages are not included in the Sage releases source tarball. Rather than adding them there I propose to split the upstream/* part of the tarball into something optional - which is represented by a list of files to download, and which is just not needed if you build while connected to the internet. This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 tarball is 1.3Gb, and without it is smaller than 0.25Gb. Note that as William writes, the desire to have Sage buildable without an internet connection was a requirement by a past Sage funder, gone about 10 years ago. Thus there's no longer an obligation to have this option. I am not aware of a similar to Sage which provides tarballs allowing for an offline build. Thus, I would like to call a vote on these two topics: 1) allow standard packages to be pip packages 2) drop the contents of upstream/ from the Sage source tarballs. --- Dima -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/0a3c1c0b-ed3f-41eb-ab4a-5235432475dbn%40googlegroups.com <https://groups.google.com/d/msgid/sage-devel/0a3c1c0b-ed3f-41eb-ab4a-5235432475dbn%40googlegroups.com?utm_medium=email&utm_source=footer> . -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/1552e761-062e-466c-9efe-4d4acd71a18dn%40googlegroups.com.
