Regarding symlinking the upstream directory: instead or in addition, what about an option to `./configure` for the location of that directory?
On Monday, February 19, 2024 at 12:09:54 PM UTC-8 Matthias Koeppe wrote: > Prompted by the discussion of space use on the *local machines* of users > and developers, I propose another item in addition to A and B: > > *C. Advertise use of "git worktree" and recommend symlinking the > "upstream" directory.* For testing a new release when you have an > existing clone of the repository, using "git clone" another time is > overkill as it creates another copy of the .git directory. And there is no > point in having multiple copies of the "upstream" directory, as the > filenames of the tarballs change whenever the contents change. > > On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote: > > This (A and B below) has the advantage of being quite explicit. The > original proposal > > 1) allow standard packages to be pip packages > > 2) drop the contents of upstream/ from the Sage source tarballs. > > sounds explicit, but the more the discussion goes on, the more I feel that > there are hidden pieces. Does the proposal also mean removing version > restrictions and all of the other claimed maintenance burden for various > components of Sage? > > Regarding item (2): if I clone the github repository, there is no upstream > directory at the start, but after building Sage, it ends up being almost as > large as in the current tarballs. (This is on OS X with a lot of homebrew > packages installed.) So how much savings are we actually talking about? > (Maybe it's not savings for the end user that are important, so *what* are > we saving? Disk space on the mirrors?) Dima, can you please provide data? > If we convert (according to (1)) to pip packages, those still need to be > downloaded, and while they may not end up in "upstream" — I don't actually > know how they work — don't they still take up disk space? So again, how > much savings are we talking about? Please provide data. > > By the way, the git clone yields a package that is 616M on my machine. A > good chunk of that is the .git directory. Are you proposing that we do not > distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.) > > Regarding item (1): can you provide a list of packages that would become > pip packages? Or describe how you would come up with a list? > > > On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote: > > I'll now offer: > > *Opinion 1. Nobody needs to care in the slightest what the size of that > release tarball is. * > > In any use cases with internet connectivity, people will be better off by > just cloning the git repo, not use the release tarball. > > If there are relevant use cases without internet connectivity (I have no > opinion to offer on this), then the release tarball has exactly the right > contents. > > *Proposed action items: * > *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so > that "git clone" is described as the primary way to obtain the Sage > sources. That the big release tarball is available can be a footnote in the > Installation Guide ( > https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps) > > for the limited no-internet connectivity use case. > > *B. *Likewise, get rid of all of these "Download Sage source code" pages ( > https://www.sagemath.org/download-source.html, > https://www.sagemath.org/download-latest.html), mirror selection, etc. > from the Sage website. > > > On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote: > > Currently the standard packages cannot be pip packages, i.e. we must, in > effect, vendor them. This entails an extra effort which is often not > needed, in particular as we patch only very few Python packages. > Pip packages are on the other hand installed straight from PyPI. > > Good examples of standard packages which can become pip ones are tox, > pytest (not yet standard). > > > The other difference is that by default these packages are not included in > the Sage releases source tarball. > > Rather than adding them there I propose to split the upstream/* part of > the tarball into something optional - which is represented by a list of > files to download, and which is just not needed if you build while > connected to the internet. > > This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 > tarball is 1.3Gb, and without it is smaller than 0.25Gb. > > Note that as William writes, the desire to have Sage buildable without an > internet connection was a requirement by a past Sage funder, gone about 10 > years ago. Thus there's no longer an obligation to have this option. > I am not aware of a similar to Sage which provides tarballs allowing for > an offline build. > > Thus, I would like to call a vote on these two topics: > > 1) allow standard packages to be pip packages > > 2) drop the contents of upstream/ from the Sage source tarballs. > > > --- > Dima > > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/9bf9962a-a410-4302-a085-6c3fe81afad9n%40googlegroups.com.
