[sage-devel] Re: One year of Sage development on GitHub

[Matthias Koeppe]

On Monday, February 19, 2024 at 7:02:26 PM UTC-8 Kwankyu Lee wrote:


*3. Are the labels on GitHub Issues / PRs helpful?*
- Note that new contributors who are not in the Triage team cannot 
set/change labels! 
- This includes component labels, but also status labels such as "needs 
review".


We may drop "needs review" label, and start to use github "draft" 
functionality. This will remove one major obstacle for new contributors.


I would support this.

Can the status label sync workflow help with this transition, without 
getting in the way? For example, when the _author_ removes the "needs 
review" label (without setting "positive review"), set the PR to "draft"? 


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/4b510b39-2ed6-4c57-8f2f-95d49a6e1abdn%40googlegroups.com.

[sage-devel] Re: One year of Sage development on GitHub

[Kwankyu Lee]

*3. Are the labels on GitHub Issues / PRs helpful?*
- Note that new contributors who are not in the Triage team cannot 
set/change labels! 
- This includes component labels, but also status labels such as "needs 
review".


We may drop "needs review" label, and start to use github "draft" 
functionality. This will remove one major obstacle for new contributors.

 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/f12f8c2a-da74-42c9-868d-48ab2fe3aae4n%40googlegroups.com.

[sage-devel] Please use "Issue #" in your code comments

[Kwankyu Lee]

Hi,

In 

https://github.com/sagemath/sage/pull/37403

we are changing how to render ":issue:" role in the documentation. For 
example, current 

By github issue #7797, there is a different implementation ...

will change to

By Issue #7797, there is a different implementation ...

Hence, likewise, please start to use the short form "Issue #" in your code 
comments  # in accordance with Issue #37405

Thanks for attention!


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/257c5bf2-3294-43f6-b73c-c3958e833e08n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Nathan Dunfield]

On Monday, February 19, 2024 at 3:08:54 PM UTC-6 John H Palmieri wrote, 
responding to Dima:

You said: "The difference between wheel packages vs pip packages is that 
the latter don't require pre-fetched wheels, and absence of the need for 
package (micro)management." The implication is that changing the package 
management system is, maybe not part of this proposal, but a next step. In 
other words, I'm getting this impression from your words, not by other 
"certain parties."

You said: "My proposal is in fact aimed at reducing the number of pinned 
Sage dependecies, drastically." (You have made similar comments elsewhere 
in this thread.) How does (1) accomplish this? Either I'm missing something 
or you have not spelled everything out in your proposal.

You said '"Allow" does not mean "Make all of the", it should be obvious.'

"Allow" does not cause any changes to happen drastically. So what exactly 
are you proposing to accomplish these drastic changes? If you have a 
roadmap in mind, it would be helpful if you described it.


My understanding is that allowing standard packages to be pip packages 
could greatly reduce the number of pinned Sage dependencies for two reasons:

1) a build-from-source or wheel package must explicitly pin its version, 
but, more importantly,

2) a pip package is allowed to install additional dependencies of PyPI that 
are not recorded anywhere in the Sage repo.

A simple example is pytest.  Here it is as an optional pip package:

https://github.com/sagemath/sage/tree/develop/build/pkgs/pytest

To be upgraded to a standard package, under the current policy would need 
to be turned into a "wheel package" requires adding its dependencies like 
so:

https://github.com/sagemath/sage/pull/37301

Here, pytest has just a few dependencies, but jupyterlab has more like 50 
when you include dependencies of dependencies. 



Personally, I think the current system of having everything pinned and 
explicitly recorded is the right choice, being more stable in my experience 
with other projects.  In any event, switching to a pip package for e.g. 
jupterlab doesn't affect the final size or complexity of Sage as installed, 
just how many moving pieces there appear to be if you look in 
"sage/build/pkgs".

Best,

Nathan

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/f81b8284-cb48-44fe-a3f7-158be2438335n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

On Monday, February 19, 2024 at 2:41:11 PM UTC-8 Dima Pasechnik wrote:

On Mon, Feb 19, 2024 at 10:29 PM Matthias Koeppe  
wrote:

An option to "./configure" could work too, except that the "bootstrap" 
phase already downloads the "configure" tarball into that directory.


an option to ./bootstrap then would be logical


... except that "bootstrap" is invoked by "make configure", so it would 
have to be a "make" variable.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/bf4ca5ba-f880-4d30-9140-1cd3a02d8c7dn%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[John H Palmieri]

You're right, plus I was confusing "./bootstrap -d" (which is run by "make 
configure") with "./bootstrap -D" which forces the download.

On Monday, February 19, 2024 at 3:17:11 PM UTC-8 Matthias Koeppe wrote:

> On Monday, February 19, 2024 at 3:09:55 PM UTC-8 John H Palmieri wrote:
>
> By the way, I just cloned the Sage repo and ran "make configure", which 
> runs `./bootstrap`. The upstream directory is empty after that.
>
>
> You probably have autoconf/automake/... installed. In this case, it just 
> uses them to build the configure script. Downloading is a fallback that is 
> used when these "bootstrapping prerequisites" are not installed.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/ed16aa03-8b75-4dd1-bf74-567882d16646n%40googlegroups.com.

Re: [sage-devel] Re: One year of Sage development on GitHub

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 9:20 PM seb@gmail.com 
wrote:

>  > *Is it time for the next step with syncing status labels
> (https://github.com/sagemath/sage/issues/35927
> )? *
>
> The reason this is blocked is because there is a bug in the GitHub web
> interface that might cause confusion when labels are added or removed by
> the bot. More precisely, the panel with labels is not updated immediately
> after such an action. I have opened two bug reports (2448092
> , 2573072
> ). Both were closed
> without a satisfactory answer, only informing that it is a known bug that
> will be fixed one day.
>

As far as I know you have an API to manipulate github labels, e.g. it's
supported by gh.
Is this what's used by the bot?

(by the way, your bug reports are not visible - probably only visible to
you.


>
> Matthias Koeppe schrieb am Freitag, 16. Februar 2024 um 19:00:52 UTC+1:
>
>> On Thursday, February 8, 2024 at 10:16:58 AM UTC-8 Matthias Koeppe wrote:
>>
>> On Wednesday, February 7, 2024 at 1:11:14 PM UTC-8 Matthias Koeppe wrote:
>>
>> Let's also use this anniversary as an opportunity to discuss what still
>> needs improving in our development workflows.
>>
>> *1. We have a low development velocity.* For example, some simple PRs
>> sit for weeks or months before receiving any review comments. What can we
>> do to improve this?
>>
>> *2. Is our community aware of the sagemath/sage GitHub wiki?*
>> https://github.com/sagemath/sage/wiki
>> - Are the contents of the wiki front page useful?
>> - Are the links to Issue and Pull Request queries helpful?
>>
>>
>> *3. Are the labels on GitHub Issues / PRs helpful?*
>> - Note that new contributors who are not in the Triage team cannot
>> set/change labels!
>> - This includes component labels, but also status labels such as "needs
>> review".
>> - Is it time for the next step with syncing status labels (
>> https://github.com/sagemath/sage/issues/35927)?
>> - Wishlist item: Component auto-labeler for GitHub PRs (
>> https://github.com/sagemath/sage/issues/37373)
>> - Wishlist item: PR size labeler (
>> https://github.com/sagemath/sage/pull/37262)
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/6745500d-2077-42b7-9406-d38c3e280407n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq1_T%2BxGhd4k7DiGx1ByiSCqOorCAuQ_u4LrJ%3DNkczjauw%40mail.gmail.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

On Monday, February 19, 2024 at 3:09:55 PM UTC-8 John H Palmieri wrote:

By the way, I just cloned the Sage repo and ran "make configure", which 
runs `./bootstrap`. The upstream directory is empty after that.


You probably have autoconf/automake/... installed. In this case, it just 
uses them to build the configure script. Downloading is a fallback that is 
used when these "bootstrapping prerequisites" are not installed.


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/50bcb775-87a5-40dc-b0c5-e31000fc337dn%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[John H Palmieri]

If we keep a "configure" tarball in each separate Sage installation but 
they share the rest of "upstream", then we save a lot of space and a lot of 
downloading. A workflow like

% make configure
% configure --with-lots-of-options
% make

would be familiar and unchanged from the status quo when working with the 
git clone. Currently users don't have to run `./bootstrap` manually just to 
build Sage, and it would be nice to keep it this way.

By the way, I just cloned the Sage repo and ran "make configure", which 
runs `./bootstrap`. The upstream directory is empty after that. If it's 
getting used for temporary storage for this tarball, that can be done 
elsewhere (.upstream.d? config? some temporary directory?).

Another option for the location of a shared "upstream" would be Yet Another 
Environment Variable.

On Monday, February 19, 2024 at 2:41:11 PM UTC-8 Dima Pasechnik wrote:

> On Mon, Feb 19, 2024 at 10:29 PM Matthias Koeppe  
> wrote:
>
>> An option to "./configure" could work too, except that the "bootstrap" 
>> phase already downloads the "configure" tarball into that directory.
>
>
> an option to ./bootstrap then would be logical
>  
>
>>
>> Another possible direction: I've been thinking about creating a "./sage 
>> worktree" command, see https://github.com/sagemath/sage/issues/34744
>>
>>
>> On Monday, February 19, 2024 at 2:24:40 PM UTC-8 John H Palmieri wrote:
>>
>>> Regarding symlinking the upstream directory: instead or in addition, 
>>> what about an option to `./configure` for the location of that directory?
>>>
>>>
>>> On Monday, February 19, 2024 at 12:09:54 PM UTC-8 Matthias Koeppe wrote:
>>>
 Prompted by the discussion of space use on the *local machines* of 
 users and developers, I propose another item in addition to A and B:

 *C. Advertise use of "git worktree" and recommend symlinking the 
 "upstream" directory.* For testing a new release when you have an 
 existing clone of the repository, using "git clone" another time is 
 overkill as it creates another copy of the .git directory. And there is no 
 point in having multiple copies of the "upstream" directory, as the 
 filenames of the tarballs change whenever the contents change.

 On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote:

 This (A and B below) has the advantage of being quite explicit. The 
 original proposal

 1) allow standard packages to be pip packages 

 2) drop the contents of upstream/ from the Sage source tarballs. 

 sounds explicit, but the more the discussion goes on, the more I feel 
 that there are hidden pieces. Does the proposal also mean removing version 
 restrictions and all of the other claimed maintenance burden for various 
 components of Sage? 

 Regarding item (2): if I clone the github repository, there is no 
 upstream directory at the start, but after building Sage, it ends up being 
 almost as large as in the current tarballs. (This is on OS X with a lot of 
 homebrew packages installed.) So how much savings are we actually talking 
 about? (Maybe it's not savings for the end user that are important, so 
 *what* are we saving? Disk space on the mirrors?) Dima, can you please 
 provide data? If we convert (according to (1)) to pip packages, those 
 still 
 need to be downloaded, and while they may not end up in "upstream" — I 
 don't actually know how they work — don't they still take up disk space? 
 So 
 again, how much savings are we talking about? Please provide data.

 By the way, the git clone yields a package that is 616M on my machine. 
 A good chunk of that is the .git directory. Are you proposing that we do 
 not distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)

 Regarding item (1): can you provide a list of packages that would 
 become pip packages? Or describe how you would come up with a list?


 On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:

 I'll now offer:

 *Opinion 1. Nobody needs to care in the slightest what the size of that 
 release tarball is. *

 In any use cases with internet connectivity, people will be better off 
 by just cloning the git repo, not use the release tarball.

 If there are relevant use cases without internet connectivity (I have 
 no opinion to offer on this), then the release tarball has exactly the 
 right contents.

 *Proposed action items: *
 *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so 
 that "git clone" is described as the primary way to obtain the Sage 
 sources. That the big release tarball is available can be a footnote in 
 the 
 Installation Guide (
 https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
  
 for the limited no-internet 

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Matthias Koeppe]

(Obviously I meant to say "does NOT set a version 'constraint'.)

On Monday, February 19, 2024 at 9:31:23 AM UTC-8 Matthias Koeppe wrote:

Tobias, you must have missed that making the package standard does set a 
version "constraint". Those are set in "install-requires.txt" files, and in 
https://github.com/sagemath/sage/pull/37301 you can see that the packages 
remain unconstrained.

On Monday, February 19, 2024 at 5:47:46 AM UTC-8 tobia...@gmx.de wrote:

+1 for the one-line change of the type from "optional" to "standard". -1 on 
everything ("standard pip" or "standard wheel") that involves an 
unnecessary version constraint and an explicit declaration of the runtime 
dependencies.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/ae342452-88b3-4fc4-90cc-4d20024d9f7bn%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 10:29 PM Matthias Koeppe 
wrote:

> An option to "./configure" could work too, except that the "bootstrap"
> phase already downloads the "configure" tarball into that directory.


an option to ./bootstrap then would be logical


>
> Another possible direction: I've been thinking about creating a "./sage
> worktree" command, see https://github.com/sagemath/sage/issues/34744
>
>
> On Monday, February 19, 2024 at 2:24:40 PM UTC-8 John H Palmieri wrote:
>
>> Regarding symlinking the upstream directory: instead or in addition, what
>> about an option to `./configure` for the location of that directory?
>>
>>
>> On Monday, February 19, 2024 at 12:09:54 PM UTC-8 Matthias Koeppe wrote:
>>
>>> Prompted by the discussion of space use on the *local machines* of
>>> users and developers, I propose another item in addition to A and B:
>>>
>>> *C. Advertise use of "git worktree" and recommend symlinking the
>>> "upstream" directory.* For testing a new release when you have an
>>> existing clone of the repository, using "git clone" another time is
>>> overkill as it creates another copy of the .git directory. And there is no
>>> point in having multiple copies of the "upstream" directory, as the
>>> filenames of the tarballs change whenever the contents change.
>>>
>>> On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote:
>>>
>>> This (A and B below) has the advantage of being quite explicit. The
>>> original proposal
>>>
>>> 1) allow standard packages to be pip packages
>>>
>>> 2) drop the contents of upstream/ from the Sage source tarballs.
>>>
>>> sounds explicit, but the more the discussion goes on, the more I feel
>>> that there are hidden pieces. Does the proposal also mean removing version
>>> restrictions and all of the other claimed maintenance burden for various
>>> components of Sage?
>>>
>>> Regarding item (2): if I clone the github repository, there is no
>>> upstream directory at the start, but after building Sage, it ends up being
>>> almost as large as in the current tarballs. (This is on OS X with a lot of
>>> homebrew packages installed.) So how much savings are we actually talking
>>> about? (Maybe it's not savings for the end user that are important, so
>>> *what* are we saving? Disk space on the mirrors?) Dima, can you please
>>> provide data? If we convert (according to (1)) to pip packages, those still
>>> need to be downloaded, and while they may not end up in "upstream" — I
>>> don't actually know how they work — don't they still take up disk space? So
>>> again, how much savings are we talking about? Please provide data.
>>>
>>> By the way, the git clone yields a package that is 616M on my machine. A
>>> good chunk of that is the .git directory. Are you proposing that we do not
>>> distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)
>>>
>>> Regarding item (1): can you provide a list of packages that would become
>>> pip packages? Or describe how you would come up with a list?
>>>
>>>
>>> On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:
>>>
>>> I'll now offer:
>>>
>>> *Opinion 1. Nobody needs to care in the slightest what the size of that
>>> release tarball is. *
>>>
>>> In any use cases with internet connectivity, people will be better off
>>> by just cloning the git repo, not use the release tarball.
>>>
>>> If there are relevant use cases without internet connectivity (I have no
>>> opinion to offer on this), then the release tarball has exactly the right
>>> contents.
>>>
>>> *Proposed action items: *
>>> *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so
>>> that "git clone" is described as the primary way to obtain the Sage
>>> sources. That the big release tarball is available can be a footnote in the
>>> Installation Guide (
>>> https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
>>> for the limited no-internet connectivity use case.
>>>
>>> *B. *Likewise, get rid of all of these "Download Sage source code"
>>> pages (https://www.sagemath.org/download-source.html,
>>> https://www.sagemath.org/download-latest.html), mirror selection, etc.
>>> from the Sage website.
>>>
>>>
>>> On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:
>>>
>>> Currently the standard packages cannot be pip packages, i.e. we must, in
>>> effect, vendor them. This entails an extra effort which is often not
>>> needed, in particular as we patch only very few Python packages.
>>> Pip packages are on the other hand installed straight from PyPI.
>>>
>>> Good examples of standard packages which can become pip ones are tox,
>>> pytest (not yet standard).
>>>
>>>
>>> The other difference is that by default these packages are not included
>>> in the Sage releases source tarball.
>>>
>>> Rather than adding them there I propose to split the upstream/* part of
>>> the tarball into something optional - which is represented by a list of
>>> files to download, and which 

[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

An option to "./configure" could work too, except that the "bootstrap" 
phase already downloads the "configure" tarball into that directory.

Another possible direction: I've been thinking about creating a "./sage 
worktree" command, see https://github.com/sagemath/sage/issues/34744


On Monday, February 19, 2024 at 2:24:40 PM UTC-8 John H Palmieri wrote:

> Regarding symlinking the upstream directory: instead or in addition, what 
> about an option to `./configure` for the location of that directory?
>
>
> On Monday, February 19, 2024 at 12:09:54 PM UTC-8 Matthias Koeppe wrote:
>
>> Prompted by the discussion of space use on the *local machines* of users 
>> and developers, I propose another item in addition to A and B:
>>
>> *C. Advertise use of "git worktree" and recommend symlinking the 
>> "upstream" directory.* For testing a new release when you have an 
>> existing clone of the repository, using "git clone" another time is 
>> overkill as it creates another copy of the .git directory. And there is no 
>> point in having multiple copies of the "upstream" directory, as the 
>> filenames of the tarballs change whenever the contents change.
>>
>> On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote:
>>
>> This (A and B below) has the advantage of being quite explicit. The 
>> original proposal
>>
>> 1) allow standard packages to be pip packages 
>>
>> 2) drop the contents of upstream/ from the Sage source tarballs. 
>>
>> sounds explicit, but the more the discussion goes on, the more I feel 
>> that there are hidden pieces. Does the proposal also mean removing version 
>> restrictions and all of the other claimed maintenance burden for various 
>> components of Sage? 
>>
>> Regarding item (2): if I clone the github repository, there is no 
>> upstream directory at the start, but after building Sage, it ends up being 
>> almost as large as in the current tarballs. (This is on OS X with a lot of 
>> homebrew packages installed.) So how much savings are we actually talking 
>> about? (Maybe it's not savings for the end user that are important, so 
>> *what* are we saving? Disk space on the mirrors?) Dima, can you please 
>> provide data? If we convert (according to (1)) to pip packages, those still 
>> need to be downloaded, and while they may not end up in "upstream" — I 
>> don't actually know how they work — don't they still take up disk space? So 
>> again, how much savings are we talking about? Please provide data.
>>
>> By the way, the git clone yields a package that is 616M on my machine. A 
>> good chunk of that is the .git directory. Are you proposing that we do not 
>> distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)
>>
>> Regarding item (1): can you provide a list of packages that would become 
>> pip packages? Or describe how you would come up with a list?
>>
>>
>> On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:
>>
>> I'll now offer:
>>
>> *Opinion 1. Nobody needs to care in the slightest what the size of that 
>> release tarball is. *
>>
>> In any use cases with internet connectivity, people will be better off by 
>> just cloning the git repo, not use the release tarball.
>>
>> If there are relevant use cases without internet connectivity (I have no 
>> opinion to offer on this), then the release tarball has exactly the right 
>> contents.
>>
>> *Proposed action items: *
>> *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so 
>> that "git clone" is described as the primary way to obtain the Sage 
>> sources. That the big release tarball is available can be a footnote in the 
>> Installation Guide (
>> https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
>>  
>> for the limited no-internet connectivity use case.
>>
>> *B. *Likewise, get rid of all of these "Download Sage source code" pages 
>> (https://www.sagemath.org/download-source.html, 
>> https://www.sagemath.org/download-latest.html), mirror selection, etc. 
>> from the Sage website. 
>>
>>
>> On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:
>>
>> Currently the standard packages cannot be pip packages, i.e. we must, in 
>> effect, vendor them. This entails an extra effort which is often not 
>> needed, in particular as we patch only very few Python packages. 
>> Pip packages are on the other hand installed straight from PyPI. 
>>
>> Good examples of standard packages which can become pip ones are tox, 
>> pytest (not yet standard). 
>>
>>
>> The other difference is that by default these packages are not included 
>> in the Sage releases source tarball. 
>>
>> Rather than adding them there I propose to split the upstream/* part of 
>> the tarball into something optional - which is represented by a list of 
>> files to download, and which is just not needed if you build while 
>> connected to the internet. 
>>
>> This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 
>> tarball is 

[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[John H Palmieri]

Regarding symlinking the upstream directory: instead or in addition, what 
about an option to `./configure` for the location of that directory?


On Monday, February 19, 2024 at 12:09:54 PM UTC-8 Matthias Koeppe wrote:

> Prompted by the discussion of space use on the *local machines* of users 
> and developers, I propose another item in addition to A and B:
>
> *C. Advertise use of "git worktree" and recommend symlinking the 
> "upstream" directory.* For testing a new release when you have an 
> existing clone of the repository, using "git clone" another time is 
> overkill as it creates another copy of the .git directory. And there is no 
> point in having multiple copies of the "upstream" directory, as the 
> filenames of the tarballs change whenever the contents change.
>
> On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote:
>
> This (A and B below) has the advantage of being quite explicit. The 
> original proposal
>
> 1) allow standard packages to be pip packages 
>
> 2) drop the contents of upstream/ from the Sage source tarballs. 
>
> sounds explicit, but the more the discussion goes on, the more I feel that 
> there are hidden pieces. Does the proposal also mean removing version 
> restrictions and all of the other claimed maintenance burden for various 
> components of Sage? 
>
> Regarding item (2): if I clone the github repository, there is no upstream 
> directory at the start, but after building Sage, it ends up being almost as 
> large as in the current tarballs. (This is on OS X with a lot of homebrew 
> packages installed.) So how much savings are we actually talking about? 
> (Maybe it's not savings for the end user that are important, so *what* are 
> we saving? Disk space on the mirrors?) Dima, can you please provide data? 
> If we convert (according to (1)) to pip packages, those still need to be 
> downloaded, and while they may not end up in "upstream" — I don't actually 
> know how they work — don't they still take up disk space? So again, how 
> much savings are we talking about? Please provide data.
>
> By the way, the git clone yields a package that is 616M on my machine. A 
> good chunk of that is the .git directory. Are you proposing that we do not 
> distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)
>
> Regarding item (1): can you provide a list of packages that would become 
> pip packages? Or describe how you would come up with a list?
>
>
> On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:
>
> I'll now offer:
>
> *Opinion 1. Nobody needs to care in the slightest what the size of that 
> release tarball is. *
>
> In any use cases with internet connectivity, people will be better off by 
> just cloning the git repo, not use the release tarball.
>
> If there are relevant use cases without internet connectivity (I have no 
> opinion to offer on this), then the release tarball has exactly the right 
> contents.
>
> *Proposed action items: *
> *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so 
> that "git clone" is described as the primary way to obtain the Sage 
> sources. That the big release tarball is available can be a footnote in the 
> Installation Guide (
> https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
>  
> for the limited no-internet connectivity use case.
>
> *B. *Likewise, get rid of all of these "Download Sage source code" pages (
> https://www.sagemath.org/download-source.html, 
> https://www.sagemath.org/download-latest.html), mirror selection, etc. 
> from the Sage website. 
>
>
> On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:
>
> Currently the standard packages cannot be pip packages, i.e. we must, in 
> effect, vendor them. This entails an extra effort which is often not 
> needed, in particular as we patch only very few Python packages. 
> Pip packages are on the other hand installed straight from PyPI. 
>
> Good examples of standard packages which can become pip ones are tox, 
> pytest (not yet standard). 
>
>
> The other difference is that by default these packages are not included in 
> the Sage releases source tarball. 
>
> Rather than adding them there I propose to split the upstream/* part of 
> the tarball into something optional - which is represented by a list of 
> files to download, and which is just not needed if you build while 
> connected to the internet. 
>
> This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 
> tarball is 1.3Gb, and without it is smaller than 0.25Gb. 
>
> Note that as William writes, the desire to have Sage buildable without an 
> internet connection was a requirement by a past Sage funder, gone about 10 
> years ago. Thus there's no longer an obligation to have this option. 
> I am not aware of a similar to Sage which provides tarballs allowing for 
> an offline build. 
>
> Thus, I would like to call a vote on these two topics: 
>
> 1) allow standard 

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Dima Pasechnik]

On 19 February 2024 21:08:54 GMT, John H Palmieri  
wrote:
>
>
>On Monday, February 19, 2024 at 12:10:49 PM UTC-8 Dima Pasechnik wrote:
>
>On Mon, Feb 19, 2024 at 7:42 PM John H Palmieri wrote:
>
>This (A and B below) has the advantage of being quite explicit. The 
>original proposal
>
>1) allow standard packages to be pip packages 
>
>2) drop the contents of upstream/ from the Sage source tarballs. 
>
>sounds explicit, but the more the discussion goes on, the more I feel that 
>there are hidden pieces. Does the proposal also mean removing version 
>restrictions and all of the other claimed maintenance burden for various 
>components of Sage? 
>
>
>No, this is simply FUD spread by certain parties here. 
>
>
>You said: "The difference between wheel packages vs pip packages is that 
>the latter don't require pre-fetched wheels, and absence of the need for 
>package (micro)management." The implication is that changing the package 
>management system is, maybe not part of this proposal, but a next step. In 
>other words, I'm getting this impression from your words, not by other 
>"certain parties."
>
>You said: "My proposal is in fact aimed at reducing the number of pinned 
>Sage dependecies, drastically." (You have made similar comments elsewhere 
>in this thread.) How does (1) accomplish this? Either I'm missing something 
>or you have not spelled everything out in your proposal.
>
>You said '"Allow" does not mean "Make all of the", it should be obvious.'
>
>"Allow" does not cause any changes to happen drastically. So what exactly 
>are you proposing to accomplish these drastic changes? If you have a 
>roadmap in mind, it would be helpful if you described it.

I think we can drastically, by hundreds, reduce the number of Python packages 
we catalogue/vendor in build/pkgs/. Namely, these which are only used deep 
inside Jupyter, Sphinx, and Python build system, if we convert more standard 
Python packages into pip packages.

We can also hope to convert e.g. matplotlib, numpy, scipy, etc 
into pip packages, but this is not as obvious.

This is based on very limited experiments, though.
Anyhow, we have obvious, and safe, candidates for pip package conversion, such 
as pytest.

>
>
> 
>
>pytest is good example of package which can be elevated to standard, but 
>kept pip. In no place my
>proposal 1) demands anything done for all packages.
>
>
>
>Regarding item (2): if I clone the github repository, there is no upstream 
>directory at the start, but after building Sage, it ends up being almost as 
>large as in the current tarballs. (This is on OS X with a lot of homebrew 
>packages installed.) So how much savings are we actually talking about? 
>(Maybe it's not savings for the end user that are important, so *what* are 
>we saving? Disk space on the mirrors?) Dima, can you please provide data? 
>If we convert (according to (1)) to pip packages, those still need to be 
>downloaded, and while they may not end up in "upstream" — I don't actually 
>know how they work — don't they still take up disk space? So again, how 
>much savings are we talking about? Please provide data.
>
>
>I am talking about saving space on mirrors, and on bandwidth, by not 
>packaging "upstream/".
>(as I wrote: This is a huge saving on the tarball size: with upstream/* in, 
>Sage 10.2 tarball is 1.3Gb, and without it is smaller than 0.25Gb.)
>Local disk space nowadays is cheap, but space on mirrors, and bandwidth, 
>don't come free.
>(not everyone is on an unlimited internet)
>
>As well, if you don't wipe up your local upstream/, its contents can  be 
>reused.
>(typically not so many packages are updated with each release after all)
>
>However, as far as I can tell, by default pip package wheels are not stored 
>in upstream/.
>Perhaps there is an easy way to change this, I don't know.
> 
> 
>
>
>By the way, the git clone yields a package that is 616M on my machine. A 
>good chunk of that is the .git directory. Are you proposing that we do not 
>distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.) 
>
>
>Regarding item (1): can you provide a list of packages that would become 
>pip packages? Or describe how you would come up with a list?
>
>
>packages not involved in sagelib directly are good candidates, e.g. pytest, 
>tox.
>sphinx and jupyterlab are good candidates too, in my limited testing 
>experiments.
>
>Dima
>
>
>On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:
>
>I'll now offer:
>
>*Opinion 1. Nobody needs to care in the slightest what the size of that 
>release tarball is. *
>
>In any use cases with internet connectivity, people will be better off by 
>just cloning the git repo, not use the release tarball.
>
>If there are relevant use cases without internet connectivity (I have no 
>opinion to offer on this), then the release tarball has exactly the right 
>contents.
>
>*Proposed action items: *
>*A.* Change https://github.com/sagemath/sage/blob/develop/README.md so that 
>"git clone" is 

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

On Monday, February 19, 2024 at 12:10:49 PM UTC-8 Dima Pasechnik wrote:

On Mon, Feb 19, 2024 at 7:42 PM John H Palmieri  wrote:

If we convert (according to (1)) to pip packages, those still need to be 
downloaded, and while they may not end up in "upstream" — I don't actually 
know how they work — don't they still take up disk space? So again, how 
much savings are we talking about? Please provide data.


as far as I can tell, by default pip package wheels are not stored in 
upstream/.
Perhaps there is an easy way to change this, I don't know.


That's correct. pip packages are installed by directly calling "pip 
install".
If pip downloads a wheel file, it is not stored anywhere.
If pip downloads an sdist and builds a wheel file, then the built wheel is 
stored in pip's cache; see output from "./sage -pip cache info" and "./sage 
-pip cache list".


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/231f8ccb-b60f-49e4-b3cb-b1b88041d351n%40googlegroups.com.

[sage-devel] Re: One year of Sage development on GitHub

[seb....@gmail.com]

 > *Is it time for the next step with syncing status labels 
(https://github.com/sagemath/sage/issues/35927 
)? *

The reason this is blocked is because there is a bug in the GitHub web 
interface that might cause confusion when labels are added or removed by 
the bot. More precisely, the panel with labels is not updated immediately 
after such an action. I have opened two bug reports (2448092 
, 2573072 
). Both were closed 
without a satisfactory answer, only informing that it is a known bug that 
will be fixed one day.


Matthias Koeppe schrieb am Freitag, 16. Februar 2024 um 19:00:52 UTC+1:

> On Thursday, February 8, 2024 at 10:16:58 AM UTC-8 Matthias Koeppe wrote:
>
> On Wednesday, February 7, 2024 at 1:11:14 PM UTC-8 Matthias Koeppe wrote:
>
> Let's also use this anniversary as an opportunity to discuss what still 
> needs improving in our development workflows. 
>
> *1. We have a low development velocity.* For example, some simple PRs sit 
> for weeks or months before receiving any review comments. What can we do to 
> improve this?
>
> *2. Is our community aware of the sagemath/sage GitHub wiki?* 
> https://github.com/sagemath/sage/wiki
> - Are the contents of the wiki front page useful?
> - Are the links to Issue and Pull Request queries helpful? 
>
>
> *3. Are the labels on GitHub Issues / PRs helpful?*
> - Note that new contributors who are not in the Triage team cannot 
> set/change labels! 
> - This includes component labels, but also status labels such as "needs 
> review".
> - Is it time for the next step with syncing status labels (
> https://github.com/sagemath/sage/issues/35927)?
> - Wishlist item: Component auto-labeler for GitHub PRs (
> https://github.com/sagemath/sage/issues/37373)
> - Wishlist item: PR size labeler (
> https://github.com/sagemath/sage/pull/37262)
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/6745500d-2077-42b7-9406-d38c3e280407n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[John H Palmieri]

On Monday, February 19, 2024 at 12:10:49 PM UTC-8 Dima Pasechnik wrote:

On Mon, Feb 19, 2024 at 7:42 PM John H Palmieri wrote:

This (A and B below) has the advantage of being quite explicit. The 
original proposal

1) allow standard packages to be pip packages 

2) drop the contents of upstream/ from the Sage source tarballs. 

sounds explicit, but the more the discussion goes on, the more I feel that 
there are hidden pieces. Does the proposal also mean removing version 
restrictions and all of the other claimed maintenance burden for various 
components of Sage? 


No, this is simply FUD spread by certain parties here. 


You said: "The difference between wheel packages vs pip packages is that 
the latter don't require pre-fetched wheels, and absence of the need for 
package (micro)management." The implication is that changing the package 
management system is, maybe not part of this proposal, but a next step. In 
other words, I'm getting this impression from your words, not by other 
"certain parties."

You said: "My proposal is in fact aimed at reducing the number of pinned 
Sage dependecies, drastically." (You have made similar comments elsewhere 
in this thread.) How does (1) accomplish this? Either I'm missing something 
or you have not spelled everything out in your proposal.

You said '"Allow" does not mean "Make all of the", it should be obvious.'

"Allow" does not cause any changes to happen drastically. So what exactly 
are you proposing to accomplish these drastic changes? If you have a 
roadmap in mind, it would be helpful if you described it.


 

pytest is good example of package which can be elevated to standard, but 
kept pip. In no place my
proposal 1) demands anything done for all packages.



Regarding item (2): if I clone the github repository, there is no upstream 
directory at the start, but after building Sage, it ends up being almost as 
large as in the current tarballs. (This is on OS X with a lot of homebrew 
packages installed.) So how much savings are we actually talking about? 
(Maybe it's not savings for the end user that are important, so *what* are 
we saving? Disk space on the mirrors?) Dima, can you please provide data? 
If we convert (according to (1)) to pip packages, those still need to be 
downloaded, and while they may not end up in "upstream" — I don't actually 
know how they work — don't they still take up disk space? So again, how 
much savings are we talking about? Please provide data.


I am talking about saving space on mirrors, and on bandwidth, by not 
packaging "upstream/".
(as I wrote: This is a huge saving on the tarball size: with upstream/* in, 
Sage 10.2 tarball is 1.3Gb, and without it is smaller than 0.25Gb.)
Local disk space nowadays is cheap, but space on mirrors, and bandwidth, 
don't come free.
(not everyone is on an unlimited internet)

As well, if you don't wipe up your local upstream/, its contents can  be 
reused.
(typically not so many packages are updated with each release after all)

However, as far as I can tell, by default pip package wheels are not stored 
in upstream/.
Perhaps there is an easy way to change this, I don't know.
 
 


By the way, the git clone yields a package that is 616M on my machine. A 
good chunk of that is the .git directory. Are you proposing that we do not 
distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.) 


Regarding item (1): can you provide a list of packages that would become 
pip packages? Or describe how you would come up with a list?


packages not involved in sagelib directly are good candidates, e.g. pytest, 
tox.
sphinx and jupyterlab are good candidates too, in my limited testing 
experiments.

Dima


On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:

I'll now offer:

*Opinion 1. Nobody needs to care in the slightest what the size of that 
release tarball is. *

In any use cases with internet connectivity, people will be better off by 
just cloning the git repo, not use the release tarball.

If there are relevant use cases without internet connectivity (I have no 
opinion to offer on this), then the release tarball has exactly the right 
contents.

*Proposed action items: *
*A.* Change https://github.com/sagemath/sage/blob/develop/README.md so that 
"git clone" is described as the primary way to obtain the Sage sources. 
That the big release tarball is available can be a footnote in the 
Installation Guide (
https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
 
for the limited no-internet connectivity use case.

*B. *Likewise, get rid of all of these "Download Sage source code" pages (
https://www.sagemath.org/download-source.html, 
https://www.sagemath.org/download-latest.html), mirror selection, etc. from 
the Sage website. 


On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:

Currently the standard packages cannot be pip packages, i.e. we must, in 
effect, vendor them. This entails an extra 

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Matthias Koeppe]

On Monday, February 19, 2024 at 12:46:01 PM UTC-8 Dima Pasechnik wrote:

On Mon, Feb 19, 2024 at 8:27 PM Matthias Koeppe  
wrote:

On Monday, February 19, 2024 at 11:51:46 AM UTC-8 Dima Pasechnik wrote:

The purpose of "install-requires.txt" is rather unclear, especially in 
presence
of "package-version.txt".  What is it even doing, in the presence of hard 
version numbers in
package-version.txt ?


Just so I know where I need to improve the explanations -- is it unclear 
*before* or *after* reading the documentation in 
https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#python-based-packages
?


I don't think the docs describe the interaction between package-version.txt 
and install-requires.txt
(and between potential version constraints in spkg-configure.m4)


Let's take this as an opportunity to improve the documentation where 
needed. 
I have opened https://github.com/sagemath/sage/pull/37401 for this, let's 
take the discussion there.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/0feaeda1-ec86-4555-bd86-0f4389bd6a5dn%40googlegroups.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 8:27 PM Matthias Koeppe 
wrote:

> On Monday, February 19, 2024 at 11:51:46 AM UTC-8 Dima Pasechnik wrote:
>
> The purpose of "install-requires.txt" is rather unclear, especially in
> presence
> of "package-version.txt".  What is it even doing, in the presence of hard
> version numbers in
> package-version.txt ?
>
>
> Just so I know where I need to improve the explanations -- is it unclear
> *before* or *after* reading the documentation in
> https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#python-based-packages
> ?
>
>
I don't think the docs describe the interaction between package-version.txt
and install-requires.txt
(and between potential version constraints in spkg-configure.m4)



> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/26d4ab2d-8982-47b9-8707-1ed1d29f8e0en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq2hRL7%3DHe3CKqA8cOL%3DjsTQR6UQ-Zc4qkJ1SYb36f_J9w%40mail.gmail.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Matthias Koeppe]

On Monday, February 19, 2024 at 11:51:46 AM UTC-8 Dima Pasechnik wrote:

The purpose of "install-requires.txt" is rather unclear, especially in 
presence
of "package-version.txt".  What is it even doing, in the presence of hard 
version numbers in
package-version.txt ?


Just so I know where I need to improve the explanations -- is it unclear 
*before* or *after* reading the documentation in 
https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#python-based-packages?

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/26d4ab2d-8982-47b9-8707-1ed1d29f8e0en%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 7:42 PM John H Palmieri 
wrote:

> This (A and B below) has the advantage of being quite explicit. The
> original proposal
>
> 1) allow standard packages to be pip packages
>
> 2) drop the contents of upstream/ from the Sage source tarballs.
>
> sounds explicit, but the more the discussion goes on, the more I feel that
> there are hidden pieces. Does the proposal also mean removing version
> restrictions and all of the other claimed maintenance burden for various
> components of Sage?
>

No, this is simply FUD spread by certain parties here. "Allow" does not
mean "Make all of the", it should be
obvious.

pytest is good example of package which can be elevated to standard, but
kept pip. In no place my
proposal 1) demands anything done for all packages.



> Regarding item (2): if I clone the github repository, there is no upstream
> directory at the start, but after building Sage, it ends up being almost as
> large as in the current tarballs. (This is on OS X with a lot of homebrew
> packages installed.) So how much savings are we actually talking about?
> (Maybe it's not savings for the end user that are important, so *what* are
> we saving? Disk space on the mirrors?) Dima, can you please provide data?
> If we convert (according to (1)) to pip packages, those still need to be
> downloaded, and while they may not end up in "upstream" — I don't actually
> know how they work — don't they still take up disk space? So again, how
> much savings are we talking about? Please provide data.
>

I am talking about saving space on mirrors, and on bandwidth, by not
packaging "upstream/".
(as I wrote: This is a huge saving on the tarball size: with upstream/* in,
Sage 10.2 tarball is 1.3Gb, and without it is smaller than 0.25Gb.)
Local disk space nowadays is cheap, but space on mirrors, and bandwidth,
don't come free.
(not everyone is on an unlimited internet)

As well, if you don't wipe up your local upstream/, its contents can  be
reused.
(typically not so many packages are updated with each release after all)

However, as far as I can tell, by default pip package wheels are not stored
in upstream/.
Perhaps there is an easy way to change this, I don't know.



>
> By the way, the git clone yields a package that is 616M on my machine. A
> good chunk of that is the .git directory. Are you proposing that we do not
> distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)
>

> Regarding item (1): can you provide a list of packages that would become
> pip packages? Or describe how you would come up with a list?
>

packages not involved in sagelib directly are good candidates, e.g. pytest,
tox.
sphinx and jupyterlab are good candidates too, in my limited testing
experiments.

Dima


On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:
>
>> I'll now offer:
>>
>> *Opinion 1. Nobody needs to care in the slightest what the size of that
>> release tarball is. *
>>
>> In any use cases with internet connectivity, people will be better off by
>> just cloning the git repo, not use the release tarball.
>>
>> If there are relevant use cases without internet connectivity (I have no
>> opinion to offer on this), then the release tarball has exactly the right
>> contents.
>>
>> *Proposed action items: *
>> *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so
>> that "git clone" is described as the primary way to obtain the Sage
>> sources. That the big release tarball is available can be a footnote in the
>> Installation Guide (
>> https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
>> for the limited no-internet connectivity use case.
>>
>> *B. *Likewise, get rid of all of these "Download Sage source code" pages
>> (https://www.sagemath.org/download-source.html,
>> https://www.sagemath.org/download-latest.html), mirror selection, etc.
>> from the Sage website.
>>
>>
>> On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:
>>
>>> Currently the standard packages cannot be pip packages, i.e. we must, in
>>> effect, vendor them. This entails an extra effort which is often not
>>> needed, in particular as we patch only very few Python packages.
>>> Pip packages are on the other hand installed straight from PyPI.
>>>
>>> Good examples of standard packages which can become pip ones are tox,
>>> pytest (not yet standard).
>>>
>>>
>>> The other difference is that by default these packages are not included
>>> in the Sage releases source tarball.
>>>
>>> Rather than adding them there I propose to split the upstream/* part of
>>> the tarball into something optional - which is represented by a list of
>>> files to download, and which is just not needed if you build while
>>> connected to the internet.
>>>
>>> This is a huge saving on the tarball size: with upstream/* in, Sage 10.2
>>> tarball is 1.3Gb, and without it is smaller than 0.25Gb.
>>>
>>> Note that as William writes, the desire to have Sage buildable without

[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

Prompted by the discussion of space use on the *local machines* of users 
and developers, I propose another item in addition to A and B:

*C. Advertise use of "git worktree" and recommend symlinking the "upstream" 
directory.* For testing a new release when you have an existing clone of 
the repository, using "git clone" another time is overkill as it creates 
another copy of the .git directory. And there is no point in having 
multiple copies of the "upstream" directory, as the filenames of the 
tarballs change whenever the contents change.

On Monday, February 19, 2024 at 11:42:01 AM UTC-8 John H Palmieri wrote:

This (A and B below) has the advantage of being quite explicit. The 
original proposal

1) allow standard packages to be pip packages 

2) drop the contents of upstream/ from the Sage source tarballs. 

sounds explicit, but the more the discussion goes on, the more I feel that 
there are hidden pieces. Does the proposal also mean removing version 
restrictions and all of the other claimed maintenance burden for various 
components of Sage? 

Regarding item (2): if I clone the github repository, there is no upstream 
directory at the start, but after building Sage, it ends up being almost as 
large as in the current tarballs. (This is on OS X with a lot of homebrew 
packages installed.) So how much savings are we actually talking about? 
(Maybe it's not savings for the end user that are important, so *what* are 
we saving? Disk space on the mirrors?) Dima, can you please provide data? 
If we convert (according to (1)) to pip packages, those still need to be 
downloaded, and while they may not end up in "upstream" — I don't actually 
know how they work — don't they still take up disk space? So again, how 
much savings are we talking about? Please provide data.

By the way, the git clone yields a package that is 616M on my machine. A 
good chunk of that is the .git directory. Are you proposing that we do not 
distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)

Regarding item (1): can you provide a list of packages that would become 
pip packages? Or describe how you would come up with a list?


On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:

I'll now offer:

*Opinion 1. Nobody needs to care in the slightest what the size of that 
release tarball is. *

In any use cases with internet connectivity, people will be better off by 
just cloning the git repo, not use the release tarball.

If there are relevant use cases without internet connectivity (I have no 
opinion to offer on this), then the release tarball has exactly the right 
contents.

*Proposed action items: *
*A.* Change https://github.com/sagemath/sage/blob/develop/README.md so that 
"git clone" is described as the primary way to obtain the Sage sources. 
That the big release tarball is available can be a footnote in the 
Installation Guide (
https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
 
for the limited no-internet connectivity use case.

*B. *Likewise, get rid of all of these "Download Sage source code" pages (
https://www.sagemath.org/download-source.html, 
https://www.sagemath.org/download-latest.html), mirror selection, etc. from 
the Sage website. 


On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:

Currently the standard packages cannot be pip packages, i.e. we must, in 
effect, vendor them. This entails an extra effort which is often not 
needed, in particular as we patch only very few Python packages. 
Pip packages are on the other hand installed straight from PyPI. 

Good examples of standard packages which can become pip ones are tox, 
pytest (not yet standard). 


The other difference is that by default these packages are not included in 
the Sage releases source tarball. 

Rather than adding them there I propose to split the upstream/* part of the 
tarball into something optional - which is represented by a list of files 
to download, and which is just not needed if you build while connected to 
the internet. 

This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 
tarball is 1.3Gb, and without it is smaller than 0.25Gb. 

Note that as William writes, the desire to have Sage buildable without an 
internet connection was a requirement by a past Sage funder, gone about 10 
years ago. Thus there's no longer an obligation to have this option. 
I am not aware of a similar to Sage which provides tarballs allowing for an 
offline build. 

Thus, I would like to call a vote on these two topics: 

1) allow standard packages to be pip packages 

2) drop the contents of upstream/ from the Sage source tarballs. 


--- 
Dima 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 7:34 PM Matthias Koeppe 
wrote:

> On Monday, February 19, 2024 at 10:35:45 AM UTC-8 Dima Pasechnik wrote:
>
> On Mon, Feb 19, 2024 at 5:31 PM Matthias Koeppe  > wrote:
>
> On Monday, February 19, 2024 at 5:47:46 AM UTC-8 tobia...@gmx.de wrote:
>
> +1 for the one-line change of the type from "optional" to "standard". -1
> on everything ("standard pip" or "standard wheel") that involves an
> unnecessary version constraint and an explicit declaration of the runtime
> dependencies.
>
>
> Tobias, you must have missed that making the package standard does set a
> version "constraint". Those are set in "install-requires.txt" files, and in
> https://github.com/sagemath/sage/pull/37301 you can see that the packages
> remain unconstrained.
>
>
> What does this comment have to do with the question at hand?
>
>
> Not sure what you're asking. I've clearly quoted what I'm responding to.
>
>
> Package versions of standard packages at the moment  are set up in
> package-version.txt
> files, as it clear for all to see in your own PR making pytest* into
> standard packages:
> https://github.com/sagemath/sage/pull/37301
>
>
> "For all to see" in the PRs also that there are the files
> "install-requires.txt" that I mentioned.
> Their purpose is documented in our Developer Guide -
> https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#python-based-packages
> But for convenience I'll include an explanation below.
>

your explanation is not very clear. E.g.  'Pinning to specific versions is
done using a file "requirements.txt"'
should read 'Pinning to specific versions, or to version ranges, can be
done using a file "requirements.txt"'


>
> (with all the mess of checksums, tarballs/wheels, etc etc)
>
>
> Calling it a "mess" is not helpful for the discussion.
> I'd ask for using much more restraint in using such rhetorical devices.
>

A mess is still a mess. The purpose of "install-requires.txt" is rather
unclear, especially in presence
of "package-version.txt".  What is it even doing, in the presence of hard
version numbers in
package-version.txt ?

By the way, "you must have missed that ..." is certainly a rhetorical, and
I'd say, patronizing, device.



>
> Pip package versions need not to be explicit, however, and one won't need
> to
> worry about checksums, tarballs/wheels (unless installing Sage on an
> air-gap system...)
>
>
> In the Python (pip) world:
> - Version constraints of dependencies of a Python project are declared in
> pyproject.toml [project] dependencies (
> https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency);
> or, equivalently. in the older formats: setup.cfg install-requires, or
> setup.py install_requires
> - Pinning to specific versions is done using a file "requirements.txt" (
> https://pip.pypa.io/en/stable/reference/requirements-file-format/)
> - This file can be created and updated by using "pip freeze".
>
> In the conda world:
> - Version constraints of dependencies of a project are declared in
> environment.yml
> - Pinning to specific versions can be done in separate environment file
> such as conda-lock.yml
> - Updating the lock file can be done using conda-lock (
> https://github.com/conda/conda-lock?tab=readme-ov-file#why)
> - Example in Sage:
> https://github.com/sagemath/sage/blob/develop/src/environment-dev-3.11-macos-arm64.yml#L329
> pins "pytest=7.4.3=pyhd8ed1ab_0" and its dependency
> "iniconfig=2.0.0=pyhd8ed1ab_0" (
> https://github.com/sagemath/sage/blob/develop/src/environment-dev-3.11-macos-arm64.yml#L194
> )
>
> In the Sage distribution:
> - Version constraints of normal Python packages are declared in files
> build/pkgs/*/install-requires.txt
> - Pinning to specific versions is done in the files
> build/pkgs/*/package-version.txt
> - Updating the pins is done using "sage --package update" or "sage
> --package update-latest".
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/94060e30-f224-44fd-841e-7b2060a222f1n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq2yw6wLHsyMyaWYWAYjAbzo9VSc_B8Xkac-iw%3DQ5%2BpYgQ%40mail.gmail.com.

[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[John H Palmieri]

This (A and B below) has the advantage of being quite explicit. The 
original proposal

1) allow standard packages to be pip packages 

2) drop the contents of upstream/ from the Sage source tarballs. 

sounds explicit, but the more the discussion goes on, the more I feel that 
there are hidden pieces. Does the proposal also mean removing version 
restrictions and all of the other claimed maintenance burden for various 
components of Sage? 

Regarding item (2): if I clone the github repository, there is no upstream 
directory at the start, but after building Sage, it ends up being almost as 
large as in the current tarballs. (This is on OS X with a lot of homebrew 
packages installed.) So how much savings are we actually talking about? 
(Maybe it's not savings for the end user that are important, so *what* are 
we saving? Disk space on the mirrors?) Dima, can you please provide data? 
If we convert (according to (1)) to pip packages, those still need to be 
downloaded, and while they may not end up in "upstream" — I don't actually 
know how they work — don't they still take up disk space? So again, how 
much savings are we talking about? Please provide data.

By the way, the git clone yields a package that is 616M on my machine. A 
good chunk of that is the .git directory. Are you proposing that we do not 
distribute this? (A recent beta tarball is 1.4G, unpacked 1.6G.)

Regarding item (1): can you provide a list of packages that would become 
pip packages? Or describe how you would come up with a list?


On Monday, February 12, 2024 at 3:52:29 PM UTC-8 Matthias Koeppe wrote:

> I'll now offer:
>
> *Opinion 1. Nobody needs to care in the slightest what the size of that 
> release tarball is. *
>
> In any use cases with internet connectivity, people will be better off by 
> just cloning the git repo, not use the release tarball.
>
> If there are relevant use cases without internet connectivity (I have no 
> opinion to offer on this), then the release tarball has exactly the right 
> contents.
>
> *Proposed action items: *
> *A.* Change https://github.com/sagemath/sage/blob/develop/README.md so 
> that "git clone" is described as the primary way to obtain the Sage 
> sources. That the big release tarball is available can be a footnote in the 
> Installation Guide (
> https://deploy-livedoc--sagemath.netlify.app/html/en/installation/source#installation-steps)
>  
> for the limited no-internet connectivity use case.
>
> *B. *Likewise, get rid of all of these "Download Sage source code" pages (
> https://www.sagemath.org/download-source.html, 
> https://www.sagemath.org/download-latest.html), mirror selection, etc. 
> from the Sage website. 
>
>
> On Sunday, February 11, 2024 at 11:23:42 AM UTC-8 Dima Pasechnik wrote:
>
>> Currently the standard packages cannot be pip packages, i.e. we must, in 
>> effect, vendor them. This entails an extra effort which is often not 
>> needed, in particular as we patch only very few Python packages. 
>> Pip packages are on the other hand installed straight from PyPI. 
>>
>> Good examples of standard packages which can become pip ones are tox, 
>> pytest (not yet standard). 
>>
>>
>> The other difference is that by default these packages are not included 
>> in the Sage releases source tarball. 
>>
>> Rather than adding them there I propose to split the upstream/* part of 
>> the tarball into something optional - which is represented by a list of 
>> files to download, and which is just not needed if you build while 
>> connected to the internet. 
>>
>> This is a huge saving on the tarball size: with upstream/* in, Sage 10.2 
>> tarball is 1.3Gb, and without it is smaller than 0.25Gb. 
>>
>> Note that as William writes, the desire to have Sage buildable without an 
>> internet connection was a requirement by a past Sage funder, gone about 10 
>> years ago. Thus there's no longer an obligation to have this option. 
>> I am not aware of a similar to Sage which provides tarballs allowing for 
>> an offline build. 
>>
>> Thus, I would like to call a vote on these two topics: 
>>
>> 1) allow standard packages to be pip packages 
>>
>> 2) drop the contents of upstream/ from the Sage source tarballs. 
>>
>>
>> --- 
>> Dima 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/0a3c1c0b-ed3f-41eb-ab4a-5235432475dbn%40googlegroups.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Matthias Koeppe]

On Monday, February 19, 2024 at 10:35:45 AM UTC-8 Dima Pasechnik wrote:

On Mon, Feb 19, 2024 at 5:31 PM Matthias Koeppe https://groups.google.com/>> wrote:

On Monday, February 19, 2024 at 5:47:46 AM UTC-8 tobia...@gmx.de wrote:

+1 for the one-line change of the type from "optional" to "standard". -1 on 
everything ("standard pip" or "standard wheel") that involves an 
unnecessary version constraint and an explicit declaration of the runtime 
dependencies.


Tobias, you must have missed that making the package standard does set a 
version "constraint". Those are set in "install-requires.txt" files, and in 
https://github.com/sagemath/sage/pull/37301 you can see that the packages 
remain unconstrained.


What does this comment have to do with the question at hand?


Not sure what you're asking. I've clearly quoted what I'm responding to.
 

Package versions of standard packages at the moment  are set up in 
package-version.txt
files, as it clear for all to see in your own PR making pytest* into 
standard packages:
https://github.com/sagemath/sage/pull/37301


"For all to see" in the PRs also that there are the files 
"install-requires.txt" that I mentioned.
Their purpose is documented in our Developer Guide - 
https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#python-based-packages
But for convenience I'll include an explanation below.
 

(with all the mess of checksums, tarballs/wheels, etc etc)


Calling it a "mess" is not helpful for the discussion. 
I'd ask for using much more restraint in using such rhetorical devices.

Pip package versions need not to be explicit, however, and one won't need to
worry about checksums, tarballs/wheels (unless installing Sage on an 
air-gap system...)


In the Python (pip) world:
- Version constraints of dependencies of a Python project are declared in 
pyproject.toml [project] dependencies 
(https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency);
 
or, equivalently. in the older formats: setup.cfg install-requires, or 
setup.py install_requires
- Pinning to specific versions is done using a file "requirements.txt" 
(https://pip.pypa.io/en/stable/reference/requirements-file-format/)
- This file can be created and updated by using "pip freeze".

In the conda world:
- Version constraints of dependencies of a project are declared in 
environment.yml
- Pinning to specific versions can be done in separate environment file 
such as conda-lock.yml
- Updating the lock file can be done using conda-lock 
(https://github.com/conda/conda-lock?tab=readme-ov-file#why)
- Example in 
Sage: 
https://github.com/sagemath/sage/blob/develop/src/environment-dev-3.11-macos-arm64.yml#L329
 
pins "pytest=7.4.3=pyhd8ed1ab_0" and its dependency 
"iniconfig=2.0.0=pyhd8ed1ab_0" 
(https://github.com/sagemath/sage/blob/develop/src/environment-dev-3.11-macos-arm64.yml#L194)

In the Sage distribution:
- Version constraints of normal Python packages are declared in files 
build/pkgs/*/install-requires.txt
- Pinning to specific versions is done in the files 
build/pkgs/*/package-version.txt
- Updating the pins is done using "sage --package update" or "sage 
--package update-latest".



-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/94060e30-f224-44fd-841e-7b2060a222f1n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 5:16 PM Matthias Koeppe 
wrote:

> On Monday, February 19, 2024 at 5:42:08 AM UTC-8 tobia...@gmx.de wrote:
>
> This discussion about the need to fix the version of pytest *and its
> runtime dependencies* is almost comical.
>
>
> No, you are in the wrong thread.
>
> This thread is about the general policy for standard packages, not about
> pytest.
>

Come on! Tobias is in the right thread.
pytest is a good example of standard package which can well be made a pip
package, that's why it's mentioned here.


>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/e6477c9c-9a52-4b71-829d-b718cbbd5402n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq1xQ9fG%3DGLqvsWKTduQTiamQV2vPyvpphHXYxMFC5n21w%40mail.gmail.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Dima Pasechnik]

On Mon, Feb 19, 2024 at 5:31 PM Matthias Koeppe 
wrote:

> Tobias, you must have missed that making the package standard does set a
> version "constraint". Those are set in "install-requires.txt" files, and in
> https://github.com/sagemath/sage/pull/37301 you can see that the packages
> remain unconstrained.


What does this comment have to do with the question at hand?

Package versions of standard packages at the moment  are set up in
package-version.txt
files, as it clear for all to see in your own PR making pytest* into
standard packages:
https://github.com/sagemath/sage/pull/37301
(with all the mess of checksums, tarballs/wheels, etc etc)

Pip package versions need not to be explicit, however, and one won't need to
worry about checksums, tarballs/wheels (unless installing Sage on an
air-gap system...)




>
> On Monday, February 19, 2024 at 5:47:46 AM UTC-8 tobia...@gmx.de wrote:
>
>> +1 for the one-line change of the type from "optional" to "standard". -1
>> on everything ("standard pip" or "standard wheel") that involves an
>> unnecessary version constraint and an explicit declaration of the runtime
>> dependencies.
>>
>>
>>
>> On Saturday, February 17, 2024 at 4:51:45 PM UTC+8 Dima Pasechnik wrote:
>>
>>> I have made it clear under what condition you can count my vote as +1.
>>>
>>> To make it clear: it is -1 if the condition is not met.
>>>
>>>
>>> On 17 February 2024 04:26:14 GMT, Matthias Koeppe 
>>> wrote:
>>>
 Note that posting a proposal here on sage-devel to make the packages
 standard followed the policies of our project.
 "optional packages stay in that status for at least a year, after which
 they can be proposed to be included as standard packages in Sage. For this
 a GitHub PR is opened with the label c: packages: standard. Then make a
 proposal in the Google Group sage-devel."
 https://doc.sagemath.org/html/en/developer/packaging.html#inclusion-procedure-for-new-and-updated-packages

 Multicasting "negative reviews" to tickets is not part of the
 procedures of our project; cf.
 https://github.com/sagemath/sage/pull/36726#issuecomment-1820148873 on
 the emergence of this idea.

 On Friday, February 16, 2024 at 4:34:12 PM UTC-8 Dima Pasechnik wrote:

>
>
> On 16 February 2024 23:33:48 GMT, Matthias Koeppe <
> matthia...@gmail.com> wrote:
> >On Friday, February 16, 2024 at 1:25:13 PM UTC-8 Dima Pasechnik
> wrote:
> >
> >>On Friday, February 16, 2024 at 9:38:16 AM UTC-8 Dima Pasechnik
> wrote:
> >My vote for is conditional on them remaining pip packages, and that's
> not
> >what your PRs do.
> >
> >
> >I'll count that as 0.
>
> you can count that as negative reviews on your PRs
> as they are now.
>
>
> >
>
 --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/0b84b09a-3a20-4a5f-91ea-f38566b33ed4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq3%2B%3DDCbm3SHyNU10WZscOddgkHXx9OfEXWKxdQVeJxtPA%40mail.gmail.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

[Matthias Koeppe]

Tobias, you must have missed that making the package standard does set a 
version "constraint". Those are set in "install-requires.txt" files, and 
in https://github.com/sagemath/sage/pull/37301 you can see that the 
packages remain unconstrained.

On Monday, February 19, 2024 at 5:47:46 AM UTC-8 tobia...@gmx.de wrote:

> +1 for the one-line change of the type from "optional" to "standard". -1 
> on everything ("standard pip" or "standard wheel") that involves an 
> unnecessary version constraint and an explicit declaration of the runtime 
> dependencies.
>
>
>
> On Saturday, February 17, 2024 at 4:51:45 PM UTC+8 Dima Pasechnik wrote:
>
>> I have made it clear under what condition you can count my vote as +1.
>>
>> To make it clear: it is -1 if the condition is not met.
>>
>>
>> On 17 February 2024 04:26:14 GMT, Matthias Koeppe  
>> wrote:
>>
>>> Note that posting a proposal here on sage-devel to make the packages 
>>> standard followed the policies of our project. 
>>> "optional packages stay in that status for at least a year, after which 
>>> they can be proposed to be included as standard packages in Sage. For this 
>>> a GitHub PR is opened with the label c: packages: standard. Then make a 
>>> proposal in the Google Group sage-devel." 
>>> https://doc.sagemath.org/html/en/developer/packaging.html#inclusion-procedure-for-new-and-updated-packages
>>>
>>> Multicasting "negative reviews" to tickets is not part of the procedures 
>>> of our project; cf. 
>>> https://github.com/sagemath/sage/pull/36726#issuecomment-1820148873 on 
>>> the emergence of this idea.
>>>
>>> On Friday, February 16, 2024 at 4:34:12 PM UTC-8 Dima Pasechnik wrote:
>>>


 On 16 February 2024 23:33:48 GMT, Matthias Koeppe  
 wrote: 
 >On Friday, February 16, 2024 at 1:25:13 PM UTC-8 Dima Pasechnik wrote: 
 > 
 >>On Friday, February 16, 2024 at 9:38:16 AM UTC-8 Dima Pasechnik 
 wrote: 
 >My vote for is conditional on them remaining pip packages, and that's 
 not 
 >what your PRs do. 
 > 
 > 
 >I'll count that as 0. 

 you can count that as negative reviews on your PRs 
 as they are now. 


 > 

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/0b84b09a-3a20-4a5f-91ea-f38566b33ed4n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

On Monday, February 19, 2024 at 5:42:08 AM UTC-8 tobia...@gmx.de wrote:

This discussion about the need to fix the version of pytest *and its 
runtime dependencies* is almost comical.


No, you are in the wrong thread. 

This thread is about the general policy for standard packages, not about 
pytest. 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/e6477c9c-9a52-4b71-829d-b718cbbd5402n%40googlegroups.com.

Re: [sage-devel] Re: Proposal: Make pytest, pytest_xdist, pytest_mock, python_build standard packages

['tobia...@gmx.de' via sage-devel]

+1 for the one-line change of the type from "optional" to "standard". -1 on 
everything ("standard pip" or "standard wheel") that involves an 
unnecessary version constraint and an explicit declaration of the runtime 
dependencies.



On Saturday, February 17, 2024 at 4:51:45 PM UTC+8 Dima Pasechnik wrote:

> I have made it clear under what condition you can count my vote as +1.
>
> To make it clear: it is -1 if the condition is not met.
>
>
> On 17 February 2024 04:26:14 GMT, Matthias Koeppe  
> wrote:
>
>> Note that posting a proposal here on sage-devel to make the packages 
>> standard followed the policies of our project. 
>> "optional packages stay in that status for at least a year, after which 
>> they can be proposed to be included as standard packages in Sage. For this 
>> a GitHub PR is opened with the label c: packages: standard. Then make a 
>> proposal in the Google Group sage-devel." 
>> https://doc.sagemath.org/html/en/developer/packaging.html#inclusion-procedure-for-new-and-updated-packages
>>
>> Multicasting "negative reviews" to tickets is not part of the procedures 
>> of our project; cf. 
>> https://github.com/sagemath/sage/pull/36726#issuecomment-1820148873 on 
>> the emergence of this idea.
>>
>> On Friday, February 16, 2024 at 4:34:12 PM UTC-8 Dima Pasechnik wrote:
>>
>>>
>>>
>>> On 16 February 2024 23:33:48 GMT, Matthias Koeppe  
>>> wrote: 
>>> >On Friday, February 16, 2024 at 1:25:13 PM UTC-8 Dima Pasechnik wrote: 
>>> > 
>>> >>On Friday, February 16, 2024 at 9:38:16 AM UTC-8 Dima Pasechnik wrote: 
>>> >My vote for is conditional on them remaining pip packages, and that's 
>>> not 
>>> >what your PRs do. 
>>> > 
>>> > 
>>> >I'll count that as 0. 
>>>
>>> you can count that as negative reviews on your PRs 
>>> as they are now. 
>>>
>>>
>>> > 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/265a2a83-5d12-4f4b-b243-f207744aa6b0n%40googlegroups.com.

Re: [sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

['tobia...@gmx.de' via sage-devel]

This discussion about the need to fix the version of pytest *and its 
runtime dependencies* is almost comical. We are installing and running 
pytest successfully since 3 years without any version requirement via pip 
in ci and experienced zero issues. We are also not alone in that. For 
example, scipy also doesn't pin any pytest version 
(https://github.com/search?q=repo%3Ascipy%2Fscipy%20pytest%20lang%3Ayml=code).
 
Can you name a single project that needed to pin one of the runtime 
dependencies of pytest because of a bug they experienced?

On Monday, February 19, 2024 at 7:47:26 AM UTC+8 Matthias Koeppe wrote:

> On Sunday, February 18, 2024 at 10:05:21 AM UTC-8 Dima Pasechnik wrote:
>
> On Sun, Feb 18, 2024 at 5:24 PM Matthias Koeppe  
> wrote:
>
> On Sunday, February 18, 2024 at 9:07:04 AM UTC-8 Dima Pasechnik wrote:
>
>   
>
> 2) The major improvement is that sagelib will be easier to install into an 
> existing venv, and that's a wish of quite a number of users. Much more 
> Pythonic, too.
>
>
> The pip-installability of sagelib has absolutely nothing to do with this 
> discussion.
>
>
> Of course it does, a lot. As having less deps pinning will model 
> installability and useability of sagelib in a "foreign" venv. At the moment 
> using sagelib in a foreign venv is complicated and error-prone, and 
> untested.
>
>
> The dependencies of sagelib are declared in the template 
> https://github.com/sagemath/sage/blob/develop/src/setup.cfg.m4#L13, which 
> is filled with the data from the "install-requires.txt" files (such as 
> https://github.com/sagemath/sage/blob/develop/build/pkgs/cysignals/install-requires.txt
> ).
> These files provide version ranges. They do NOT use the specific versions 
> pinned in the Sage distribution.
>
> I don't think it's necessary to comment much on the idea that we should 
> change our production environment (unpinning the pinned versions of the 
> Sage distribution) to make it "model installability and usability of 
> sagelib" in a less controlled environment.
>
> Testing sagelib (the sagemath-standard distribution package) in different 
> environments is not "complicated" -- because I've built the convenient 
> tooling for that. One does not have to make changes to the Sage 
> distribution for that. See 
> https://github.com/sagemath/sage/blob/develop/pkgs/sagemath-standard/tox.ini#L18
>  
> for the different options. See also the developer's guide 
> https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging_sage_library#testing-the-distribution-in-virtual-environments-with-tox,
>  
> where all of this is documented.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/8c75300e-ae84-49ea-bd05-03443dd745ebn%40googlegroups.com.