[sage-devel] Re: [Proposal] allow standard packages to be pip packages, reduce source tarball size

[Matthias Koeppe]

On Friday, May 31, 2024 at 9:38:34 AM UTC-7 Dima Pasechnik wrote:

Before looking at 
https://groups.google.com/g/sage-devel/c/lPLoA7zaoyg/m/dGE1B1jQEQAJ
we should look at this proposal again, as pytest is a very suitable 
candidate for the kinds of packages (standard pip packages)
proposed here


David requested over there that I explain a bit more about pip packages, so 
here goes.

The "pip" package type was added to the Sage build system in 2015 by Jeroen 
as a quick and dirty way to keep packages such as "biopython", previously 
available as an "old-style SPKG", available for installing after switching 
to the "git layout" and phasing out "old-style SPKGs". We have kept this 
mechanism around for exactly such use cases: To advertise packages for 
which there is no real interest among developers in packaging them (and 
updating them) properly. 

And the "pip" package type has not seen any development since 2015. In 
particular, it has not been updated in any way to account for the use of 
build isolation that has become the default in the Python world around 
2020. For example, say a "pip" package has a build dependency on numpy -- 
then during the build phase it's not even going to use our installed numpy 
package but rather it's going to use a random binary wheel from PyPI. And 
all of that is resolved against the index server too, so there will be 
retroactive breakage.

Moreover, when a "pip" package is installed the first time, the Sage 
distribution really just calls "pip install ..."; this contacts the index 
server (PyPI). And the Sage build system does not keep or check 
installation records for these packages, so it just calls "pip install ..." 
again *every time* that its installation is requested directly or 
indirectly as a dependency. And "pip install" will happily do anything that 
is requested, such as replacing any currently installed Python packages, 
including packages installed by the Sage distribution. Nothing is pinned, 
everything is up for grabs, and what is finally installed will also depend 
on the order of operations.

It's really very surprising to me why anyone would be championing using 
these kinds of package types more. 

Well, it does sound all very nice and simple, but unfortunately in reality 
it's not. My suggestion to those who are actually interested in discussing 
the build system of the Sage distribution: Learn how the Sage distribution 
builds its Python packages, about its wheelhouse in 
"venv/var/lib/sage/wheels" and the generated requirements files in 
"venv/var/lib/sage/scripts", which is how the Sage distributions achieves a 
reliable and robust installation of its Python packages.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/c3737fdd-1002-470f-b856-2381efd218afn%40googlegroups.com.

Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 1:38:39 PM UTC-7 David Roe wrote:

On Wed, Jun 5, 2024 at 4:25 PM Matthias Koeppe  wrote:

And the policy exists because it was made in awareness of the limitations 
of "pip" packages.


I see the policy you refer to here 
, but what 
limitations of pip packages prevent them from being standard?


This discussion does not belong into this thread, but I'll explain a bit of 
this in the other thread.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/1a28375b-b06b-4b23-ba65-f59b0a75dd05n%40googlegroups.com.

Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages

[David Roe]

On Wed, Jun 5, 2024 at 4:25 PM Matthias Koeppe 
wrote:

> On Monday, June 3, 2024 at 12:29:14 PM UTC-7 dim...@gmail.com wrote:
>
> pytest can be kept a pip package, just promoted to standard.
>
>
> It cannot, per existing policy.
>
> And the policy exists because it was made in awareness of the limitations
> of "pip" packages.
>

I see the policy you refer to here
, but what
limitations of pip packages prevent them from being standard?
David


> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-devel/4be6a58d-8fb6-45f0-84ae-c689b43e4502n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAChs6_mNZ%2B16vVWZQ0orWYh8mg6KNS0mW3JEKN7CzkBNdcR%3DXA%40mail.gmail.com.

[sage-devel] Re: Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages

[Matthias Koeppe]

All, please send your comments on this proposal.

I hope that we can count votes here by the end of the week.

On Thursday, May 30, 2024 at 3:25:08 PM UTC-7 Matthias Koeppe wrote:

> We added the packages as optional "pip" packages (see 
> https://deploy-livedoc--sagemath.netlify.app/html/en/developer/packaging#package-types
>  for 
> the terminology), each more than 1 year ago.
>
> - 
> https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest#spkg-pytest
>  (added 
> in 2020)
> - 
> https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_mock#spkg-pytest-mock
>  (added 
> in 2022)
> - 
> https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/pytest_xdist#spkg-pytest-xdist
>  (added 
> in 2022)
>
> "pytest" is the current gold standard for running tests of Python 
> packages. Various standard packages in the Sage distribution already 
> declare pytest in "dependencies_check" as a conditional dependency for use 
> when SAGE_CHECK=yes is set. The support for testing parts of the Sage 
> library with pytest was from an effort largely stalled after 2022 -- and 
> which has been completely broken since early 2024 with the arrival of 
> pytest 8.x, which I have just fixed in 
> https://github.com/sagemath/sage/pull/37999 (to be merged). I'll note 
> that recent versions of pytest have added support for PEP-420 implicit 
> namespace package, which the Sage library uses as part of the 
> modularization effort.
>
> By making pytest a standard package, I would hope to help revive this 
> effort, and thus the larger effort to "adopt mainstream Python 
> testing/linting infrastructure" (see 
> https://github.com/sagemath/sage/issues/28936). 
>
> I'm proposing to make the packages (and their dependencies) standard 
> (wheel) packages according to the procedures in our developer guide. 
> - Reference to previous such proposals following our project's procedures: 
> https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/KyOOQzkzAAAJ
> - Link to our packaging documentation and explanation how making it a 
> standard package compares to version pinning done for example using 
> conda-lock:  
> https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/N2vuKb-TAAAJ
>
> The other pytest_* packages are related technical packages.
> The PR that implements it: https://github.com/sagemath/sage/pull/37301
>
> This is a redo of the 2nd part of my proposal 
> https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc/m/NsyUa7iXAgAJ from 
> Feb 10, which was stalled. (The 1st part (on python_build) was redone in 
> https://groups.google.com/g/sage-devel/c/EMeTJ6Hwgns/m/9_zh6usoAAAJ and 
> approved.)
>
>
> I ask everyone to focus on the specifics of this proposal.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/c9fbfdc6-acb2-4f79-87fb-1eab4c43de4dn%40googlegroups.com.

Re: [sage-devel] Proposal (redo): Make pytest, pytest_mock, pytest_xdist + dependencies standard packages

[Matthias Koeppe]

On Monday, June 3, 2024 at 12:29:14 PM UTC-7 dim...@gmail.com wrote:

pytest can be kept a pip package, just promoted to standard.


It cannot, per existing policy.

And the policy exists because it was made in awareness of the limitations 
of "pip" packages.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/4be6a58d-8fb6-45f0-84ae-c689b43e4502n%40googlegroups.com.

Re: [sage-devel] Re: Python version support for SymPy/Sage

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 12:37:38 PM UTC-7 Oscar Benjamin wrote:

On Wed, 5 Jun 2024 at 19:32, Matthias Koeppe  wrote: 
> On Wednesday, June 5, 2024 at 5:31:30 AM UTC-7 Oscar Benjamin wrote: 
> > Following those specifications in coordination with other scientific 
> > Python packages would mean dropping both 3.8 and 3.9 now and then also 
> > dropping 3.10 in a few months. I don't propose to do this right now 
> > with the SymPy 1.13 release but once 1.13 is released I want to drop 
> > at least 3.8 if not also 3.9 
> 
> No problem; by the time that SymPy 1.14 is ready, we'll likely have 
dropped 3.9. 

Okay great. We'll drop both Python 3.8 and 3.9 then after sympy 1.13 
is released. 

You haven't mentioned 3.10. I infer from this that it currently seems 
problematic for Sage to drop support for 3.10 in a few months (and 
that it might be awkward if SymPy did so).


Sorry for missing your question regarding 3.10.

That's really where it becomes relevant what "following the SPEC 0/NEP 29 
schedule" really means for a project. 
(I have a bit of analysis on the actual practices of projects in our wiki 
page; it varies quite a bit, and more variation comes in simply from the 
coarse granularity of release schedules.)
If you drop Python 3.10 in preparation for a SymPy 1.14 release at the end 
of 2024, I think that you are likely going to be early in the pack with 
this, and in Sage we might be holding off on the update for a while -- 
unless there's a SymPy feature that is of particular importance for us, or 
we notice that Python 3.10 does not matter regarding system Python support 
on distributions, or NumPy/SciPy releases that drop the support are ready 
also, etc.

Matthias

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/9a3cd9c6-978f-4acf-bd8d-c15c0c6b7e71n%40googlegroups.com.

Re: [sage-devel] Re: Python version support for SymPy/Sage

[Oscar Benjamin]

Thanks Matthias.

On Wed, 5 Jun 2024 at 19:32, Matthias Koeppe  wrote:
> On Wednesday, June 5, 2024 at 5:31:30 AM UTC-7 Oscar Benjamin wrote:
>
> > Ordinarily SymPy would have dropped support for Python 3.8 by now
> > anyway regardless of SPEC 0 or NEP 29. I can't remember where this was
> > discussed but I think that the reason Python 3.8 is still supported is
> > just because we thought it was needed by Sage.
>
> We dropped Python 3.8 in the Sage 10.2 cycle (August 2023).

Okay, well we'll keep it in SymPy 1.13 for now. The release has
already been delayed long enough so I don't want to change anything at
this point.

> > Following those specifications in coordination with other scientific
> > Python packages would mean dropping both 3.8 and 3.9 now and then also
> > dropping 3.10 in a few months. I don't propose to do this right now
> > with the SymPy 1.13 release but once 1.13 is released I want to drop
> > at least 3.8 if not also 3.9
>
> No problem; by the time that SymPy 1.14 is ready, we'll likely have dropped 
> 3.9.

Okay great. We'll drop both Python 3.8 and 3.9 then after sympy 1.13
is released.

You haven't mentioned 3.10. I infer from this that it currently seems
problematic for Sage to drop support for 3.10 in a few months (and
that it might be awkward if SymPy did so).

--
Oscar

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAHVvXxSipFTSK7yGqBYeUw1aRX1NbEZqdM1WKHtBA5hSSgqFdg%40mail.gmail.com.

[sage-devel] Re: Current status of possibility of integrating libraries written in Rust into Sage

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 9:46:05 AM UTC-7 Dima Pasechnik wrote:

On Monday, June 3, 2024 at 8:16:30 PM UTC+1 Matthias Koeppe wrote:

Unlikely that we would add a package to the Sage distribution that builds a 
Rust library from source. 

Not so long ago we added support for installing Python packages from 
platform-independent wheels. We did this to sidestep the concern of 
shipping more and more of Javascript (Node.js) infrastructure that is 
needed for building JupyterLab components.

Likewise, we will soon add support for installing Python packages from 
platform-dependent wheels. This is needed for updating some Jupyter 
components that have started to use Rust (https://github.com/crate-py/rpds, 
a dependency of jsonschema).


Could you be more specific on this, please? [...] Where would these wheels 
come from? From PyPI?



(In the Sage distribution, they won't be "pip" packages -- which are an 
underdeveloped mechanism in the build system of the Sage distribution -- 
but rather a variant of the existing "wheel" packages.)

As the upstream developer, you would publish binary wheels to PyPI, likely 
in the same way as is done in 
https://github.com/crate-py/rpds/blob/main/.github/workflows/CI.yml


- Like I wrote below the fold: The upstream project makes the binary wheels 
and uploads them to PyPI as usual.
- When we make a package update in the Sage distribution, we download the 
wheels from PyPI (the upstream_url of the package) and upload them to the 
mirrors (and to the GitHub Release Assets).
- When the package is installed, the Sage distribution downloads the wheels 
from GitHub Release Assets, falling back to mirrors, then falling back to 
the upstream_url.

In other words, just like what we do for any other normal/wheel package in 
the Sage distribution. 

The only technical change: There's more than one "tarball" associated with 
these packages.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/4b9c20f2-3cf2-4fdd-9606-3e446ecbd719n%40googlegroups.com.

Re: [sage-devel] VOTE: Follow NEP 29: Recommended Python version

[Dima Pasechnik]

Actually, it doesn't seem that the votes have ever been counted here.

In view of SymPy willing to follow NEP 29 (cf. 
https://groups.google.com/g/sage-devel/c/0BPkiiWYrIU/m/c1vlJoMGEwAJ), it 
seems natural to join this trend and follow NEP 29.

On Tuesday, May 30, 2023 at 10:35:21 AM UTC+1 G. M.-S. wrote:

>
> My vote is empty, for the following reasons:
> —I think the question asked is not clear enough, as per reactions.
> —My "dream" is having an easy to install recent version of SageMath for 
> everybody wishing to do mathematics with it.  Currently this is only the 
> case for macOS and perhaps some flavours of Linux, AFAICT (my experience 
> with my students and colleagues is not very good, especially under Windows).
>
> Guillermo
>
> On Fri, 26 May 2023 at 12:12, Tobias Diez  wrote:
>
>> Dear Sage developers,
>>
>> the NumPy enhancement proposal 29: "Recommend Python and Numpy version 
>> support as a community policy standard" (available at 
>> https://numpy.org/neps/nep-0029-deprecation_policy.html) specifies when 
>> it's okay to drop support for old Python version. 
>>
>> Namely, a release should support "all minor versions of Python released 
>> 42 months prior to the project, and at minimum the two latest minor 
>> versions. ". In particular, this means:
>> - Currently, Sage should support > 3.8.
>> - On Apr 05, 2024 we should drop support for Python 3.9 (initially 
>> released on Oct 05, 2020)
>>
>> Based on previous discussions on this topic (
>> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/2sTiwdKPBQAJ, 
>> https://github.com/sagemath/sage/issues/30384, 
>> https://github.com/sagemath/sage/pull/35403), I'm calling for a vote on 
>> adapting the Python version support of NEP 29 in Sage. Voting ends at the 
>> 7th June,  AoE. Please use this thread only for sending votes, to make it 
>> easier to count them afterward; and use the thread 
>> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/2sTiwdKPBQAJ for 
>> discussion.
>>
>> *Summary *of the points brought forward in the discussions linked above
>> 1. The current practice in Sage is to evaluate whether to increase the 
>> minimum version of Python supported at the beginning of each release cycle. 
>> With regard to such a practice, the NEP 29 documents remarks "As there is 
>> no objective threshold to when the minimum version should be dropped, it is 
>> easy for these version support discussions to devolve into bike shedding 
>> 
>>  
>> and acrimony." Sadly, an example of this can be found in the current 
>> discussion of dropping Python 3.8 support in 
>> https://github.com/sagemath/sage/pull/35404 with emotions running so 
>> high that sage-abuse had to step in. Adopting a version policy would 
>> prevent such discussions. On the other hand, by following a given policy, 
>> we would loose some flexibility.
>> 2. The main idea of NEP 29 is to have a community-wide standard. It is 
>> followed by many scientific packages such as Scipy, Matplotlib, IPython, 
>> Jupyter, Pandas, scikit, astropy, cuda, cirq, jax, pytorch among others. The 
>> adoption of NEP 29 will harmonize Sage's deprecation policy with these 
>> other major libraries. 
>> 3. The NEP 29 drop schedule is much faster than the EOL schedule of 
>> Python itself. Python 3.8 is supported until 2024-10, but NEP 29 already 
>> drops it 2023-04. However, adhering to the EOL schedule would prevent us to 
>> updating these packages that follow NEP 29.
>> 4. The NEP 29 schedule is about one release cycle faster than the 
>> previous drops (e.g. Python 3.7 support was dropped in Sage 9.7 while 
>> according to NEP 29 it would have been Sage 9.6).
>> 5. The faster drop schedule will free developer resources (less systems 
>> to test) and potentially increase developer productivity as it allows us to 
>> use newer language features.
>> 6. The faster drop schedule might be inconvenient for users who rely on 
>> older Python versions. To some extend this is remedied by our python 
>> install package, and relatively straightforward upgrade paths on most 
>> system. One should also note that users relying on other scientific python 
>> packages are likely forced to upgrade anyway, since these other packages 
>> likely follow NEP 29.
>> 7. The faster drop schedule would force users to upgrade to newer Python 
>> versions and thereby profit from fewer bugs and security issues. It is 
>> however questionable if Sage should play this educator role.
>> 8. One of the main goals of NEP 29 is to improve downstream project 
>> planning by having a community-wide standard. This is currently not very 
>> relevant for us as Sage is currently upstream of nothing except for some 
>> user packages. With the modularization effort, this may change in the 
>> future.
>> 9. There are not many other documented policies. As said above, most 
>> scientific python projects follow NEP 29. Most projects in web developm

[sage-devel] Re: Python version support for SymPy/Sage

[Matthias Koeppe]

On Wednesday, June 5, 2024 at 5:31:30 AM UTC-7 Oscar Benjamin wrote:

My question here is: would it be problematic for Sage if SymPy were to 
follow SPEC 0/NEP 29 which would mean dropping support for older 
Python versions more quickly?


Quick answer: Probably it would not be very problematic on our side because 
many of other upstream packages of Sage (NumPy, SciPy, NetworkX, ...) 
already follow that schedule. But it could mean that we sometimes hold off 
on updating SymPy to the latest version for a little while, like we 
currently do with other packages.

Example: SPEC 0/NEP 29 dropped Python 3.9 earlier this year, and NetworkX 
has already pushed out a version 3.3 which no longer supports Python 3.9.
But that's an outlier from our perspective; the released versions of NumPy 
and SciPy (including the upcoming NumPy 2.0) still support Python 3.9.
So without major cost to us, we are able to keep Python 3.9 a bit longer.
 

We are about to release SymPy 1.13 which will support Python 3.8 to 
3.13.


Great, I hope we can still squeeze it in to our upcoming Sage 10.4 release!
 

It has been asked on the SymPy mailing list whether SymPy could 
adopt SPEC 0 which is basically the same as NEP 29: 

https://groups.google.com/g/sympy/c/VNz2xJ1Sywo/m/0H1-KmaJAgAJ?utm_medium=email&utm_source=footer
 
https://scientific-python.org/specs/spec-/ 

There are older discussions about this as well: 

https://github.com/sympy/sympy/issues/21884 

This relates to previous discussions on this list about Sage following NEP 
29: 

https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/Ap_1pHlsBQAJ?utm_medium=email&utm_source=footer
 
https://groups.google.com/g/sage-devel/c/3Zoq0CNE1hE/m/m40v2e_cBwAJ


We also have a wiki page about NEP 29 with many collected facts and 
discussion points:
https://github.com/sagemath/sage/wiki/NEP-29:-Python-version-strategy
(last edited by me in October 2023; should update for SPEC 0 etc.)

Ordinarily SymPy would have dropped support for Python 3.8 by now 
anyway regardless of SPEC 0 or NEP 29. I can't remember where this was 
discussed but I think that the reason Python 3.8 is still supported is 
just because we thought it was needed by Sage.


We dropped Python 3.8 in the Sage 10.2 cycle (August 2023).
 

Following those specifications in coordination with other scientific 
Python packages would mean dropping both 3.8 and 3.9 now and then also 
dropping 3.10 in a few months. I don't propose to do this right now 
with the SymPy 1.13 release but once 1.13 is released I want to drop 
at least 3.8 if not also 3.9 


No problem; by the time that SymPy 1.14 is ready, we'll likely have dropped 
3.9.
 

It is not particularly difficult for SymPy itself to support a wide 
range of Python versions


Likewise for the Sage library. 

Personally I am in favour of SymPy coordinating with the rest of the 
scientific Python ecosystem by adopting SPEC 0/NEP 29 so that there is 
a clearly defined policy that is understood and depended on by 
everything downstream. Old versions of SymPy will continue to work 
with old versions of Python so people who like packaging old versions 
of everything (e.g. Linux distros) can do so.


Yes, this makes sense and is, of course, very much the point of SPEC 0/NEP 
29.

As Sage currently is "downstream of everything" and "upstream of nothing", 
we have effectively been following NEP 29 + an additive constant. See 
https://github.com/sagemath/sage/wiki/NEP-29:-Python-version-strategy#schedule-with-comparison-to-nep-29
(I hope that this will change in the longer term with the modularization of 
the Sage library, but currently it is not a relevant factor.)

[...Remarks on pyenv, uv, vscode, conda elided...]

It is not clear to me what the benefit is of supporting old Python 
versions especially for Sage: building/installing Sage is a vastly 
bigger issue than building/installing a new Python version.


One important factor regarding on our side has really been cultural rather 
than technical: To counter the perception that "Sage is so isolated from 
the rest of the world, it even brings its own Python", we have gone to 
great lengths to support the use of (venvs over) the system-provided 
Python; even though that goes counter to the common mantra among Python 
devs that "system Python is not for you". Because of this, the available 
(and default) Python versions on common distributions have been a factor in 
our version support strategy.


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/14718742-f2d6-4942-9502-049508020210n%40googlegroups.com.

[sage-devel] Re: Current status of possibility of integrating libraries written in Rust into Sage

[Dima Pasechnik]

On Monday, June 3, 2024 at 8:16:30 PM UTC+1 Matthias Koeppe wrote:

Unlikely that we would add a package to the Sage distribution that builds a 
Rust library from source. 

Not so long ago we added support for installing Python packages from 
platform-independent wheels. We did this to sidestep the concern of 
shipping more and more of Javascript (Node.js) infrastructure that is 
needed for building JupyterLab components.

Likewise, we will soon add support for installing Python packages from 
platform-dependent wheels. This is needed for updating some Jupyter 
components that have started to use Rust (https://github.com/crate-py/rpds, 
a dependency of jsonschema).


Could you be more specific on this, please? There are different way this 
can be interpreted. Where would these wheels come from? From PyPI?

 

 
(In the Sage distribution, they won't be "pip" packages -- which are an 
underdeveloped mechanism in the build system of the Sage distribution -- 
but rather a variant of the existing "wheel" packages.)

As the upstream developer, you would publish binary wheels to PyPI, likely 
in the same way as is done in 
https://github.com/crate-py/rpds/blob/main/.github/workflows/CI.yml



On Saturday, June 1, 2024 at 4:46:14 AM UTC-7 Jing Guo wrote:

Dear all,

Recently we released a library for counting graph homomorphisms [0] in 
Sage. Due to performance and parallelism reasons, I was considering the 
possibility of re-writing some/all of the algorithms in Rust. I found a 
Rust library called `pyo3` [1] seems to be good for Python-Rust interop.

The latest post I could found is from one year ago [2], so I was wondering 
what the current status of possibility of integrating libraries written in 
Rust into Sage? Is the recommended approach still to make it an optional 
package? For instance, what I have something in mind is like `addcombq` [3] 
-- a Rust library which is callable from Sage/Python.

Many thanks!

Jing

[0]: https://github.com/guojing0/count-graph-homs
[1]: https://github.com/PyO3/pyo3
[2]: https://groups.google.com/g/sage-devel/c/OpBIfmbOlPA/m/hFKTdyE4CgAJ
[3]: https://github.com/Torrencem/addcombq

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/dddb2979-c656-4668-bc6b-43fe00eb5b95n%40googlegroups.com.

[sage-devel] Python version support for SymPy/Sage

[Oscar Benjamin]

Hi all,

My question here is: would it be problematic for Sage if SymPy were to
follow SPEC 0/NEP 29 which would mean dropping support for older
Python versions more quickly?

We are about to release SymPy 1.13 which will support Python 3.8 to
3.13. It has been asked on the SymPy mailing list whether SymPy could
adopt SPEC 0 which is basically the same as NEP 29:

https://groups.google.com/g/sympy/c/VNz2xJ1Sywo/m/0H1-KmaJAgAJ?utm_medium=email&utm_source=footer

https://scientific-python.org/specs/spec-/

There are older discussions about this as well:

https://github.com/sympy/sympy/issues/21884

This relates to previous discussions on this list about Sage following NEP 29:

https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/Ap_1pHlsBQAJ?utm_medium=email&utm_source=footer
https://groups.google.com/g/sage-devel/c/3Zoq0CNE1hE/m/m40v2e_cBwAJ

Ordinarily SymPy would have dropped support for Python 3.8 by now
anyway regardless of SPEC 0 or NEP 29. I can't remember where this was
discussed but I think that the reason Python 3.8 is still supported is
just because we thought it was needed by Sage.

Following those specifications in coordination with other scientific
Python packages would mean dropping both 3.8 and 3.9 now and then also
dropping 3.10 in a few months. I don't propose to do this right now
with the SymPy 1.13 release but once 1.13 is released I want to drop
at least 3.8 if not also 3.9 and then ideally we would have a clear
policy for this going forwards.

It is not particularly difficult for SymPy itself to support a wide
range of Python versions but it creates potential problems for other
packages that depend on SymPy. For example when SymPy 1.13 is released
(soon) any packages that depend on SymPy potentially need to put out
updates for all of the same Python versions that the SymPy release
claims support for. If the SymPy 1.13 update breaks something then
pushing it out to as many Python versions as possible is like
spreading the risk of breakage as far as possible. We need to be
careful about this because besides direct end users or Sage there are
also very widely used packages like pytorch that depend on SymPy which
has forced us into rushing out fix releases recently:
https://github.com/sympy/sympy/issues/26273

Personally I am in favour of SymPy coordinating with the rest of the
scientific Python ecosystem by adopting SPEC 0/NEP 29 so that there is
a clearly defined policy that is understood and depended on by
everything downstream. Old versions of SymPy will continue to work
with old versions of Python so people who like packaging old versions
of everything (e.g. Linux distros) can do so.

People who want to use the latest versions of Python packages should
also use recent versions of Python. My recommendation to a user who
just wants to use Python would generally be to install the second most
recent version of Python i.e. currently Python 3.11 but soon 3.12. I
would not recommend anyone to install Python 3.8 right now.

For now I use pyenv to install different Python versions but hopefully
in future it will become easier to install a particular Python version
from binary on any OS. I expect that soon uv will be able to do this
using the PyBI binaries at which point I will probably switch to using
uv in replacement of both pyenv and pip. I expect that in future
editors like vscode will be able to handle installing a particular
Python version and all the packages automatically for users who will
not even need to run uv/pip etc directly.

Conda already makes it easy to install particular Python versions for
people using conda. Windows/MacOS users will typically download the
newest Python version from the python.org download page. Linux distros
usually have ways of installing newer Python versions from repos and
it's not hard to build Python on Linux anyway.

It is not clear to me what the benefit is of supporting old Python
versions especially for Sage: building/installing Sage is a vastly
bigger issue than building/installing a new Python version.

My question then is would SymPy adopting SPEC 0/NEP 29 be a problem for Sage?

If SymPy 1.14 is released later this year and drops support for Python
3.8, 3.9 and 3.10 would that be a problem?

Oscar

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAHVvXxR0FpHr25XrZezE4sKFjjum3mO4xMF3PARc3UEWtBAj_g%40mail.gmail.com.