Re: [arch-dev-public] Add active Python versions to the repos
On Thu, 2020-11-26 at 21:21 +0100, Jelle van der Waa wrote: > On 21/11/2020 15:34, Filipe Laíns via arch-dev-public wrote: > > Hi all, > > > > I want to propose adding all active Python versions to [community], not > > just the latest one. This would only entail adding the interpreter > > itself, no other packages. > > > > Having access to interpreters for older active versions is really > > helpful for Python developers. This allows them to easily run test > > suites against older versions. It is very common for developers to > > maintain software against a couple major releases. Tools like tox or > > nox are able to automate testing against multiple Python versions, just > > needing the interpreter. > > > > The current active Python releases are: > > - 3.9 > > - 3.8 > > - 3.7 > > - 3.6 > > - 2.7 > > > > The list can be found here[1]. > > > > So, I propose introducing 2 new packages: > > - python3.7 > > - python3.6 > > > > And when we update the python package to 3.9: > > - python3.8 > > I am not super in favor of providing an old version, as we strive to > provide the latest and greatest. However some things come to mind: We would still provide the latest and greatest :P > * Does the Python foundation still maintain older Python versions and do > they provide proper security updates? It depends on the version. Python 3.6 and 3.7 are in security maintenance status. I am okay limiting my proposal to interpreters which are getting security updates, which at the time being doesn't change anything regarding the versions affected. It is also not clear in the proposal, but I have mentioned in the replies. I only want the interpreters there as long as they do not give us any issues. If a interpreter starts needing older dependencies or otherwise becomes a maintenance burden, it would be removed. > * This seems to be purely for Development right? Users would use this > Python version to bootstrap a virtualenv I guess? The target is development, yes. You are correct, users, or tools, could use the interpreter to bootstrap a virtual environment using either virtualenv, which supports using external interpreters, or the builtin venv module. > > Greetings, > > Jelle > Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On 21/11/2020 15:34, Filipe Laíns via arch-dev-public wrote: Hi all, I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages. Having access to interpreters for older active versions is really helpful for Python developers. This allows them to easily run test suites against older versions. It is very common for developers to maintain software against a couple major releases. Tools like tox or nox are able to automate testing against multiple Python versions, just needing the interpreter. The current active Python releases are: - 3.9 - 3.8 - 3.7 - 3.6 - 2.7 The list can be found here[1]. So, I propose introducing 2 new packages: - python3.7 - python3.6 And when we update the python package to 3.9: - python3.8 I am not super in favor of providing an old version, as we strive to provide the latest and greatest. However some things come to mind: * Does the Python foundation still maintain older Python versions and do they provide proper security updates? * This seems to be purely for Development right? Users would use this Python version to bootstrap a virtualenv I guess? Greetings, Jelle OpenPGP_signature Description: OpenPGP digital signature
Re: [arch-dev-public] Add active Python versions to the repos
Remember, write the angry email first, wait a half hour, then re-write. Please keep the IRC flamewars to IRC where some of us can avoid it without losing contact with important discussion. :) signature.asc Description: PGP signature
Re: [arch-dev-public] Add active Python versions to the repos
On Sun, 2020-11-22 at 11:45 -0500, Eli Schwartz via arch-dev-public wrote: > On 11/22/20 10:05 AM, Filipe Laíns wrote: > > On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote: > > > Your analysis is correct, it is indeed hell. I'm not sure why that is an > > > argument in favor of doing even more of it though. > > > > > > Now, if you were proposing to get rid of some of this, I could get > > > behind that. > > > > It was not an analysis, "hell" was an used here as an idiom. > > https://i.imgur.com/axJmn.gif > > You were unintentionally accurate. > > > > > even Python. > > > > > > I don't know if maybe you've missed the fact that we've spent like a > > > year now aggressively upgrading or removing leaf packages depending on > > > python2 in search of that glorious Promised Future where we can finally > > > remove the python2 package itself? > > > > I did not miss that. I absolutely want to see the python2 modules removed, > > not > > _necessarily_ the python2 interpreter. > > Now I'm starting to see part of the reason why people are having a > difference of opinions here. > > I think I probably speak for a bunch of people when I say I'd like to > see the interpreter dropped due to not being needed by any packages > *and* being an end of life interpreter. The interpreter is only useful > inasmuch as people use it to run software, so if we don't have any... > > I expect by the time we finally remove all reverse dependencies for it, > the gap between the python2 EOL and its removal from the distro will be > quite a bit biger than it currently is. > > > > And we're doing that even for a major incompatible release that most > > > people argue is actually a different language. > > > > > > I'm not sure how this suddenly became an argument to package a ton of > > > point releases that aren't even incompatible. > > > > Except they are... Python does not follow semver, only patch releases are > > supposed to be compatible. They do try to keep breakage to a minimum, but it > > definitely exists. One very obvious example is the introduction of "async" > > and > > "await" as reserved keywords. There are a bunch of other backwards > > incompatible > > changes, so yes, they are incompatible releases, to the point you could call > > them different languages. > > This is the first time I ever heard anyone describe "backwards > incompatible changes means it is now a different programming language". You literally mentioned that in your reply. Though I assume you meant more significant change than what is being discussed. If the languages semantics are different one could argue it is actually a different language. Anyway, this does not really matter for the issue being discussed. > I had backwards incompatible changes in a patchlevel bugfix in the > python 3.8.x release line causing immediate traceback and SystemExit on > a program's startup, does that mean every bugfix is now a different > programming language? > > (I mean this quite literally. Python bugfixed a function not working as > expected under certain situations. The program it broke was working > around the failure by manually doing $thing, the traceback happened when > you try to do $thing twice.) > > > > > I don't think having people opening bugs because they are > > > > deliberately using an older version of Python is a big problem. It > > > > hasn't been for any of the other languages, I don't think it will be > > > > here. > > > > I think this is an hypothetical that doesn't really materialize into > > > > reality. > > > > > > You're proposing something far beyond the scope of what we do for other > > > languages, and ignoring that for other languages we only do it if driven > > > to in desperation because another official repository package depends on > > > it. > > > > > > We don't introduce leaf packages of ancient versions of currently > > > packaged interpreters just for fun. I don't even think anyone else does > > > that either. > > > > That was not of the point of me bringing up other languages. The whole point > > of > > me bringing up other languages was to show that Andreas' concern is unlikely > > to > > materialize into reality. > > You think no one reports bugs for python-* packages that we should > provide the python2 version? I promise you, you're wrong. I've closed > those bugs, so I know they exist. That is not "users raising bugs that something doesn't work with an older version of python", and is not what I argued against. > You specifically brought up Java and Javascript, languages where each > application is forced to vendor their entire dependency list rather than > using system modules. For this reason, people don't report the kind of > bugs Andreas is concerned about for Java/nodejs... They are not necessarily forced, but whatever, just ignore that. > So I assumed you could not possibly be talking about Andreas' concern, > and I responded somewhat tangentially myself. We can, of course,
Re: [arch-dev-public] Add active Python versions to the repos
On 11/22/20 10:05 AM, Filipe Laíns wrote: > On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote: >> Your analysis is correct, it is indeed hell. I'm not sure why that is an >> argument in favor of doing even more of it though. >> >> Now, if you were proposing to get rid of some of this, I could get >> behind that. > > It was not an analysis, "hell" was an used here as an idiom. https://i.imgur.com/axJmn.gif You were unintentionally accurate. >>> even Python. >> >> I don't know if maybe you've missed the fact that we've spent like a >> year now aggressively upgrading or removing leaf packages depending on >> python2 in search of that glorious Promised Future where we can finally >> remove the python2 package itself? > > I did not miss that. I absolutely want to see the python2 modules removed, not > _necessarily_ the python2 interpreter. Now I'm starting to see part of the reason why people are having a difference of opinions here. I think I probably speak for a bunch of people when I say I'd like to see the interpreter dropped due to not being needed by any packages *and* being an end of life interpreter. The interpreter is only useful inasmuch as people use it to run software, so if we don't have any... I expect by the time we finally remove all reverse dependencies for it, the gap between the python2 EOL and its removal from the distro will be quite a bit biger than it currently is. >> And we're doing that even for a major incompatible release that most >> people argue is actually a different language. >> >> I'm not sure how this suddenly became an argument to package a ton of >> point releases that aren't even incompatible. > > Except they are... Python does not follow semver, only patch releases are > supposed to be compatible. They do try to keep breakage to a minimum, but it > definitely exists. One very obvious example is the introduction of "async" and > "await" as reserved keywords. There are a bunch of other backwards > incompatible > changes, so yes, they are incompatible releases, to the point you could call > them different languages. This is the first time I ever heard anyone describe "backwards incompatible changes means it is now a different programming language". I had backwards incompatible changes in a patchlevel bugfix in the python 3.8.x release line causing immediate traceback and SystemExit on a program's startup, does that mean every bugfix is now a different programming language? (I mean this quite literally. Python bugfixed a function not working as expected under certain situations. The program it broke was working around the failure by manually doing $thing, the traceback happened when you try to do $thing twice.) >>> I don't think having people opening bugs because they are >>> deliberately using an older version of Python is a big problem. It >>> hasn't been for any of the other languages, I don't think it will be >>> here. >>> I think this is an hypothetical that doesn't really materialize into >>> reality. >> >> You're proposing something far beyond the scope of what we do for other >> languages, and ignoring that for other languages we only do it if driven >> to in desperation because another official repository package depends on it. >> >> We don't introduce leaf packages of ancient versions of currently >> packaged interpreters just for fun. I don't even think anyone else does >> that either. > > That was not of the point of me bringing up other languages. The whole point > of > me bringing up other languages was to show that Andreas' concern is unlikely > to > materialize into reality. You think no one reports bugs for python-* packages that we should provide the python2 version? I promise you, you're wrong. I've closed those bugs, so I know they exist. You specifically brought up Java and Javascript, languages where each application is forced to vendor their entire dependency list rather than using system modules. For this reason, people don't report the kind of bugs Andreas is concerned about for Java/nodejs... So I assumed you could not possibly be talking about Andreas' concern, and I responded somewhat tangentially myself. We can, of course, talk about Andreas' concern, but given I've now submitted the authoritative facts on the matter, I'm not sure what else there is to say. It does indisputably happen right now. The logical inference is, it will happen *more*, if we add more runtimes for people to submit bugtracker tickets for. >> You want to package old versions of the python3 interpreter "but not >> modules", because "people would like to use it for tox". > > People can use it tox or to create virtual environments directly. The main > point > of the proposal is that people have use for the old interpreters and I would > like to provide them. The main point of my objection is that their use isn't to run software, it's to regression test it -- something more suited to Continuous Integration which already targets a
Re: [arch-dev-public] Add active Python versions to the repos
On Sun, 2020-11-22 at 09:43 -0600, Doug Newgard via arch-dev-public wrote: > On Sun, 22 Nov 2020 15:32:22 + > Filipe Laíns via arch-dev-public wrote: > > > On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote: > > > Why did you start the discussion if you plan on ignoring other people's > > > opinions anyway? > > > > > > Doug > > > > I am not ignoring Andreas' or Eli's opinion, I understand that they don't > > want > > these packages in the repos. What I am asking is why they don't want them > > there? > > I don't appreciate you trimming the paragraph I was replying to. I makes my > comments look completely different without context. Sorry, I only replied to the specific reply because my email client is messing up the format and I have not fixed it yet. I did not meant to take it out of context. > The paragraph specifically said that you see no reason to block anything as > long as someone is willing to maintain it. That means that other's opinions > in this disussion don't matter, unless I misunderstood? > > Doug That was not meant that way. Similarly to your reply, the context matters. The comment was meant as "If this helps people and does not impact you, why would you block it?", I said this because Eli seems hell bent on not having the packages in the repos. Sorry if it came out the wrong way. Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sun, 22 Nov 2020 15:32:22 + Filipe Laíns via arch-dev-public wrote: > On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote: > > Why did you start the discussion if you plan on ignoring other people's > > opinions anyway? > > > > Doug > > I am not ignoring Andreas' or Eli's opinion, I understand that they don't want > these packages in the repos. What I am asking is why they don't want them > there? I don't appreciate you trimming the paragraph I was replying to. I makes my comments look completely different without context. The paragraph specifically said that you see no reason to block anything as long as someone is willing to maintain it. That means that other's opinions in this disussion don't matter, unless I misunderstood? Doug
Re: [arch-dev-public] Add active Python versions to the repos
On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote: > Why did you start the discussion if you plan on ignoring other people's > opinions anyway? > > Doug I am not ignoring Andreas' or Eli's opinion, I understand that they don't want these packages in the repos. What I am asking is why they don't want them there? I have addressed the concern Andreas raised. I am not sure if satisfactorily to him as he has not replied yet. Eli, so far, has presented me with the current status quo, stating this is not we currently do. Which was never what I was arguing, that's why I have started this thread, in hope of changing that. From his reply, the only part I see as actual reasoning against the proposed change is this: > I don't see any reason why [community] needs to grow to support the needs of exclusively Continuous Integration testsuite matrixes. Which I have addressed. Please let me know if I've missed any of his points. Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sun, 22 Nov 2020 15:05:04 + Filipe Laíns via arch-dev-public wrote: > As long as there is someone willing to maintain the packages, why would we > block > it? I doesn't really have any effect on the rest of the people. If the person > start having trouble maintaining them, they can be dropped. It doesn't affect > you and helps other people. > > Cheers, > Filipe Laíns Why did you start the discussion if you plan on ignoring other people's opinions anyway? Doug
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
> Your analysis is correct, it is indeed hell. I'm not sure why that is an
> argument in favor of doing even more of it though.
>
> Now, if you were proposing to get rid of some of this, I could get
> behind that.
It was not an analysis, "hell" was an used here as an idiom.
> > even Python.
>
> I don't know if maybe you've missed the fact that we've spent like a
> year now aggressively upgrading or removing leaf packages depending on
> python2 in search of that glorious Promised Future where we can finally
> remove the python2 package itself?
I did not miss that. I absolutely want to see the python2 modules removed, not
_necessarily_ the python2 interpreter.
> And we're doing that even for a major incompatible release that most
> people argue is actually a different language.
>
> I'm not sure how this suddenly became an argument to package a ton of
> point releases that aren't even incompatible.
Except they are... Python does not follow semver, only patch releases are
supposed to be compatible. They do try to keep breakage to a minimum, but it
definitely exists. One very obvious example is the introduction of "async" and
"await" as reserved keywords. There are a bunch of other backwards incompatible
changes, so yes, they are incompatible releases, to the point you could call
them different languages.
> > I don't think having people opening bugs because they are
> > deliberately using an older version of Python is a big problem. It
> > hasn't been for any of the other languages, I don't think it will be
> > here.
> > I think this is an hypothetical that doesn't really materialize into
> > reality.
>
> You're proposing something far beyond the scope of what we do for other
> languages, and ignoring that for other languages we only do it if driven
> to in desperation because another official repository package depends on it.
>
> We don't introduce leaf packages of ancient versions of currently
> packaged interpreters just for fun. I don't even think anyone else does
> that either.
That was not of the point of me bringing up other languages. The whole point of
me bringing up other languages was to show that Andreas' concern is unlikely to
materialize into reality.
> > > It's sad enough we still have python2 and gtk2 around. To have gcc9
> > > around and other duplicates is not what I want to see growing in
> > > Arch.
> >
> > What you call sad I call a bad UX. Why do we need to force users to
> > compile active releases of the Python interpreters themselves, which
> > can take a long time if they are building with optimization, or to
> > resort to pre-built repos with much lower security standards than us,
> > when there are people willing to maintain them?
> >
> > I can't understand how it's sad to help out users by not forcing them
> > to resort the sort of things I mentioned above, as long as we are not
> > struggling to do so. I like helping people, that's why I am a packager,
> > that is the opposite of sad for me, so I really can't understand this.
>
> I have... no idea what the problem is supposed to be here? I'm
> desperately trying to understand what the actual point of your proposal is.
>
> You want to package old versions of the python3 interpreter "but not
> modules", because "people would like to use it for tox".
People can use it tox or to create virtual environments directly. The main point
of the proposal is that people have use for the old interpreters and I would
like to provide them.
> Old versions of stuff that people need for who knows what weird reason
> is *the textbook reason* why the AUR exists. How long it takes to
> compile stuff in the AUR is not our problem, and we don't have a
> rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a
> lot of diverse compilers "to test that it still works on really old
> compilers".
AFAIK, the textbook reason why the AUR exists is to provide software that isn't
in the official repos. It being older versions should be completely incidental,
but not the "textbook reason".
> Likewise, if people want to test support for old versions of python3,
> they have options which don't involve uploading trash into [community].
I like how you refereed to the currently active Python releases as "trash", will
keep that in mind.
> - Do what normal people do, and build AUR packages they need
> - Ask FFY00 to provide signed repos
> - Use travis CI, which doesn't use distro packages but provides a
> diverse set of hand-compiled python versions, which is apparently some
> kind of gold standard for doing regression testing on lots of
> different python releases ;)
> - Use pyenv
>
> Apparently none of these are viable options in your mind, and the most
> Arch-like of them, option #1 "use the AUR" you believe to be problematic
> because it takes a long time to build with optimizations... even though
> you yourself said those packages don't enable
Re: [arch-dev-public] Add active Python versions to the repos
On 11/21/20 11:58 AM, Filipe Laíns via arch-dev-public wrote: > pyenv forces users to compile the Python interpreter themselves, which > can take a long time with --enable-optimizations. > None of the user repos available builds with optimizations, or has > signed packages AFAIK. Of course they could in the future, but I think > having the packages in the repos is much better in terms of security > and usability. I run one of the user which provides these packages and > I don't see myself fixing any of the issues I pointed out due to > technical limitations. Incidentally... if you don't see yourself fixing the --enable-optimizations or gpg --detach-sign problem for your own builds, I'm morbidly curious as to the technical limitations and would like to know why you think these technical limitations will vanish once you're uploading them to repos.archlinux.org specifically. -- Eli Schwartz Bug Wrangler and Trusted User OpenPGP_0x84818A6819AF4A9B.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: [arch-dev-public] Add active Python versions to the repos
On 11/21/20 11:59 AM, Filipe Laíns via arch-dev-public wrote:
> On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public
> wrote:
>> Am Sat, 21 Nov 2020 14:34:24 +
>> schrieb Filipe Laíns via arch-dev-public
>> :
>>
>>
>>> Does anyone have any big issue with this? What are your thoughts?
>>>
>>> [1] https://www.python.org/downloads/
>>>
>>> Cheers,
>>> Filipe Laíns
>>
>> -1
>>
>> Arch is yours. Whoever needs more and older releases on the system -
>> just do it yourself! In the past we said "use abs and AUR - Arch is
>> the base to make it your own".
>
> This argument can be used to deny adding any package to the repos. You
> need this library, tool, etc.? Just add it yourself.
>
> Why are we packaging software that is used by far less people but we
> can't package these Python interpreters which are being actively missed
> by people?
>
>> I don't want to see users raising bugs that something doesn't work
>> with an older version of python. And I don't want to see these
>> requests
>> pop up every now and then to add even more stuff in different
>> versions.
>
> We already have multiple versions of Java, Ruby, Javascript, etc. hell,
Your analysis is correct, it is indeed hell. I'm not sure why that is an
argument in favor of doing even more of it though.
Now, if you were proposing to get rid of some of this, I could get
behind that.
> even Python.
I don't know if maybe you've missed the fact that we've spent like a
year now aggressively upgrading or removing leaf packages depending on
python2 in search of that glorious Promised Future where we can finally
remove the python2 package itself?
And we're doing that even for a major incompatible release that most
people argue is actually a different language.
I'm not sure how this suddenly became an argument to package a ton of
point releases that aren't even incompatible.
> I don't think having people opening bugs because they are
> deliberately using an older version of Python is a big problem. It
> hasn't been for any of the other languages, I don't think it will be
> here.
> I think this is an hypothetical that doesn't really materialize into
> reality.
You're proposing something far beyond the scope of what we do for other
languages, and ignoring that for other languages we only do it if driven
to in desperation because another official repository package depends on it.
We don't introduce leaf packages of ancient versions of currently
packaged interpreters just for fun. I don't even think anyone else does
that either.
>> It's sad enough we still have python2 and gtk2 around. To have gcc9
>> around and other duplicates is not what I want to see growing in
>> Arch.
>
> What you call sad I call a bad UX. Why do we need to force users to
> compile active releases of the Python interpreters themselves, which
> can take a long time if they are building with optimization, or to
> resort to pre-built repos with much lower security standards than us,
> when there are people willing to maintain them?
>
> I can't understand how it's sad to help out users by not forcing them
> to resort the sort of things I mentioned above, as long as we are not
> struggling to do so. I like helping people, that's why I am a packager,
> that is the opposite of sad for me, so I really can't understand this.
I have... no idea what the problem is supposed to be here? I'm
desperately trying to understand what the actual point of your proposal is.
You want to package old versions of the python3 interpreter "but not
modules", because "people would like to use it for tox".
Old versions of stuff that people need for who knows what weird reason
is *the textbook reason* why the AUR exists. How long it takes to
compile stuff in the AUR is not our problem, and we don't have a
rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a
lot of diverse compilers "to test that it still works on really old
compilers".
Likewise, if people want to test support for old versions of python3,
they have options which don't involve uploading trash into [community].
- Do what normal people do, and build AUR packages they need
- Ask FFY00 to provide signed repos
- Use travis CI, which doesn't use distro packages but provides a
diverse set of hand-compiled python versions, which is apparently some
kind of gold standard for doing regression testing on lots of
different python releases ;)
- Use pyenv
Apparently none of these are viable options in your mind, and the most
Arch-like of them, option #1 "use the AUR" you believe to be problematic
because it takes a long time to build with optimizations... even though
you yourself said those packages don't enable optimizations and don't
take a long time???
(If you're using python-$old exclusively for regression tests in tox,
you don't need a super optimized version.)
>> I don't want to see our distribution transformed into another Debian.
>
> That is not what is happening.
Yeah, even Debian doesn't do this.
Re: [arch-dev-public] Add active Python versions to the repos
Whoops, my email client messed up the formatting :/ Here's the reply without the quotes: Looking at some package stats for pyenv[1], we see a usage of 3,4%. I think that is a big enough number. pyenv does have another usage, which is managing custom Python interpreters, which is mainly used in CPython development, but I'd say that only a small percentage of the users use it uniquely for that. We can also look directly at the python36 and python37 packages reported, which users might be getting from the AUR or user repos. python37 has 1.18% [2] python36 has 1.63% [3] The main reason why the packages might be required, as I explained in the original email, is for package testing. It puts every maintainer and most contributors to projects that target Python 3.6 or 3.7 in need of an interpreter. Most open source Python project target at least a few different Python versions. These metrics are not perfect by any means, but I do think they present relevant enough data to show an existent need. [1] https://pkgstats.archlinux.de/api/packages/pyenv [2] https://pkgstats.archlinux.de/api/packages/python36 [3] https://pkgstats.archlinux.de/api/packages/python37 Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 19:11 +0100, Bartłomiej Piotrowski via arch-dev-public wrote: On 21/11/2020 18.48, Filipe Laíns via arch-dev-public wrote: > I understand that. I am not asking to put all releases of Python on > the > repos, only the active ones, which people are using. I presume you can back it up with numbers how widely 3.7 and 3.6 are used by Arch users. All I can see is that both have less than 30 votes in AUR. Even if I take into account how irrelevant AUR votes are, I assume the problem is exaggerated. I have some metrics that might be relevant. The people that are missing these interpreters generally use pyenv to make up for it, which is not an optimal solution, but the most common when the interpreters are missing. Looking at some package stats for pyenv[1], we see a usage of 3,4%. I think that is a big enough number. pyenv does have another usage, which is managing custom Python interpreters, which is mainly used in CPython development, but I'd say that only a small percentage of the users use it uniquely for that. We can also look directly at the python36 and python37 packages reported, which users might be getting from the AUR or user repos. python37 has 1.18% [2] python36 has 1.63% [3] The main reason why the packages might be required, as I explained in the original email, is for package testing. It puts every maintainer and most contributors to projects that target Python 3.6 or 3.7 in need of an interpreter. Most open source Python project target at least a few different Python versions. These metrics are not perfect by any means, but I do think they present relevant enough data to show an existent need. Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people? Our approach of "YOLO push random new packages to repos" is something I never agreed with, for the record. Unfortunately it's a kingdom with almost no rules. BP [1] https://pkgstats.archlinux.de/api/packages/pyenv [2] https://pkgstats.archlinux.de/api/packages/python36 [3] https://pkgstats.archlinux.de/api/packages/python37 Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On 21/11/2020 18.48, Filipe Laíns via arch-dev-public wrote: > I understand that. I am not asking to put all releases of Python on the > repos, only the active ones, which people are using. I presume you can back it up with numbers how widely 3.7 and 3.6 are used by Arch users. All I can see is that both have less than 30 votes in AUR. Even if I take into account how irrelevant AUR votes are, I assume the problem is exaggerated. > Why are we packaging software that is used by far less people but we > can't package these Python interpreters which are being actively missed > by people? Our approach of "YOLO push random new packages to repos" is something I never agreed with, for the record. Unfortunately it's a kingdom with almost no rules. BP
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 11:24 -0600, Doug Newgard via arch-dev-public wrote: > On Sat, 21 Nov 2020 16:59:21 + > Filipe Laíns via arch-dev-public > wrote: > > > On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev- > > public > > wrote: > > > Am Sat, 21 Nov 2020 14:34:24 + > > > schrieb Filipe Laíns via arch-dev-public > > > : > > > > > > > > > > Does anyone have any big issue with this? What are your > > > > thoughts? > > > > > > > > [1] https://www.python.org/downloads/ > > > > > > > > Cheers, > > > > Filipe Laíns > > > > > > -1 > > > > > > Arch is yours. Whoever needs more and older releases on the > > > system - > > > just do it yourself! In the past we said "use abs and AUR - Arch > > > is > > > the base to make it your own". > > > > This argument can be used to deny adding any package to the repos. > > You > > need this library, tool, etc.? Just add it yourself. > > > > Why are we packaging software that is used by far less people but > > we > > can't package these Python interpreters which are being actively > > missed > > by people? > > > > > I don't want to see users raising bugs that something doesn't > > > work > > > with an older version of python. And I don't want to see these > > > requests > > > pop up every now and then to add even more stuff in different > > > versions. > > > > We already have multiple versions of Java, Ruby, Javascript, etc. > > hell, > > even Python. I don't think having people opening bugs because they > > are > > deliberately using an older version of Python is a big problem. It > > hasn't been for any of the other languages, I don't think it will > > be > > here. > > I think this is an hypothetical that doesn't really materialize > > into > > reality. > > > > > It's sad enough we still have python2 and gtk2 around. To have > > > gcc9 > > > around and other duplicates is not what I want to see growing in > > > Arch. > > > > What you call sad I call a bad UX. Why do we need to force users to > > compile active releases of the Python interpreters themselves, > > which > > can take a long time if they are building with optimization, or to > > resort to pre-built repos with much lower security standards than > > us, > > when there are people willing to maintain them? > > > > I can't understand how it's sad to help out users by not forcing > > them > > to resort the sort of things I mentioned above, as long as we are > > not > > struggling to do so. I like helping people, that's why I am a > > packager, > > that is the opposite of sad for me, so I really can't understand > > this. > > It's more concerning to me that you can't understand this argument > than > anything else so far. Arch keeps old things around in the repos when > they're > required by other things in the repos. It's a necessary evil, not > something to > be actively encouraged. I understand that. I am not asking to put all releases of Python on the repos, only the active ones, which people are using. > > > I don't want to see our distribution transformed into another > > > Debian. > > > > That is not what is happening. > > > > Cheers, > > Filipe Laíns > -- Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 21 Nov 2020 16:59:21 + Filipe Laíns via arch-dev-public wrote: > On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public > wrote: > > Am Sat, 21 Nov 2020 14:34:24 + > > schrieb Filipe Laíns via arch-dev-public > > : > > > > > > > Does anyone have any big issue with this? What are your thoughts? > > > > > > [1] https://www.python.org/downloads/ > > > > > > Cheers, > > > Filipe Laíns > > > > -1 > > > > Arch is yours. Whoever needs more and older releases on the system - > > just do it yourself! In the past we said "use abs and AUR - Arch is > > the base to make it your own". > > This argument can be used to deny adding any package to the repos. You > need this library, tool, etc.? Just add it yourself. > > Why are we packaging software that is used by far less people but we > can't package these Python interpreters which are being actively missed > by people? > > > I don't want to see users raising bugs that something doesn't work > > with an older version of python. And I don't want to see these > > requests > > pop up every now and then to add even more stuff in different > > versions. > > We already have multiple versions of Java, Ruby, Javascript, etc. hell, > even Python. I don't think having people opening bugs because they are > deliberately using an older version of Python is a big problem. It > hasn't been for any of the other languages, I don't think it will be > here. > I think this is an hypothetical that doesn't really materialize into > reality. > > > It's sad enough we still have python2 and gtk2 around. To have gcc9 > > around and other duplicates is not what I want to see growing in > > Arch. > > What you call sad I call a bad UX. Why do we need to force users to > compile active releases of the Python interpreters themselves, which > can take a long time if they are building with optimization, or to > resort to pre-built repos with much lower security standards than us, > when there are people willing to maintain them? > > I can't understand how it's sad to help out users by not forcing them > to resort the sort of things I mentioned above, as long as we are not > struggling to do so. I like helping people, that's why I am a packager, > that is the opposite of sad for me, so I really can't understand this. It's more concerning to me that you can't understand this argument than anything else so far. Arch keeps old things around in the repos when they're required by other things in the repos. It's a necessary evil, not something to be actively encouraged. > > I don't want to see our distribution transformed into another Debian. > > That is not what is happening. > > Cheers, > Filipe Laíns pgpjkdiILK286.pgp Description: OpenPGP digital signature
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public wrote: > Am Sat, 21 Nov 2020 14:34:24 + > schrieb Filipe Laíns via arch-dev-public > : > > > > Does anyone have any big issue with this? What are your thoughts? > > > > [1] https://www.python.org/downloads/ > > > > Cheers, > > Filipe Laíns > > -1 > > Arch is yours. Whoever needs more and older releases on the system - > just do it yourself! In the past we said "use abs and AUR - Arch is > the base to make it your own". This argument can be used to deny adding any package to the repos. You need this library, tool, etc.? Just add it yourself. Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people? > I don't want to see users raising bugs that something doesn't work > with an older version of python. And I don't want to see these > requests > pop up every now and then to add even more stuff in different > versions. We already have multiple versions of Java, Ruby, Javascript, etc. hell, even Python. I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality. > It's sad enough we still have python2 and gtk2 around. To have gcc9 > around and other duplicates is not what I want to see growing in > Arch. What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them? I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this. > I don't want to see our distribution transformed into another Debian. That is not what is happening. Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 16:59 +0100, Morten Linderud via arch-dev-public wrote: > On Sat, Nov 21, 2020 at 04:47:27PM +0100, David Runge wrote: > > On 2020-11-21 14:34:24 (+), Filipe Laíns via arch-dev-public > > wrote: > > > Hi all, > > > > > > I want to propose adding all active Python versions to > > > [community], not > > > just the latest one. This would only entail adding the > > > interpreter > > > itself, no other packages. > > > > > An alternative (in a per-user setup) can be to use pyenv [1]. It > > works > > reasonably well with tox etc. and I have used it in the past > > successfully to develop against several python interpreter > > versions. > > I'm personally not very excited for the idea of adding more python > interpreters. > I'm a bit concerned that we today say "no packages" but in the future > we relax a > bit and end up with python36-$pkgname, as it's the pragmatic option > as opposed > to blocking entire rebuilds or package updates. We can add a guideline blocking this. > What is the downsides of utilizing something like pyenv? There are > user > repositories providing older python interpreters as well if people > need it. > pyenv forces users to compile the Python interpreter themselves, which can take a long time with --enable-optimizations. None of the user repos available builds with optimizations, or has signed packages AFAIK. Of course they could in the future, but I think having the packages in the repos is much better in terms of security and usability. I run one of the user which provides these packages and I don't see myself fixing any of the issues I pointed out due to technical limitations. Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, Nov 21, 2020 at 04:47:27PM +0100, David Runge wrote: > On 2020-11-21 14:34:24 (+), Filipe Laíns via arch-dev-public wrote: > > Hi all, > > > > I want to propose adding all active Python versions to [community], not > > just the latest one. This would only entail adding the interpreter > > itself, no other packages. > > > An alternative (in a per-user setup) can be to use pyenv [1]. It works > reasonably well with tox etc. and I have used it in the past > successfully to develop against several python interpreter versions. I'm personally not very excited for the idea of adding more python interpreters. I'm a bit concerned that we today say "no packages" but in the future we relax a bit and end up with python36-$pkgname, as it's the pragmatic option as opposed to blocking entire rebuilds or package updates. What is the downsides of utilizing something like pyenv? There are user repositories providing older python interpreters as well if people need it. -- Morten Linderud PGP: 9C02FF419FECBE16 signature.asc Description: PGP signature
Re: [arch-dev-public] Add active Python versions to the repos
Am Sat, 21 Nov 2020 14:34:24 + schrieb Filipe Laíns via arch-dev-public : > Does anyone have any big issue with this? What are your thoughts? > > [1] https://www.python.org/downloads/ > > Cheers, > Filipe Laíns -1 Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own". I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions. It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch. I don't want to see our distribution transformed into another Debian. -Andy pgp6EiOKQF09B.pgp Description: Digitale Signatur von OpenPGP
Re: [arch-dev-public] Add active Python versions to the repos
On 2020-11-21 14:34:24 (+), Filipe Laíns via arch-dev-public wrote: > Hi all, > > I want to propose adding all active Python versions to [community], not > just the latest one. This would only entail adding the interpreter > itself, no other packages. > > Having access to interpreters for older active versions is really > helpful for Python developers. This allows them to easily run test > suites against older versions. It is very common for developers to > maintain software against a couple major releases. Tools like tox or > nox are able to automate testing against multiple Python versions, just > needing the interpreter. > > The current active Python releases are: > - 3.9 > - 3.8 > - 3.7 > - 3.6 > - 2.7 > > The list can be found here[1]. > > So, I propose introducing 2 new packages: > - python3.7 > - python3.6 > > And when we update the python package to 3.9: > - python3.8 > > Does anyone have any big issue with this? What are your thoughts? I guess a downside is the increase in maintenance burden to provide this and it potentially leading to things not being ported/ fixed (ergo fragmentation) as maintainers might want to rely on several interpreters being around in the future (to build packages against). However, as you are specifically stating you would like to only have the interpreters added, maybe that's fine? An alternative (in a per-user setup) can be to use pyenv [1]. It works reasonably well with tox etc. and I have used it in the past successfully to develop against several python interpreter versions. Best, David [1] https://www.archlinux.org/packages/community/any/pyenv/ -- https://sleepmap.de signature.asc Description: PGP signature
Re: [arch-dev-public] Add active Python versions to the repos
On Sat, 2020-11-21 at 07:51 -0700, Zach Himsel wrote: > What would determine what the "default" python be? The latest? > > > -- > Zach Himsel > mailto://z...@himsel.net > http://zach.himsel.net Yep, that would stay exactly as it is right now. We would only additionally provide interpreters (and interpreters only!) for the older Python versions. Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
