Re: Re: dropping python2 [was Re: scientific python stack transitions]
On 7/16/19 5:42 PM, Matthias Klose wrote: > no, that's your interpretation, not mine. If an important enough application > still needs it, we should ship it. Could you please clarify what you call "an important enough application"? Important for who/what? Who's to decide, and on what criteria? Also, if that application is important enough, why nobody worked on porting it to Python 3? Cheers, Thomas Goirand (zigo)
Re: Re: dropping python2 [was Re: scientific python stack transitions]
On July 16, 2019 3:42:47 PM UTC, Matthias Klose wrote: >On 16.07.19 17:31, Julien Puydt wrote: >> Le 16/07/2019 à 17:21, Andrey Rahmatullin a écrit : >>> Python 2 is included in buster and so will be supported for several >>> years. >>> >> >> The starting point of the thread was : >> 1. Ok, buster has Python 2 so even if upstream drops it, we will >still >> support it for the years to come for our users ; >> 2. but the next stable will come out after upstream has EOL-ed Python >2, >> so we must kick it out as soon as possible from testing+unstable, > >> because there's no way we'll support Python 2 in that next stable! > >no, that's your interpretation, not mine. If an important enough >application >still needs it, we should ship it. I completely agree. The reason I put in for a tracker is to make it easier to see where people can focus to start burning down the stack. Not because I am confident we'll get there this cycle. Scott K
Re: Re: dropping python2 [was Re: scientific python stack transitions]
On 16.07.19 17:31, Julien Puydt wrote: > Le 16/07/2019 à 17:21, Andrey Rahmatullin a écrit : >> Python 2 is included in buster and so will be supported for several >> years. >> > > The starting point of the thread was : > 1. Ok, buster has Python 2 so even if upstream drops it, we will still > support it for the years to come for our users ; > 2. but the next stable will come out after upstream has EOL-ed Python 2, > so we must kick it out as soon as possible from testing+unstable, > because there's no way we'll support Python 2 in that next stable! no, that's your interpretation, not mine. If an important enough application still needs it, we should ship it.
Re: dropping python2 [was Re: scientific python stack transitions]
On Tue, Jul 16, 2019 at 11:20 AM Matthias Klose wrote: > > On 16.07.19 16:52, Paul Tagliamonte wrote: > > I lost some of this thread - should we request a transition > > from the release team? I was looking for the list of blockers > > to dropping Python 2 and couldn't find anything except this > > thread (where we're still figuring out what to do, of course) > > #931659 Perfect, thank you! > > > I do want to keep in mind that Python 2 is EOL and dead > > in 5 months, 15 days and 12 hours. > > yes, that's the upstream status, not the distro status. Does this mean we plan to commit to providing security updates to Python 2 alone? Do we have the bandwidth and willing maintainers? Thank you! paultag -- :wq
Re: Re: dropping python2 [was Re: scientific python stack transitions]
Le 16/07/2019 à 17:21, Andrey Rahmatullin a écrit : > Python 2 is included in buster and so will be supported for several > years. > The starting point of the thread was : 1. Ok, buster has Python 2 so even if upstream drops it, we will still support it for the years to come for our users ; 2. but the next stable will come out after upstream has EOL-ed Python 2, so we must kick it out as soon as possible from testing+unstable, because there's no way we'll support Python 2 in that next stable! And I think no-one really objected to those two points :-P JP
Re: dropping python2 [was Re: scientific python stack transitions]
Le 16/07/2019 à 16:52, Paul Tagliamonte a écrit : > I lost some of this thread - should we request a transition > from the release team? I was looking for the list of blockers > to dropping Python 2 and couldn't find anything except this > thread (where we're still figuring out what to do, of course) > > I do want to keep in mind that Python 2 is EOL and dead > in 5 months, 15 days and 12 hours. > Since you're asking about what can block, and since I didn't follow the thread closely enough to know if that was already mentioned : sagemath still depends on Python 2... upstream is busy with trying to depend on Python 3, almost there, but not there yet: https://trac.sagemath.org/query?status=!closed=python3 Cheers, JP
Re: dropping python2 [was Re: scientific python stack transitions]
On Tue, Jul 16, 2019 at 10:52:14AM -0400, Paul Tagliamonte wrote: > I lost some of this thread - should we request a transition > from the release team? I was looking for the list of blockers > to dropping Python 2 and couldn't find anything except this > thread (where we're still figuring out what to do, of course) > > I do want to keep in mind that Python 2 is EOL and dead > in 5 months, 15 days and 12 hours. Python 2 is included in buster and so will be supported for several years. -- WBR, wRAR signature.asc Description: PGP signature
Re: dropping python2 [was Re: scientific python stack transitions]
On 2019-07-08 10:00, Elena ``of Valhalla'' wrote: > I don't think it would be accepted by backports, since it goes against > the requirement that stuff in backports is in testing (and meant to > remain there when it becomes stable). I'm not sure, but building an additional binary package from the same source package might be OK for bpo. Of course, d/control etc. would differ, but that's common.
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/9/19 12:22 AM, Scott Kitterman wrote: > On Monday, July 8, 2019 5:45:17 PM EDT Thomas Goirand wrote: >> How can I get debtree to use Sid instead of Buster (as I'd prefer to >> keep this VM running Buster)? I could set this VM up and a cron job for >> how long we need it... Though it looks like I probably have over-sized it. > > I think that's a useful view of the problem space, but it seems to miss some > things. As I wrote earlier last night, I restarted the calculation, pushing the limits to maximum. It took 20 minutes for graphviz to calculate. Here's the result: http://py2graph.infomaniak.ch/py2.7.deps.svg As you can see the graph is big in size (a large 11 MB svg file), and large in the screen (it doesn't fit on my work's 38" screen). It's also very hard to interpret, because too big. The consequence I see with it, is that it's going to be really too slow and almost impossible to do the Py2 removal the correct way in just 2 years of time. This reinforce my first impression that we unfortunately must allow ourselves to break reverse dependencies when doing the job, even if we should, at least, attempt to first remove Py2 from leaf packages as much as possible. > I think a transition tracker would also be useful (start at the top > level, not the bottom). I'll ask the release team to set one up. Thanks, that'd be indeed super useful, thanks for it. Hopefully, more than this huge graph that hardly helps the decision making process. Cheers, Thomas Goirand (zigo)
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/9/19 12:05 AM, Rick Thomas wrote: > > >> On Jul 8, 2019, at 2:45 PM, Thomas Goirand wrote: >> >> What I did so far: >> debtree -R --rdeps-depth=100 python2.7 >py2.7.deps.dot >> dot -Tsvg -o py2.7.deps.svg py2.7.deps.dot >> >> The result is here: >> http://py2graph.infomaniak.ch/py2.7.deps.svg > > Fascinating… > Can you give us some hints to interpret the graph? > In particular, what do the different colors on the arrows mean? The graph is missing lots of bits. I've pushed the limits of debtree more, and now it's been running for 15 minutes already. Let's see if it is still running when I wake up tomorrow! :) Thomas
Re: dropping python2 [was Re: scientific python stack transitions]
On Monday, July 8, 2019 5:45:17 PM EDT Thomas Goirand wrote: > On 7/8/19 6:28 PM, Stewart Ferguson wrote: > > On 2019-07-08 00:13:45, Thomas Goirand wrote: > >> On 7/7/19 5:31 PM, Matthias Klose wrote: > >>> you can start dropping it now, however please don't drop anything yet > >>> with > >>> reverse dependencies. So leaf packages first. > >> > >> I'm sorry, but I think I need to contest this. Doing things in order, > >> first leaf, then go all the way back, will take too long, and this is > >> IMO unnecessary effort. > > > > I maintain tuna, a leaf package. Upstream is working on the py2->py3 > > migration. I expect it to be ready this cycle. I was planning to replace > > the py2 deps with that upstream release. I can release a stripped down > > (headless) version in python3 and I'll do that if my package is going to > > be removed, but I'd rather wait for upstream's py3 release. > > Everyone would like to have better upstream. For OpenStack, swift (the > object storage) is still there, without a full Py2 support. I don't want > to loose Swift, though if it is removed from Testing for a short time, > while upstream finishes to fix things, it's not a big deal, there's > always Buster where it can be consumed in the mean time. Do you feel > differently for your own package? > > > Can we set dates for removing packages so maintainers can plan what's > > happening? > We already have setup dates: right after Buster is released. This was > last Saturday. Could you explain why we should delay it more, and how > this will be beneficial to the Python 2 removal plan? > > > I suspect it wouldn't be hard to build a tree from an apt-rdepends graph > > to set up a schedule, guaranteeing all traces of python2 are removed by > > bullseye and defining the latest possible removal dates for each package > > in testing. > A constantly updated graph of Python 2 dependencies would definitively > be super helpful so we could walk through it in reverse order (as much > as possible). This way, we could also know when we're breaking things, > and file RC bugs when needed. Thanks a lot for this idea. > > Though unfortunately, I'm not convinced setting-up deadlines will work, > given the way Debian works (ie: you cannot force any contributor to do > something). > > If needed, I can provide the infrastructure (ie: a large VM hosting the > graph, directly connected to a Debian mirror at 10 Gbits/s). Though I > haven't played much with graphviz to produce such a graph. Does any of > us know? > > What I did so far: > debtree -R --rdeps-depth=100 python2.7 >py2.7.deps.dot > dot -Tsvg -o py2.7.deps.svg py2.7.deps.dot > > The result is here: > http://py2graph.infomaniak.ch/py2.7.deps.svg > > How can I get debtree to use Sid instead of Buster (as I'd prefer to > keep this VM running Buster)? I could set this VM up and a cron job for > how long we need it... Though it looks like I probably have over-sized it. I think that's a useful view of the problem space, but it seems to miss some things. I think a transition tracker would also be useful (start at the top level, not the bottom). I'll ask the release team to set one up. Scott K
Re: dropping python2 [was Re: scientific python stack transitions]
> On Jul 8, 2019, at 2:45 PM, Thomas Goirand wrote: > > What I did so far: > debtree -R --rdeps-depth=100 python2.7 >py2.7.deps.dot > dot -Tsvg -o py2.7.deps.svg py2.7.deps.dot > > The result is here: > http://py2graph.infomaniak.ch/py2.7.deps.svg Fascinating… Can you give us some hints to interpret the graph? In particular, what do the different colors on the arrows mean? Thanks! Rick
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/8/19 6:28 PM, Stewart Ferguson wrote: > On 2019-07-08 00:13:45, Thomas Goirand wrote: >> On 7/7/19 5:31 PM, Matthias Klose wrote: >>> you can start dropping it now, however please don't drop anything yet with >>> reverse dependencies. So leaf packages first. >> >> I'm sorry, but I think I need to contest this. Doing things in order, >> first leaf, then go all the way back, will take too long, and this is >> IMO unnecessary effort. > > I maintain tuna, a leaf package. Upstream is working on the py2->py3 > migration. > I expect it to be ready this cycle. I was planning to replace the py2 deps > with > that upstream release. I can release a stripped down (headless) version in > python3 and I'll do that if my package is going to be removed, but I'd rather > wait for upstream's py3 release. Everyone would like to have better upstream. For OpenStack, swift (the object storage) is still there, without a full Py2 support. I don't want to loose Swift, though if it is removed from Testing for a short time, while upstream finishes to fix things, it's not a big deal, there's always Buster where it can be consumed in the mean time. Do you feel differently for your own package? > Can we set dates for removing packages so maintainers can plan what's > happening? We already have setup dates: right after Buster is released. This was last Saturday. Could you explain why we should delay it more, and how this will be beneficial to the Python 2 removal plan? > I suspect it wouldn't be hard to build a tree from an apt-rdepends graph to > set > up a schedule, guaranteeing all traces of python2 are removed by bullseye and > defining the latest possible removal dates for each package in testing. A constantly updated graph of Python 2 dependencies would definitively be super helpful so we could walk through it in reverse order (as much as possible). This way, we could also know when we're breaking things, and file RC bugs when needed. Thanks a lot for this idea. Though unfortunately, I'm not convinced setting-up deadlines will work, given the way Debian works (ie: you cannot force any contributor to do something). If needed, I can provide the infrastructure (ie: a large VM hosting the graph, directly connected to a Debian mirror at 10 Gbits/s). Though I haven't played much with graphviz to produce such a graph. Does any of us know? What I did so far: debtree -R --rdeps-depth=100 python2.7 >py2.7.deps.dot dot -Tsvg -o py2.7.deps.svg py2.7.deps.dot The result is here: http://py2graph.infomaniak.ch/py2.7.deps.svg How can I get debtree to use Sid instead of Buster (as I'd prefer to keep this VM running Buster)? I could set this VM up and a cron job for how long we need it... Though it looks like I probably have over-sized it. Cheers, Thomas Goirand (zigo)
Re: dropping python2 [was Re: scientific python stack transitions]
On 2019-07-08 00:13:45, Thomas Goirand wrote: > On 7/7/19 5:31 PM, Matthias Klose wrote: >> you can start dropping it now, however please don't drop anything yet with >> reverse dependencies. So leaf packages first. > > I'm sorry, but I think I need to contest this. Doing things in order, > first leaf, then go all the way back, will take too long, and this is > IMO unnecessary effort. I maintain tuna, a leaf package. Upstream is working on the py2->py3 migration. I expect it to be ready this cycle. I was planning to replace the py2 deps with that upstream release. I can release a stripped down (headless) version in python3 and I'll do that if my package is going to be removed, but I'd rather wait for upstream's py3 release. Can we set dates for removing packages so maintainers can plan what's happening? I suspect it wouldn't be hard to build a tree from an apt-rdepends graph to set up a schedule, guaranteeing all traces of python2 are removed by bullseye and defining the latest possible removal dates for each package in testing. Stewart Ferguson signature.asc Description: This is a digitally signed message part
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/8/19 10:10 AM, Ansgar wrote: > Thomas Goirand writes: >> On 7/7/19 5:31 PM, Matthias Klose wrote: >>> you can start dropping it now, however please don't drop anything yet with >>> reverse dependencies. So leaf packages first. >> >> I'm sorry, but I think I need to contest this. Doing things in order, >> first leaf, then go all the way back, will take too long, and this is >> IMO unnecessary effort. Older binary packages will anyway stay in the >> archive as long as they are needed, and no FTP hint is added (please >> correct me if I'm wrong here... but that's what I saw in the past). > > Packages usually don't migrate to testing if they cause packages to be > uninstallable which will happen if you start breaking reverse > dependencies. Will that be the case here? Right. Which will be an incentive to fix the reverse-dependency, and a way to see what work is remaining to be done. Maybe we can also do a mass-bugfilling after a period we can discuss (probably during Debconf?). > Removing an entire language isn't a simple case, even less so when > doesn't mean we just remove all packages written in said language Right. Thomas
Re: dropping python2 [was Re: scientific python stack transitions]
On 2019-07-08 at 07:16:01 +, PICCA Frederic-Emmanuel wrote: > it would be nice, if the python2 packages could be skipped in bulleye but not > for backports in buster. > > Is it something which could be envision ? I don't think it would be accepted by backports, since it goes against the requirement that stuff in backports is in testing (and meant to remain there when it becomes stable). -- Elena ``of Valhalla''
Re: dropping python2 [was Re: scientific python stack transitions]
Thomas Goirand writes: > On 7/7/19 5:31 PM, Matthias Klose wrote: >> you can start dropping it now, however please don't drop anything yet with >> reverse dependencies. So leaf packages first. > > I'm sorry, but I think I need to contest this. Doing things in order, > first leaf, then go all the way back, will take too long, and this is > IMO unnecessary effort. Older binary packages will anyway stay in the > archive as long as they are needed, and no FTP hint is added (please > correct me if I'm wrong here... but that's what I saw in the past). Packages usually don't migrate to testing if they cause packages to be uninstallable which will happen if you start breaking reverse dependencies. Will that be the case here? > You've done some pretty destructive transitions yourself for other > stuff, so why should we bother on this simple case? Removing an entire language isn't a simple case, even less so when doesn't mean we just remove all packages written in said language (as the source packages all build for a different language as well). Ansgar
Re: dropping python2 [was Re: scientific python stack transitions]
On Mon, Jul 08, 2019 at 07:16:01AM +, PICCA Frederic-Emmanuel wrote: > it would be nice, if the python2 packages could be skipped in bulleye but not > for backports in buster. > > Is it something which could be envision ? Not in the main backports, maybe in -sloppy. And it will be hard or impossible for some packages, I think. -- WBR, wRAR signature.asc Description: PGP signature
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/8/19 12:27 AM, Scott Kitterman wrote: > Are you 100% certain that there isn't some small subset of python2 packages > we > will end up needing for bullseye? I'm 100% certain that we should as much as possible avoid this, indeed. I'm also convinced we should do our best to speed-up the Python 2 removal as much as possible, so that we can see what's remaining to be done. If we don't do it fast, then we may end up in the very bad situation you're describing. Cheers, Thomas Goirand (zigo)
Re: dropping python2 [was Re: scientific python stack transitions]
On Sunday, July 7, 2019 6:13:45 PM EDT Thomas Goirand wrote: > On 7/7/19 5:31 PM, Matthias Klose wrote: > > On 07.07.19 16:55, Drew Parsons wrote: > >> On 2019-07-07 22:46, Mo Zhou wrote: > >>> Hi science team, > >>> > >>> By the way, when do we start dropping python2 support? > >>> The upstreams of the whole python scientific computing > >>> stack had already started dropping it. > >> > >> Good question. I think it is on the agenda this cycle, but debian-python > >> will have the call on it. > > > > you can start dropping it now, however please don't drop anything yet with > > reverse dependencies. So leaf packages first. > > > > Matthias > > I'm sorry, but I think I need to contest this. Doing things in order, > first leaf, then go all the way back, will take too long, and this is > IMO unnecessary effort. Older binary packages will anyway stay in the > archive as long as they are needed, and no FTP hint is added (please > correct me if I'm wrong here... but that's what I saw in the past). > > Also, those who aren't doing the work of switching to Py2 will need to > be kicked away anyway. > > You've done some pretty destructive transitions yourself for other > stuff, so why should we bother on this simple case? > > Last, a 2 years cycle to get rid of all traces of Python 2 *will* take > some time, maybe more than we can even think of, so we'd better take > some shortcuts if we can. Are you 100% certain that there isn't some small subset of python2 packages we will end up needing for bullseye? If not, this is not a good suggestion. Scott K
Re: dropping python2 [was Re: scientific python stack transitions]
On 7/7/19 5:31 PM, Matthias Klose wrote: > On 07.07.19 16:55, Drew Parsons wrote: >> On 2019-07-07 22:46, Mo Zhou wrote: >>> Hi science team, >>> >>> By the way, when do we start dropping python2 support? >>> The upstreams of the whole python scientific computing >>> stack had already started dropping it. >> >> Good question. I think it is on the agenda this cycle, but debian-python >> will >> have the call on it. > > you can start dropping it now, however please don't drop anything yet with > reverse dependencies. So leaf packages first. > > Matthias I'm sorry, but I think I need to contest this. Doing things in order, first leaf, then go all the way back, will take too long, and this is IMO unnecessary effort. Older binary packages will anyway stay in the archive as long as they are needed, and no FTP hint is added (please correct me if I'm wrong here... but that's what I saw in the past). Also, those who aren't doing the work of switching to Py2 will need to be kicked away anyway. You've done some pretty destructive transitions yourself for other stuff, so why should we bother on this simple case? Last, a 2 years cycle to get rid of all traces of Python 2 *will* take some time, maybe more than we can even think of, so we'd better take some shortcuts if we can. Cheers, Thomas Goirand (zigo)
Re: dropping python2 [was Re: scientific python stack transitions]
On 2019-07-07 23:31, Matthias Klose wrote: On 07.07.19 16:55, Drew Parsons wrote: On 2019-07-07 22:46, Mo Zhou wrote: Hi science team, By the way, when do we start dropping python2 support? The upstreams of the whole python scientific computing stack had already started dropping it. Good question. I think it is on the agenda this cycle, but debian-python will have the call on it. you can start dropping it now, however please don't drop anything yet with reverse dependencies. So leaf packages first. That a sensible order. It puts packages on the same footing as new packages, which are already forbidden by lintian from building python2 packages. Drew
Re: dropping python2 [was Re: scientific python stack transitions]
On 07.07.19 16:55, Drew Parsons wrote: > On 2019-07-07 22:46, Mo Zhou wrote: >> Hi science team, >> >> By the way, when do we start dropping python2 support? >> The upstreams of the whole python scientific computing >> stack had already started dropping it. > > Good question. I think it is on the agenda this cycle, but debian-python will > have the call on it. you can start dropping it now, however please don't drop anything yet with reverse dependencies. So leaf packages first. Matthias
dropping python2 [was Re: scientific python stack transitions]
On 2019-07-07 22:46, Mo Zhou wrote: Hi science team, By the way, when do we start dropping python2 support? The upstreams of the whole python scientific computing stack had already started dropping it. Good question. I think it is on the agenda this cycle, but debian-python will have the call on it. Drew
