[Python-Dev] [Release] Status of Python 3.11 release
Hi everyone, # TLDR We may be pushing the final release until December if the stability of Python 3.11 doesn't improve. # Long Explanation Unfortunately, we cannot still release the next Python 3.11 beta release (3.11.0b4) because we still have a bunch of pending release blockers. Unfortunately, this is still after several preexisting release blockers have been fixed, some of them being discovered after I sent my last update. These are the current release blockers: https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Arelease-blocker+label%3A3.11+ We also have some deferred blockers (some of them should actually be release blockers): https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Adeferred-blocker+label%3A3.11 Due to this and the fact that we are already 3 weeks delayed in the release schedule, the current stability of Python 3.11 is not as good as it is supposed to be at this stage of the release schedule and more testing from end-users and library authors is required. After discussing with the Steering Council, we are considering delaying the final release until December to allow for two more beta releases. This is how we are going to proceed: * If the current release blockers are not fixed by the end of this week, two more betas will be released (1 month per beta) and we will *definitely* delay the final release until December. * If the current release blockers are fixed we will proceed to release Python 3.11.0b4 on Monday. We will target the current release date (Monday, 2022-10-03) but if more release blockers that affect fundamental parts of the Python interpreter or the standard libraries are raised, the release team will still consider adding two more betas nd pushing the final release to December. One of the goals that we are going to try to achieve from the release team is that no substantial code changes are added between the last beta and the first release candidate. This is so all the fixes that affect fundamental parts of the interpreter or the standard library can be tested by end-users before the first release candidate is released (and not with it). This is also partially because once we release the first release candidate, the ABI will be frozen and certain kinds of fixes will be more complicated. Hopefully, this addresses some of you that have reached out with concerns over the stability of Python 3.11 and the release schedule. I understand that delaying the release until December will complicate things for some Linux distributions and will affect end users and redistributors targeting the original release, but please understand that our responsibility in the release team after all is to guarantee a stable final release above all and unfortunately, we don't currently have the confidence that we would like given the current state of the release process. Please do not hesitate in reaching out if you have any questions or concerns. Thanks, everyone for your help and understanding and thanks a lot to all of you for your great work! Cheers from cloudy London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3JWVCSBPBFWY5ZWSJ7RYB6FS5NIMCEOY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: EPOLLEXCLUSIVE and selectors
> On 3 Jul 2022, at 00:42, David Gilman wrote: > > Hello, I'd like to bring up https://bugs.python.org/issue44951 / > https://github.com/python/cpython/pull/27819 to the mailing list's > consideration as it has idled for a bit. I would appreciate some > authoritative feedback on which API design choice is best. I'll also > recap the PR quickly: > > Motivation: > - There is community demand for EPOLLEXCLUSIVE in Python (see blog > posts in BPO issue) > - You can't do this with the existing stdlib code as > _BaseSelectorImpl.register() raises ValueError on non-READ/WRITE > constants. (This is why _PollLikeSelector.register() has two > variables, events and poller_events) > - It's not an invasive change. The Python API doesn't change much or > at all and the kernel's EPOLLEXCLUSIVE behavior was carefully designed > to be backwards compatible with using exclusive and non-exclusive > watches on the same file descriptor. > > I've got two approaches to this. The first extends the EpollSelector > class with a property to toggle setting the EPOLLEXCLUSIVE on newly > registered file descriptors: > > https://github.com/dgilman/cpython/commit/43174df5bd7a78eedf0692ebbe63a9b943248a74 > > The second introduces an entirely new sibling class, > EpollExclusiveSelector, that unconditionally sets it on registration: > > https://github.com/dgilman/cpython/commit/554a5bf9c16b6bd82ce47b2d0dd1833f2bdd31cb > > The first one was my first attempt but I am leaning towards the > second. It doesn't require any new API surface area. It also gets > integrated into the DefaultSelector logic, and even if that shouldn't > happen it's still easy to swap out your existing selector class for > the EpollExclusiveSelector class. > > Enabling EPOLLEXCLUSIVE by default: > >> From the research I did last year my understanding is that > EPOLLEXCLUSIVE is never a performance drawback on Linux, and in the > case of a server that gets fast traffic it's a dramatic improvement. > However, I have not done my own benchmarking (with say, gunicorn, > which uses the stdlib's selectors module). > > Note that EPOLLEXCLUSIVE does have one kernel API break: you can no > longer use EPOLL_CTL_MOD on an exclusive file descriptor. Python uses > the _MOD flag under the hood to implement epoll.modify(), which > results in EpollSelector.modify() throwing an OSError if you try to > modify an exclusive file descriptor. > > In the second PR I implemented a EpollExclusiveSelector.modify() which > unregisters and reregisters the file descriptor to get around the _MOD > behavior. This means no surprise crash when someone updates Python. > But there may be other subtle regressions here: the performance of > modify() is likely going to regress, and someone could have a > dependency on Python actually using _MOD. > > But as a rebuttal to those, I have a feeling that nobody really > depends on the performance of modify(), and even in the case where > someone does use it a lot it's likely for the data= path which is > completely unchanged here. I also am struggling to think of a place > where someone would care about the kernel-level changes between _MOD > and _ADD/_DEL but that might be my own lack of imagination or > knowledge of epoll techniques. > > Maybe a compromise is to ship EpollExclusveSelector for a release > without it being the default and bump it to the default after seeing > if anyone's turned up any incompatibilities.s This would benefit frameworks like twisted. I have seen the thundering herd because of this problem In my work. Barry > > -- > David Gilman > :DG< > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VB2BBUUJENMWPGXGFMON7UQW27ZOBIB7/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RLVYNSTSSH465JPWAVHEXJACDEE2W4G5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [Release] Python 3.11.0b4 is still blocked
Hi Petr and Miro, Thanks for communicating your concerns regarding Fedora and the release schedule. >> It would be really great to get something ABI stable at Beta Freeze and at least an RC at the Final Freeze. If that is not realistic, we would need to consider a revert. I am not sure if there is some typo, but the beta freeze is the first beta. The ABI is frozen in the first release candidate, not in beta freeze. We won't get stable ABI until the first RC is released. >> Some months sounds pretty big to me. Once the current beta is released, I'd be great to see some updated release schedule. We have just updated the main Python version Fedora 37 to 3.11 and we have some deadlines I'd like not to miss. I understand that in the (still not decided) case that the release is delayed will be quite inconvenient for Fedora and other Linux distributions. I will surely take this into account when making a decision and I will try to avoid having to fall into this, but please understand that the Release Team's responsibility is ensuring a stable release, and given the events in the In past weeks, I do not feel comfortable with the current level of testing so we may require more betas. Thanks for your understanding, Pablo Galindo Salgado On Mon, 4 Jul 2022 at 18:21, Petr Viktorin wrote: > On 04. 07. 22 19:03, Miro Hrončok wrote: > > On 04. 07. 22 18:53, Pablo Galindo Salgado wrote: > >> Hi Miro, > >> > >> >> Are all release blockers automatically blocking the next beta? > >> > >> Yes. > >> > >> >> Or does it mean this should not be released in final (and hence > >> neither rc) > >> versions? > >> > >> Release blockers block also beta releases (if the RM decides so). > >> > >> >> Would it make sense to release 3.11.0b4 with some not-yet-fixed > >> blockers? > >> > >> No, the reason is that fixes can introduce more regressions and those > >> need to be fixed. If these fixes > >> are pretty big we would be risking big changes in the RC phase, which > >> we want to avoid. The idea is that > >> the fixes to critical problems reported on beta x can be tested on > >> beta x+1. > >> > >> At the end of the day, this is all subjected to the judgement of the > >> release manager, and given how many > >> release blockers we have been getting and how many of these have been > >> reported past week *after* several > >> attempts to release the next beta, I have decided to wait. > > > > Thanks. Understood. > > > >> Additionally, I am considering pushing the full release some months in > >> the future to allow for more betas, given > >> how unstable 3.11 is currently. > > > > Some months sounds pretty big to me. Once the current beta is released, > > I'd be great to see some updated release schedule. We have just updated > > the main Python version Fedora 37 to 3.11 and we have some deadlines I'd > > like not to miss. > > > > https://fedorapeople.org/groups/schedule/f-37/f-37-key-tasks.html > > 2022-08-23 - Fedora 37 Beta Freeze > > 2022-10-04 - Fedora 37 Final Freeze > > > > It would be really great to get something ABI stable at Beta Freeze and > > at least an RC at the Final Freeze. If that is not realistic, we would > > need to consider a revert. > > Worse than a one-time revert. With the current schedule, the projects' > testing phases overlap so Fedora can test rebuilding all its Python > software with Python's alphas/betas. If the schedule is adjusted or made > unreliable, Fedora might need to add a six-month delay and rebuild with > final releases -- and find bugs much later. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/FRAKKZNIVUL46JLPMRR76H24RSYRQMP7/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DLMOU6YL6WCFHIZYHAIZHWPYTSRT7EQI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [Release] Python 3.11.0b4 is still blocked
On 04. 07. 22 19:03, Miro Hrončok wrote: On 04. 07. 22 18:53, Pablo Galindo Salgado wrote: Hi Miro, >> Are all release blockers automatically blocking the next beta? Yes. >> Or does it mean this should not be released in final (and hence neither rc) versions? Release blockers block also beta releases (if the RM decides so). >> Would it make sense to release 3.11.0b4 with some not-yet-fixed blockers? No, the reason is that fixes can introduce more regressions and those need to be fixed. If these fixes are pretty big we would be risking big changes in the RC phase, which we want to avoid. The idea is that the fixes to critical problems reported on beta x can be tested on beta x+1. At the end of the day, this is all subjected to the judgement of the release manager, and given how many release blockers we have been getting and how many of these have been reported past week *after* several attempts to release the next beta, I have decided to wait. Thanks. Understood. Additionally, I am considering pushing the full release some months in the future to allow for more betas, given how unstable 3.11 is currently. Some months sounds pretty big to me. Once the current beta is released, I'd be great to see some updated release schedule. We have just updated the main Python version Fedora 37 to 3.11 and we have some deadlines I'd like not to miss. https://fedorapeople.org/groups/schedule/f-37/f-37-key-tasks.html 2022-08-23 - Fedora 37 Beta Freeze 2022-10-04 - Fedora 37 Final Freeze It would be really great to get something ABI stable at Beta Freeze and at least an RC at the Final Freeze. If that is not realistic, we would need to consider a revert. Worse than a one-time revert. With the current schedule, the projects' testing phases overlap so Fedora can test rebuilding all its Python software with Python's alphas/betas. If the schedule is adjusted or made unreliable, Fedora might need to add a six-month delay and rebuild with final releases -- and find bugs much later. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FRAKKZNIVUL46JLPMRR76H24RSYRQMP7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [Release] Python 3.11.0b4 is still blocked
On 04. 07. 22 18:53, Pablo Galindo Salgado wrote: Hi Miro, >> Are all release blockers automatically blocking the next beta? Yes. >> Or does it mean this should not be released in final (and hence neither rc) versions? Release blockers block also beta releases (if the RM decides so). >> Would it make sense to release 3.11.0b4 with some not-yet-fixed blockers? No, the reason is that fixes can introduce more regressions and those need to be fixed. If these fixes are pretty big we would be risking big changes in the RC phase, which we want to avoid. The idea is that the fixes to critical problems reported on beta x can be tested on beta x+1. At the end of the day, this is all subjected to the judgement of the release manager, and given how many release blockers we have been getting and how many of these have been reported past week *after* several attempts to release the next beta, I have decided to wait. Thanks. Understood. Additionally, I am considering pushing the full release some months in the future to allow for more betas, given how unstable 3.11 is currently. Some months sounds pretty big to me. Once the current beta is released, I'd be great to see some updated release schedule. We have just updated the main Python version Fedora 37 to 3.11 and we have some deadlines I'd like not to miss. https://fedorapeople.org/groups/schedule/f-37/f-37-key-tasks.html 2022-08-23 - Fedora 37 Beta Freeze 2022-10-04 - Fedora 37 Final Freeze It would be really great to get something ABI stable at Beta Freeze and at least an RC at the Final Freeze. If that is not realistic, we would need to consider a revert. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EP37TDGLFXE6TKQH6YBIQWSUGKWHC32U/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [Release] Python 3.11.0b4 is still blocked
Hi Miro, >> Are all release blockers automatically blocking the next beta? Yes. >> Or does it mean this should not be released in final (and hence neither rc) versions? Release blockers block also beta releases (if the RM decides so). >> Would it make sense to release 3.11.0b4 with some not-yet-fixed blockers? No, the reason is that fixes can introduce more regressions and those need to be fixed. If these fixes are pretty big we would be risking big changes in the RC phase, which we want to avoid. The idea is that the fixes to critical problems reported on beta x can be tested on beta x+1. At the end of the day, this is all subjected to the judgement of the release manager, and given how many release blockers we have been getting and how many of these have been reported past week *after* several attempts to release the next beta, I have decided to wait. Additionally, I am considering pushing the full release some months in the future to allow for more betas, given how unstable 3.11 is currently. Pablo Galindo Salgado On Mon, 4 Jul 2022 at 15:26, Miro Hrončok wrote: > On 24. 06. 22 14:25, Pablo Galindo Salgado wrote: > > Hi everyone, > > > > A small update since the last communication from the release team > regarding the > > status of Python 3.11.0b4. > > > > Unfortunately, even if we have fixed most of the original release > blockers and > > 4 more that appear during this week, we still have a bunch of release > blockers > > to deal with. One of them has been reported today. > > > > I would like to release the next beta next week if everything looks > good, but > > there are also some items that need discussion... > > I was thinking. Are all release blockers automatically blocking the next > beta? > Or does it mean this should not be released in final (and hence neither > rc) > versions? Would it make sense to release 3.11.0b4 with some not-yet-fixed > blockers? Assuming those are not regressions that happened after 3.11.0b3 > was > released. > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/X7HRFQHFRA5C7CMSVFXOODG45WNRZ5UH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [Release] Python 3.11.0b4 is still blocked
On 24. 06. 22 14:25, Pablo Galindo Salgado wrote: Hi everyone, A small update since the last communication from the release team regarding the status of Python 3.11.0b4. Unfortunately, even if we have fixed most of the original release blockers and 4 more that appear during this week, we still have a bunch of release blockers to deal with. One of them has been reported today. I would like to release the next beta next week if everything looks good, but there are also some items that need discussion... I was thinking. Are all release blockers automatically blocking the next beta? Or does it mean this should not be released in final (and hence neither rc) versions? Would it make sense to release 3.11.0b4 with some not-yet-fixed blockers? Assuming those are not regressions that happened after 3.11.0b3 was released. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MHTM5MREGJCCJPHLZZQF6W4FWDWGAIDU/ Code of Conduct: http://python.org/psf/codeofconduct/
