Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
On 02.06.2012 02:37, Mark Lawrence wrote: On 01/06/2012 18:27, Brett Cannon wrote: About the only thing I can think of from the language summit that we discussed doing for Python 3.3 that has not come about is accepting the regex module and getting it into the stdlib. Is this still being worked towards? Umpteen versions of regex have been available on pypi for years. Umpteen bugs against the original re module have been fixed. If regex can't now go into the standard library, what on earth can? Something that isn't that big so that a maintainer can really read all of it. I really wish the bug fixes had been made to SRE, instead of rewriting it all. So I'm -0 on this regex module. If this isn't added to 3.3, I'll start encouraging people to contribute changes to SRE for 3.4, and just ignore the regex module. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On 01.06.2012 19:33, Brett Cannon wrote: Are these dead in the water or are we going to try to change our release cycle? I'm just asking since 3.3 final is due out in about 3 months and deciding on this along with shifting things if we do make a change could end up taking that long and I suspect if we don't do this for 3.3 we are probably never going to do it for Python 3 series as a whole. I'm -1 on both PEPs. For PEP 407, I fail to see what problem it solves. The PEP is short on rationale, so let me guess what the motivation for the PEP is: - Some people (Barry in particular) are in favor of timed releases. I don't know what the actual motivation for timed releases is, but ISTM that PEP 407 is an attempt to make Python generate timed releases. I'm -1 on that because of the additional effort for release managers. In particular, a strict schedule will limit vacation time, and require the release team to coordinate their vacation plans. With two alpha, one beta, and one rc, plus LTS bugfix releases, there may well be one release of some Python version every month. - Some contributors are worried about getting their contributions out, and some core committers are worried that we get fewer contributions because of that. While I well recall the feeling of getting changes out, the real concerns only exist for the very first contribution: * Those gurus on python-dev are certainly working on a fix for this very important issue already, how could they not have noticed? My work will be futile, and they'll fix it the day before I submit the patch. * Now that the patch is uploaded, can somebody *please* review it? How hard can it be to look over 20 lines of code? * Now that they committed it, when can I start telling my friends about it? The next release takes ages, and waiting is not fun. While these concerns are all real, it's really a matter of contributor education to deal with them, The longer people contribute to open source (or participate in any kind of software development), the more they learn that this is just how things work. The PEP really only addresses the third concern, whereas I think that the second is much more relevant. As for us not getting enough contributions: can we please worry about that when we have all patches processed that already have been contributed? I also think that the PEP will have negative effect on Python users: incompatible changes will spread faster (people will think that it's ok to break stuff since it was announced for three releases, when it wasn't actually announced in the last LTS). Users will feel the urgency of updating, and at the same time uneasiness about doing so as it may break stuff. People *already* get behind by two or three releases (in the 2.x series), getting behind 10 releases just will make them feel sad. For PEP 413, much the same concerns apply. In addition, I think it's too complicated, both for users, and for the actual technical implementation. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] JITted regex engine from pypy
Hi I was reading a bit about the regex module and I would like to present some other solution into speeding up the re module for Python. So, as a bit of background - pypy has a re compatible module. It's also JITted and it's also exportable as a C library (that is a library you can call from C with C API, not a python extension module). I wonder if it would be worth to put some work into it to make it a library that CPython can use. On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
Am 03.06.12 13:22, schrieb Martin v. Löwis: - Some contributors are worried about getting their contributions out, and some core committers are worried that we get fewer contributions because of that. While I well recall the feeling of getting changes out, the real concerns only exist for the very first contribution: * Those gurus on python-dev are certainly working on a fix for this very important issue already, how could they not have noticed? My work will be futile, and they'll fix it the day before I submit the patch. * Now that the patch is uploaded, can somebody *please* review it? How hard can it be to look over 20 lines of code? * Now that they committed it, when can I start telling my friends about it? The next release takes ages, and waiting is not fun. While these concerns are all real, it's really a matter of contributor education to deal with them, The longer people contribute to open source (or participate in any kind of software development), the more they learn that this is just how things work. The PEP really only addresses the third concern, whereas I think that the second is much more relevant. As a newish core developer I’d like to stress that Martin is 100% right here. Point three was never an issue to me – the biggest satisfaction is seeing the actual commit with the own name and the appearing in ACKS – you _can_ already tell your friends/tweet/blog about it at this point. And people do. OTOH point two is _very_ frustrating. The most colorful bikeshed is still much better than ignored patches. Personally, I gave up on CPython after my patches languished for weeks until Antoine revived the tickets three months later. I'm sure we've lost plenty of talent this way already and _if_ we want to attract more talented contributors, _this_ is the issue to tackle. The release process has nothing to do with that. I guess the PEPs (especially 413) are more about the bad rap the stdlib has been getting lately (e.g. http://www.leancrew.com/all-this/2012/04/where-modules-go-to-die/). As for us not getting enough contributions: can we please worry about that when we have all patches processed that already have been contributed? Realistically, that means never. Cheers, Hynek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
On Sun, Jun 3, 2012 at 7:49 AM, Maciej Fijalkowski fij...@gmail.com wrote: Hi I was reading a bit about the regex module and I would like to present some other solution into speeding up the re module for Python. So, as a bit of background - pypy has a re compatible module. It's also JITted and it's also exportable as a C library (that is a library you can call from C with C API, not a python extension module). I wonder if it would be worth to put some work into it to make it a library that CPython can use. On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? A few questions and comments about such an idea, from someone who hasn't used PyPy yet and doesn't understand the setup involved. 1) Would PyPy be required to build this as a C-compatible library, such that CPython could use it as an extension module? That is, would it make PyPy a required part of building CPython? 2) Are there benchmarks comparing the performance of this implementation to the existing re module and the proposed regex module? 3) How would the maintenance work? Where would the module live officially? Does CPython fork it or is it extracted from PyPy in a way it can be installed as an external dependency? How does CPython get changes upstream? 4) I may be remembering wrong, but I recall maintenance ease to be one of the justifications for the regex module. How would your proposal compare? Is a random developer looking to fix a bug in his way going to find this easier or more difficult to get his head around? The idea is interesting. Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
On Sun, Jun 3, 2012 at 3:06 PM, Calvin Spealman ironfro...@gmail.comwrote: On Sun, Jun 3, 2012 at 7:49 AM, Maciej Fijalkowski fij...@gmail.com wrote: Hi I was reading a bit about the regex module and I would like to present some other solution into speeding up the re module for Python. So, as a bit of background - pypy has a re compatible module. It's also JITted and it's also exportable as a C library (that is a library you can call from C with C API, not a python extension module). I wonder if it would be worth to put some work into it to make it a library that CPython can use. On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? A few questions and comments about such an idea, from someone who hasn't used PyPy yet and doesn't understand the setup involved. 1) Would PyPy be required to build this as a C-compatible library, such that CPython could use it as an extension module? That is, would it make PyPy a required part of building CPython? It depends a bit how we organize stuff. PyPy (as the pypy repository checkout, not the pypy interpreter) would be requires to build necessary C files (and as such also maintenance since the C files are not hand-editable), but pypy would not be required to compile C files. 2) Are there benchmarks comparing the performance of this implementation to the existing re module and the proposed regex module? I don't think so. It really is reasonably fast in a lot of cases and it can definitely be made faster in more cases. The main power comes from JITting - so you compile a piece of assembler per regex created. I doubt C library can come close to this approach-wise. Of course there will be cases and cases, but generally speaking the approach is superior. It would be cool if someone do the benchmarks how they look like *right now*. 3) How would the maintenance work? Where would the module live officially? Does CPython fork it or is it extracted from PyPy in a way it can be installed as an external dependency? How does CPython get changes upstream? I would honestly hope it can be maintained as a part of pypy and then cpython would just use it. But those are just hopes. 4) I may be remembering wrong, but I recall maintenance ease to be one of the justifications for the regex module. How would your proposal compare? Is a random developer looking to fix a bug in his way going to find this easier or more difficult to get his head around? I think it's relatively easy since it's python code after all, but what do I know. Someone has to have a look, it lives here - https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rsre I would like people to have opinions themselves whether it's more or less maintenance effort. On our side, we'll maintain this particular part of code anyway (so it's also easier because you leave it to others). Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
The embedded (in both senses of the term) use cases for CPython pretty much kill the idea, I'm afraid. Those cases are also one of the biggest question marks over incorporating regex wholesale instead of incrementally updating the existing engine to achieve feature parity. Publishing such a JIT compiled module via PyPI would be great, though. Cheers, Nick. -- Sent from my phone, thus the relative brevity :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
On Sun, Jun 3, 2012 at 3:46 PM, Nick Coghlan ncogh...@gmail.com wrote: The embedded (in both senses of the term) use cases for CPython pretty much kill the idea, I'm afraid. As I said it can (and should) definitely be optional. Those cases are also one of the biggest question marks over incorporating regex wholesale instead of incrementally updating the existing engine to achieve feature parity. Publishing such a JIT compiled module via PyPI would be great, though. Cheers, Nick. -- Sent from my phone, thus the relative brevity :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? The main concern for re is not speed, but functionality. The Python re module needs to grow a number of features, and correct a number of bugs. So 100% compatible is actually not good enough. 95% compatible (with the features added and the bugs fixed) would be better. OTOH, sharing the re code with PyPy would be a desirable goal, as would be writing the re code in Python (although SRE already implements significant parts in Python). As a speedup module, it's uninteresting - we want to simplify maintenance, not complicate it. So this can only work if it replaces SRE. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
On Sun, Jun 3, 2012 at 5:21 PM, Martin v. Löwis mar...@v.loewis.dewrote: On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? The main concern for re is not speed, but functionality. The Python re module needs to grow a number of features, and correct a number of bugs. So 100% compatible is actually not good enough. 95% compatible (with the features added and the bugs fixed) would be better. OTOH, sharing the re code with PyPy would be a desirable goal, as would be writing the re code in Python (although SRE already implements significant parts in Python). We did not reimplement those parts in RPython, they're still in python (so the sre engine does not accept regex, but instead the lower-level description, etc. etc.) As a speedup module, it's uninteresting - we want to simplify maintenance, not complicate it. So this can only work if it replaces SRE. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 3.3 on Windows 2000
It seems that by moving to VS 2010, we have killed Windows 2000 support, for the same reason VS 2012 would kill XP support: Windows 2000 apparently won't recognize the .exe files as executables anymore. I haven't actually tested this: can somebody please confirm? A year ago, Brian put a statement into PEP 11 that 3.3 would support Windows 2000 still, but with a warning. Under my recent PEP change, Windows 2000 does not need to be supported anymore since 13.07.2010, when Microsoft's extended support expired. So I propose to just remove the claim from the PEP that 3.3 would still be supported, and not go through any notification period. Objections? As a consequence, we could then change some of the deferred-loading stuff for new (i.e. XP+) API into proper linking. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On Jun 03, 2012, at 01:22 PM, Martin v. Löwis wrote: - Some people (Barry in particular) are in favor of timed releases. I don't know what the actual motivation for timed releases is, but Timed releases in general can provide much better predictability for others depending on those releases. E.g. folks working on things to go into Python can plan better how to make sure their stuff is ready in time, and downstreams can *much* better plan on which Python versions to include in their products and releases. Having said that, unless there's widespread consensus within the Python developers for timed releases, then it's not going to work, either within the context of those PEPs or not. After the last round of mostly negative feedback, I don't personally have much motivation to push these through. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
2012/6/3 Maciej Fijalkowski fij...@gmail.com: Hi I was reading a bit about the regex module and I would like to present some other solution into speeding up the re module for Python. IMO, the most important feature of the regex module is that it fixes long standing bugs and includes long requested features especially with respect to Unicode. That it's faster is only windfall. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On 6/3/2012 7:22 AM, Martin v. Löwis wrote: On 01.06.2012 19:33, Brett Cannon wrote: Are these dead in the water or are we going to try to change our release cycle? I'm just asking since 3.3 final is due out in about 3 months and deciding on this along with shifting things if we do make a change could end up taking that long and I suspect if we don't do this for 3.3 we are probably never going to do it for Python 3 series as a whole. I'm -1 on both PEPs. I pretty much agree. There is certainly no consensus and the possible benefit is not obviously substantially more than the cost. For PEP 407, I fail to see what problem it solves. The PEP is short on rationale, so let me guess what the motivation for the PEP is: ... While I well recall the feeling of getting changes out, the real concerns only exist for the very first contribution: ... * Now that the patch is uploaded, can somebody *please* review it? How hard can it be to look over 20 lines of code? Example http://bugs.python.org/issue13598 OP submitted revised patch in response to review 4 months ago As for us not getting enough contributions: can we please worry about that when we have all patches processed that already have been contributed? I suspect that having too many unattended patches sitting on the tracker discourages one from writing and submitting more. I also suspect, for instance, that applying some of Roger Serwy's Idle patches has encouraged him to write more. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.3 on Windows 2000
On 06/03/2012 08:32 AM, Martin v. Löwis wrote: So I propose to just remove the claim from the PEP that 3.3 would still be supported, and not go through any notification period. Did you mean s/3.3/Windows 2000/ ? //arry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.3 on Windows 2000
On 03.06.2012 18:18, Larry Hastings wrote: On 06/03/2012 08:32 AM, Martin v. Löwis wrote: So I propose to just remove the claim from the PEP that 3.3 would still be supported, and not go through any notification period. Did you mean s/3.3/Windows 2000/ ? I meant that 3.3 would support Windows 2000. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan ncogh...@gmail.com wrote: ipaddress really made it in because I personally ran into the limitations of not having IP address support in the stdlib. I ended up doing quite a bit of prompting to ensure the process of cleaning up the API to modern stdlib standards didn't stall (even now, generating a module reference from the docstrings is still a pending task) With regex, the pain isn't there, since re already covers such a large subset of what regex provides. That last statement basically suggests that something like regex would never be accepted until a CPython core developer was actually running into pain with the many flaws in the re module (especially when it comes to Unicode). I disagree with that. Per the language summit, I think we need to just do it. Put it in as re and rename the existing re module to sre. We could pull the plug on it and leave it out if substantial as yet unknown problems that can't be fixed in time for release crop up during the beta 1 or 2 (release manager's decision). My perspective is that it's now too late to make a change that big for 3.3, but the in principle approval holds for anyone that wants to work with MRAB and get the idea written up as a PEP for 3.4. Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd according to PEP 398) it can go in. I don't like to claim that a PEP for this one is *strictly* necessary but Nick raises good questions to be answered and has good suggestions for what to write up in the PEP in his earlier response that I certainly would prefer to have gathered up and documented so that is the route I suggest. The issue seems to be primarily one of who is volunteering to do it? -gps ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] JITted regex engine from pypy
Maciej Fijalkowski fij...@gmail.com wrote: On Sun, Jun 3, 2012 at 5:21 PM, Martin v. Löwis mar...@v.loewis.dewrote: On the minus side, the JIT only works on x86 and x86_64, on the plus side, since it's 100% API compatible, it can be used as a _xxx speedup module relatively easy. Do people have opinions? The main concern for re is not speed, but functionality. The Python re module needs to grow a number of features, and correct a number of bugs. So 100% compatible is actually not good enough. 95% compatible (with the features added and the bugs fixed) would be better. From my point of view, for textual data reduction, the MRAB regex now has substantial improvements which enable very different kinds of uses, like named lists and fuzzy matching, which I don't believe occur (together) in any other RE library. Along with features it shares with the existing CPython re library, such as the ability to handle very large RE's (which IronPython, for instance, is unable to handle, apparently due to its use of the standard .NET RE library). And do so fairly efficiently. Bill OTOH, sharing the re code with PyPy would be a desirable goal, as would be writing the re code in Python (although SRE already implements significant parts in Python). We did not reimplement those parts in RPython, they're still in python (so the sre engine does not accept regex, but instead the lower-level description, etc. etc.) As a speedup module, it's uninteresting - we want to simplify maintenance, not complicate it. So this can only work if it replaces SRE. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/bill%40janssen.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On Sun, Jun 3, 2012 at 9:22 PM, Martin v. Löwis mar...@v.loewis.de wrote: On 01.06.2012 19:33, Brett Cannon wrote: Are these dead in the water or are we going to try to change our release cycle? I'm just asking since 3.3 final is due out in about 3 months and deciding on this along with shifting things if we do make a change could end up taking that long and I suspect if we don't do this for 3.3 we are probably never going to do it for Python 3 series as a whole. I'm -1 on both PEPs. Unsurprisingly, I'm -1 on PEP 407. Perhaps surprisingly, I'm also -0 on my own PEP 413 (I wrote it to present what I consider a more tolerable alternative to an idea I really don't like) I think marking both as Rejected would be an accurate reflection of python-dev's collective opinion. For PEP 413, much the same concerns apply. In addition, I think it's too complicated, both for users, and for the actual technical implementation. Yup (although I think PEP 407 would need to be *at least* as complicated in practice as PEP 413 in order to make the implementation manageable, but currently glosses over the technical details). The one thing I actually *would* like to see change is for the cadence of *alpha* releases to be increased to match that of maintenance releases (that is, I'd like to see Python 3.4a1 released at the same time as Python 3.3.1: around 6 months after the release of 3.3.0). I think keeping the trunk closer to a releasable state will help encourage a more regular rate of contributions and provide earlier deadlines for big changes (e.g. it's significantly easier to say we want to have the compiler changes in place for 3.4a1 in April than it is to say we want to have these changes in place by April, but that's just an arbitrary point in time, since the nearest release deadline will still be at least 12 months away. Scheduling things like sprints and bug days also becomes more focused, since they have a nearer term goal of getting things fixed for an alpha release that's only a few months away rather than one that's more than a year out). It also lowers the bar for getting people to tinker with and provide feedback on new syntax like PEP 380 and core features like pyvenv and pysetup that behave differently when installed instead of being run from a source checkout. At the moment, the criteria for providing early feedback on new syntax is interested in the feature, and can build CPython from source, while the criteria for installed features is interested in the feature, can build CPython from source, and can install the result on a target system. Early alphas means that the criteria for providing feedback becomes simply: interested in the feature, and has access to a system that can tolerate having the alpha release installed. These alpha releases can also feed into vendor schemes such as Red Hat's tech preview program: while the system Python would always be a released version, an alpha version may still be an adequate foundation for a tech preview. As the other Python implementations catch up to the 3.x series, the alphas would also provide clear recommended synchronisation points that may make it easier for them to start targeting CPython release compatibility *before* we publish the final version. As I see it, such an approach would achieve most of the benefits of a regular release cadence with basically *none* of the seriously disruptive effects created by the more ambitious schemes described in PEP 407 or 413. I also consider it an excellent test run: if we can't even produce alpha releases of the upcoming version every 6 months or so, how on earth could we ever consider trying to create *production* releases on that schedule? Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On Mon, Jun 4, 2012 at 7:02 AM, Nick Coghlan ncogh...@gmail.com wrote: I think marking both as Rejected would be an accurate reflection of python-dev's collective opinion. Slight correction: I think it would accurately reflect python-dev's *divided* opinion, using the principle of Status quo wins a stalemate. The costs for either scheme are high, the benefits are not proven, thus the default is to stick with the status quo. Releasing alphas early, OTOH, doesn't require any real changes to our development process at all, aside from imposing a bit more discipline on trunk development in the first 12 months of the release cycle (I'm inclined to place that particular detail on the benefit side of the ledger, rather than the cost side). The *total* number of releases from the release managers and installer builders shouldn't increase much, if at all - I'd suggest we just stick with Georg's practice of 4 alpha releases, and merely space them out over the course of the release cycle rather than clustered together at the end. If Larry doesn't want to try this for 3.4, then I'll most likely volunteer as 3.5 RM and try it out then. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith g...@krypto.org wrote: On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan ncogh...@gmail.com wrote: ipaddress really made it in because I personally ran into the limitations of not having IP address support in the stdlib. I ended up doing quite a bit of prompting to ensure the process of cleaning up the API to modern stdlib standards didn't stall (even now, generating a module reference from the docstrings is still a pending task) With regex, the pain isn't there, since re already covers such a large subset of what regex provides. That last statement basically suggests that something like regex would never be accepted until a CPython core developer was actually running into pain with the many flaws in the re module (especially when it comes to Unicode). I disagree with that. No, that's not really what I meant. Driving integration of a module takes *time* and *effort*. The decision to commit that effort has to be driven by something, and personal annoyance is a great motivator. In the case of PEP 3144, I happened to be in a position to do something about a gap in the standard library after the omission was made glaringly obvious [1]. Getting this done was a combined effort from Peter (in getting the module API updated), myself and others (esp. Antoine) in reviewing the reference implementation's API and requesting changes and more recently Sandro Tosi has been doing most of the heavy lifting in getting the docs up to scratch. Per the language summit, I think we need to just do it. Put it in as re and rename the existing re module to sre. No. We almost burned Jesse out dropping multiprocessing into 2.6 at the last minute, and many longstanding issues with that module are only being addressed now that Richard has the time to be involved again. SRE already suffers from a lack of maintenance, and we've had zero indication that regex will make that situation better (and several indications that it will actually make it worse. Matthew's silence on the topic is *not* encouraging, and nobody else has even volunteered to write a PEP, let alone agree to maintain the module). We could pull the plug on it and leave it out if substantial as yet unknown problems that can't be fixed in time for release crop up during the beta 1 or 2 (release manager's decision). Unwinding changes to the build process is yet more work that may not be needed. We need to remember the purpose of the standard library: most of the time, it is *not* intended to be all things to all people. The status quo is that, if you're doing basic, primarily ASCII, regular expression processing, then import re will serve you just fine. If you're doing more than that, then you'll probably need to do pip install regex (or platform specific equivalent) and change your import to import regex as re. That's not *great* (as the number of open Unicode bugs against SRE can attest), but it's far from unworkable. I consider it preferable to adding yet another big ball of C code to the stdlib in the absence of a PEP addressing the concerns already raised. My perspective is that it's now too late to make a change that big for 3.3, but the in principle approval holds for anyone that wants to work with MRAB and get the idea written up as a PEP for 3.4. Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd according to PEP 398) it can go in. I don't like to claim that a PEP for this one is strictly necessary Why not? Requiring a PEP is the norm, not the exception. Even when there's agreement that something *should* be done, there's plenty of details to be thrashed out in turning in principle agreement into a concrete plan of action. but Nick raises good questions to be answered and has good suggestions for what to write up in the PEP in his earlier response that I certainly would prefer to have gathered up and documented so that is the route I suggest. The issue seems to be primarily one of who is volunteering to do it? Correct, both in figuring out the integration details and in agreeing to maintain it in the future. Remember, now is better than never, but never is often better than *right* now :) Cheers, Nick. [1] http://git.fedorahosted.org/git/?p=pulpdist.git;a=blob;f=src/pulpdist/core/validation.py;h=ebccf354c5bbec376258681a345fb73129eeeb95;hb=736250d85b758a11e1d09f70ec3877d1c022aa9a#l77 -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 405 (built-in virtualenv) status
Hi, Le 02/06/2012 12:59, Christian Tismer a écrit : One urgent question: will this feature be backported to Python 2.7? Features are never backported to the stable versions. virtualenv still exists as a standalone project which is compatible with 2.7 though. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
On Sun, Jun 3, 2012 at 2:38 PM, Nick Coghlan ncogh...@gmail.com wrote: On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith g...@krypto.org wrote: On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan ncogh...@gmail.com wrote: ipaddress really made it in because I personally ran into the limitations of not having IP address support in the stdlib. I ended up doing quite a bit of prompting to ensure the process of cleaning up the API to modern stdlib standards didn't stall (even now, generating a module reference from the docstrings is still a pending task) With regex, the pain isn't there, since re already covers such a large subset of what regex provides. That last statement basically suggests that something like regex would never be accepted until a CPython core developer was actually running into pain with the many flaws in the re module (especially when it comes to Unicode). I disagree with that. No, that's not really what I meant. Driving integration of a module takes *time* and *effort*. The decision to commit that effort has to be driven by something, and personal annoyance is a great motivator. In the case of PEP 3144, I happened to be in a position to do something about a gap in the standard library after the omission was made glaringly obvious [1]. Getting this done was a combined effort from Peter (in getting the module API updated), myself and others (esp. Antoine) in reviewing the reference implementation's API and requesting changes and more recently Sandro Tosi has been doing most of the heavy lifting in getting the docs up to scratch. Per the language summit, I think we need to just do it. Put it in as re and rename the existing re module to sre. No. We almost burned Jesse out dropping multiprocessing into 2.6 at the last minute, and many longstanding issues with that module are only being addressed now that Richard has the time to be involved again. SRE already suffers from a lack of maintenance, and we've had zero indication that regex will make that situation better (and several indications that it will actually make it worse. Matthew's silence on the topic is *not* encouraging, and nobody else has even volunteered to write a PEP, let alone agree to maintain the module). We could pull the plug on it and leave it out if substantial as yet unknown problems that can't be fixed in time for release crop up during the beta 1 or 2 (release manager's decision). Unwinding changes to the build process is yet more work that may not be needed. We need to remember the purpose of the standard library: most of the time, it is *not* intended to be all things to all people. The status quo is that, if you're doing basic, primarily ASCII, regular expression processing, then import re will serve you just fine. If you're doing more than that, then you'll probably need to do pip install regex (or platform specific equivalent) and change your import to import regex as re. That's not *great* (as the number of open Unicode bugs against SRE can attest), but it's far from unworkable. I consider it preferable to adding yet another big ball of C code to the stdlib in the absence of a PEP addressing the concerns already raised. My perspective is that it's now too late to make a change that big for 3.3, but the in principle approval holds for anyone that wants to work with MRAB and get the idea written up as a PEP for 3.4. Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd according to PEP 398) it can go in. I don't like to claim that a PEP for this one is strictly necessary Why not? Requiring a PEP is the norm, not the exception. Even when there's agreement that something *should* be done, there's plenty of details to be thrashed out in turning in principle agreement into a concrete plan of action. but Nick raises good questions to be answered and has good suggestions for what to write up in the PEP in his earlier response that I certainly would prefer to have gathered up and documented so that is the route I suggest. The issue seems to be primarily one of who is volunteering to do it? Correct, both in figuring out the integration details and in agreeing to maintain it in the future. Remember, now is better than never, but never is often better than *right* now :) heh. indeed. regardless, the module is available on pypi whether it goes in or not so we do at least have something to point people to when they need more than the existing undermaintained re (sre) module. There are also other options with different properties such as http://pypi.python.org/pypi/re2/. -gps ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 405 (built-in virtualenv) status
On Sun, Jun 3, 2012 at 3:33 AM, Christian Tismer tis...@stackless.com wrote: As an old windows guy, I very much agree with Kristjan. The venv approach is great. Windows is just a quite weird situation to handle in some cases, and a super-simple way to get rid of *any* built-in behavior concerning setup would be great. The idea of moving path setup stuff into the python.exe stub makes very much sense to me. This would make pythonxx.dll a really useful library to be shared. It's mainly Py_Initialize() that triggers the magic. What may be worth exploring is a variant on that which allows embedding applications to explicitly pass in *everything* that would otherwise be guessed by inspecting the environment. (Some things can be forced to particular values already, but certainly not everything). Python has IMHO too much behavior like this: 'by default, look into xxx, but if a yyy exists, behave differently'. I don't like this, because the absense of a simple file changes the whole system behavior. I would do it the other way round: As soon as you introduce the venv.cfg file, enforce its existence completely! If that file is not there, then python exits with an error message. This way you can safely ensure its existence, and the file can be made read-only and so on. A non-existent file is just a bad thing and is hard to make read-only ;-) So please let's abandon the old 'if exists ...' pattern, at least this one time. By the explicit cfg file, the file can clearly say if there is a virtual env or not. Backwards compatibility constraints mean we simply can't do that. However, as noted above, it may make sense to provide more ways for embedding applications to selectively access the behaviour through the C API. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 405 (built-in virtualenv) status
On 04.06.12 00:28, Nick Coghlan wrote: ... Backwards compatibility constraints mean we simply can't do that. However, as noted above, it may make sense to provide more ways for embedding applications to selectively access the behaviour through the C API. Why that??? I don't see this. If you have a new python version with a new file that has-to-be-there, what is then the problem? The new version carries the new file, so I don't see a compatibility issue, because this version does not want to be backward-compatible. It just introduces the new file constraint, and it produces what it needs. Am I somehow blinded, maybe? (yes, you all know that I am, so please be patient with me) -- Chris -- Christian Tismer :^)mailto:tis...@stackless.com tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key - http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
That last statement basically suggests that something like regex would never be accepted until a CPython core developer was actually running into pain with the many flaws in the re module (especially when it comes to Unicode). I disagree with that. Per the language summit, I think we need to just do it. Put it in as re and rename the existing re module to sre. There are really places where we just doesn't work, even in a community project. We will never commit anything to revision control. Individual committers commit. So if *you* want to commit it, go ahead - I think there is general approval for that. Take the praise when it works, and take the (likely) blame for when it fails in some significant way, and then work on fixing it. The issue seems to be primarily one of who is volunteering to do it? I don't think anybody is, or will be for the coming years. I wish I had trust into MRAB to stay around and work on this for the next ten years (and I think the author of the regex module really needs to commit for that timespan, see SRE's history), but I don't. So whoever commits the change now is in charge, and will either have to work hard on fixing the problems, or will be responsible for breaking Python 3 in a serious way. That's why nobody volunteers. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
heh. indeed. regardless, the module is available on pypi whether it goes in or not so we do at least have something to point people to when they need more than the existing undermaintained re (sre) module. I completely disagree that SRE is unmaintained. It has about monthly commits to it, to fix reported bugs, by various people. It may be aged software, but that has the advantage that more people are familiar with the code base now than back in the days when /F was still maintaining it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
I apologise, unmaintained is too strong a word. I mean lacking an owner sufficiently confident in their authority and expertise and with sufficient time and energy to add,or approve the addition of, substantial new features which may require significant refactoring of internal details. Perhaps unowned would be a better word? Saying yes or no to major feature requests isn't the same as fixing errors in existing features. (Compare regular email package maintenance to RDM's recent updates) -- Sent from my phone, thus the relative brevity :) On Jun 4, 2012 8:55 AM, Martin v. Löwis mar...@v.loewis.de wrote: heh. indeed. regardless, the module is available on pypi whether it goes in or not so we do at least have something to point people to when they need more than the existing undermaintained re (sre) module. I completely disagree that SRE is unmaintained. It has about monthly commits to it, to fix reported bugs, by various people. It may be aged software, but that has the advantage that more people are familiar with the code base now than back in the days when /F was still maintaining it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is happening with the regex module going into Python 3.3?
Gregory P. Smith wrote: Per the language summit, I think we need to just do it. Put it in as re and rename the existing re module to sre. I thought that the plan was to put the regex module in as regex, leaving re unchanged for backwards compatibility, with any backwards-incompatible renaming to be done some time in the future. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whither PEP 407 and 413 (release cycle PEPs)?
On Mon, Jun 4, 2012 at 7:11 AM, Nick Coghlan ncogh...@gmail.com wrote: On Mon, Jun 4, 2012 at 7:02 AM, Nick Coghlan ncogh...@gmail.com wrote: I think marking both as Rejected would be an accurate reflection of python-dev's collective opinion. Slight correction: I think it would accurately reflect python-dev's *divided* opinion, using the principle of Status quo wins a stalemate. The costs for either scheme are high, the benefits are not proven, thus the default is to stick with the status quo. Releasing alphas early, OTOH, doesn't require any real changes to our development process at all, aside from imposing a bit more discipline on trunk development in the first 12 months of the release cycle (I'm inclined to place that particular detail on the benefit side of the ledger, rather than the cost side). The *total* number of releases from the release managers and installer builders shouldn't increase much, if at all - I'd suggest we just stick with Georg's practice of 4 alpha releases, and merely space them out over the course of the release cycle rather than clustered together at the end. If Larry doesn't want to try this for 3.4, then I'll most likely volunteer as 3.5 RM and try it out then. After an off-list discussion with Larry, I'm now planning to expand on this concept in PEP form (superceding 413). There's actually a little bit more to it than just releasing the alphas early - it's about harnessing the power of external deadlines to help counter innate tendencies towards procrastination :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
