Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-13 Thread Georg Brandl
Am 04.06.2012 00:51, schrieb "Martin v. Löwis":
>> That last statement basically suggests that something like regex would
>> never be accepted until a CPython core developer was actually running
>> into pain with the many flaws in the re module (especially when it comes
>> to Unicode).  I disagree with that.
>>
>> Per the language summit, I think we need to just do it.  Put it in as re
>> and rename the existing re module to sre.
> 
> There are really places where "we" just doesn't work, even in a
> community project. "We" will never commit anything to revision control.
> Individual committers commit.
> 
> So if *you* want to commit it, go ahead - I think there is general
> approval for that. Take the praise when it works, and take the (likely)
> blame for when it fails in some significant way, and then work on fixing
> it.
> 
>> The issue seems to be primarily one of "who is volunteering to do it?"
> 
> I don't think anybody is, or will be for the coming years. I wish I had
> trust into MRAB to stay around and work on this for the next ten years
> (and I think the author of the regex module really needs to commit for
> that timespan, see SRE's history), but I don't. So whoever commits the
> change now is in charge, and will either have to work hard on fixing the
> problems, or will be responsible for breaking Python 3 in a serious way.

Agreed with all of the above.  (That's not a -1, but a warning.)

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-05 Thread MRAB

On 05/06/2012 03:40, Terry Reedy wrote:

On 6/4/2012 9:22 PM, MRAB wrote:


 I'm not planning any further changes to regex. I think it already has
 enough features...


Do you have any idea where regex + Python stands in regard to Unicode
TR18 support levels? http://unicode.org/reports/tr18/
While most of the Tailored Support Level 3 strikes me as out of scope
for the stdlib, I can imagine getting requests for any other stuff not
already included.


It supports Basic Unicode Support (Level 1), plus default word
boundaries and \X (grapheme cluster).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread Terry Reedy

On 6/4/2012 9:22 PM, MRAB wrote:


I'm not planning any further changes to regex. I think it already has
enough features...


Do you have any idea where regex + Python stands in regard to Unicode 
TR18 support levels? http://unicode.org/reports/tr18/
While most of the Tailored Support Level 3 strikes me as out of scope 
for the stdlib, I can imagine getting requests for any other stuff not 
already included.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread MRAB

On 05/06/2012 01:31, Steven D'Aprano wrote:

MRAB wrote:


 I personally am no longer that bothered about whether the regex module
 makes it into stdlib, but I am still be maintaining it on PyPI. If
 someone else wants to integrate it I would, of course, be willing to
 help out.


Are you withdrawing your offer to maintain regex in the stdlib?



 As long as they basically have the same source code, any bugs or other
 problems in the integrated module would be shared by the separate
 module, and I would want to fix them, so any fix in the separate module
 could be replicated easily in the integrated module.


But changes to the stdlib (bug fixes or functional changes) are very likely to
run at a slower pace to what third-party packages can afford. If you continue
to develop regex outside of the stdlib, that could cause complications.


I'm not planning any further changes to regex. I think it already has
enough features...

That just leaves 1) bug fixes, which you'd also want fixed in the
stdlib, and 2) functional changes requested for the stdlib, which you'd
presumably also want in the third-party package for those using earlier
versions of Python.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread Brian Curtin
On Mon, Jun 4, 2012 at 7:31 PM, Steven D'Aprano  wrote:
> But changes to the stdlib (bug fixes or functional changes) are very likely
> to run at a slower pace to what third-party packages can afford. If you
> continue to develop regex outside of the stdlib, that could cause
> complications.

Developing outside of the standard library isn't an option. You could
always backport things to the external version like the unittest2
project, but standard library modules need to be grown and fixed first
in the standard library.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread Steven D'Aprano

MRAB wrote:


I personally am no longer that bothered about whether the regex module
makes it into stdlib, but I am still be maintaining it on PyPI. If
someone else wants to integrate it I would, of course, be willing to
help out.


Are you withdrawing your offer to maintain regex in the stdlib?



As long as they basically have the same source code, any bugs or other
problems in the integrated module would be shared by the separate
module, and I would want to fix them, so any fix in the separate module
could be replicated easily in the integrated module.


But changes to the stdlib (bug fixes or functional changes) are very likely to 
run at a slower pace to what third-party packages can afford. If you continue 
to develop regex outside of the stdlib, that could cause complications.



--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread MRAB

(I've been having trouble with my email recently, so I missed this
thread amongst others.)

I personally am no longer that bothered about whether the regex module
makes it into stdlib, but I am still be maintaining it on PyPI. If
someone else wants to integrate it I would, of course, be willing to
help out.

As long as they basically have the same source code, any bugs or other
problems in the integrated module would be shared by the separate
module, and I would want to fix them, so any fix in the separate module
could be replicated easily in the integrated module.

It already works with Python 3.3 alpha, including PEP 393, BTW.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-04 Thread Martin v. Löwis

On 04.06.2012 01:28, Nick Coghlan wrote:

I apologise, "unmaintained" is too strong a word. I mean "lacking an
owner sufficiently confident in their authority and expertise and with
sufficient time and energy to add,or approve the addition of,
substantial new features which may require significant refactoring of
internal details".

Perhaps "unowned" would be a better word? Saying yes or no to major
feature requests isn't the same as fixing errors in existing features.
(Compare regular email package maintenance to RDM's recent updates)


I see the same risk for regex. Maybe somebody steps forward and 
integrates the code, but I doubt that someone would then "own" the

code in the sense you refer to, i.e. decide on major new features, or
perform a significant internal refactoring. It would all be up to
MRAB.

Also, there is a chance that a maintainer for SRE may come back.
Gustavo Niemeyer had that role for some time after /F left, and
anybody sufficiently interested in a specific new feature might
grow into that role as well.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Steven D'Aprano

Gregory P. Smith wrote:


Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.


I thought that the plan was to put the regex module in as regex, leaving re 
unchanged for backwards compatibility, with any backwards-incompatible 
renaming to be done some time in the future.




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Nick Coghlan
I apologise, "unmaintained" is too strong a word. I mean "lacking an owner
sufficiently confident in their authority and expertise and with sufficient
time and energy to add,or approve the addition of, substantial new features
which may require significant refactoring of internal details".

Perhaps "unowned" would be a better word? Saying yes or no to major feature
requests isn't the same as fixing errors in existing features. (Compare
regular email package maintenance to RDM's recent updates)

--
Sent from my phone, thus the relative brevity :)
On Jun 4, 2012 8:55 AM, Martin v. Löwis  wrote:

> heh.  indeed.  regardless, the module is available on pypi whether it
>> goes in or not so we do at least have something to point people to when
>> they need more than the existing undermaintained re (sre) module.
>>
>
> I completely disagree that SRE is unmaintained. It has about monthly
> commits to it, to fix reported bugs, by various people.
>
> It may be aged software, but that has the advantage that more people
> are familiar with the code base now than back in the days when /F
> was still maintaining it.
>
> Regards,
> Martin
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Martin v. Löwis

heh.  indeed.  regardless, the module is available on pypi whether it
goes in or not so we do at least have something to point people to when
they need more than the existing undermaintained re (sre) module.


I completely disagree that SRE is unmaintained. It has about monthly 
commits to it, to fix reported bugs, by various people.


It may be aged software, but that has the advantage that more people
are familiar with the code base now than back in the days when /F
was still maintaining it.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Martin v. Löwis

That last statement basically suggests that something like regex would
never be accepted until a CPython core developer was actually running
into pain with the many flaws in the re module (especially when it comes
to Unicode).  I disagree with that.

Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.


There are really places where "we" just doesn't work, even in a
community project. "We" will never commit anything to revision control.
Individual committers commit.

So if *you* want to commit it, go ahead - I think there is general
approval for that. Take the praise when it works, and take the (likely)
blame for when it fails in some significant way, and then work on fixing
it.


The issue seems to be primarily one of "who is volunteering to do it?"


I don't think anybody is, or will be for the coming years. I wish I had
trust into MRAB to stay around and work on this for the next ten years
(and I think the author of the regex module really needs to commit for
that timespan, see SRE's history), but I don't. So whoever commits the
change now is in charge, and will either have to work hard on fixing the
problems, or will be responsible for breaking Python 3 in a serious way.

That's why nobody volunteers.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Gregory P. Smith
On Sun, Jun 3, 2012 at 2:38 PM, Nick Coghlan  wrote:

> On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith  wrote:
> >
> > On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:
> >>
> >> ipaddress really made it in because I personally ran into the
> limitations
> >> of not having IP address support in the stdlib. I ended up doing quite
> a bit
> >> of prompting to ensure the process of cleaning up the API to modern
> stdlib
> >> standards didn't stall (even now, generating a module reference from the
> >> docstrings is still a pending task)
> >>
> >> With regex, the pain isn't there, since re already covers such a large
> >> subset of what regex provides.
> >
> > That last statement basically suggests that something like regex would
> never
> > be accepted until a CPython core developer was actually running into pain
> > with the many flaws in the re module (especially when it comes to
> Unicode).
> >  I disagree with that.
>
> No, that's not really what I meant. Driving integration of a module
> takes *time* and *effort*. The decision to commit that effort has to
> be driven by something, and personal annoyance is a great motivator.
> In the case of PEP 3144, I happened to be in a position to do
> something about a gap in the standard library after the omission was
> made glaringly obvious [1].
>
> Getting this done was a combined effort from Peter (in getting the
> module API updated), myself and others (esp. Antoine) in reviewing the
> reference implementation's API and requesting changes and more
> recently Sandro Tosi has been doing most of the heavy lifting in
> getting the docs up to scratch.
>
> > Per the language summit, I think we need to just do it.  Put it in as re
> and
> > rename the existing re module to sre.
>
> No. We almost burned Jesse out dropping multiprocessing into 2.6 at
> the last minute, and many longstanding issues with that module are
> only being addressed now that Richard has the time to be involved
> again. SRE already suffers from a lack of maintenance, and we've had
> zero indication that regex will make that situation better (and
> several indications that it will actually make it worse. Matthew's
> silence on the topic is *not* encouraging, and nobody else has even
> volunteered to write a PEP, let alone agree to maintain the module).
>
> > We could pull the plug on it and leave it out if substantial as yet
> unknown
> > problems that can't be fixed in time for release crop up during the beta
> 1
> > or 2 (release manager's decision).
>
> Unwinding changes to the build process is yet more work that may not
> be needed. We need to remember the purpose of the standard library:
> most of the time, it is *not* intended to be all things to all people.
> The status quo is that, if you're doing basic, primarily ASCII,
> regular expression processing, then "import re" will serve you just
> fine. If you're doing more than that, then you'll probably need to do
> "pip install regex" (or platform specific equivalent) and change your
> import to "import regex as re".
>
> That's not *great* (as the number of open Unicode bugs against SRE can
> attest), but it's far from unworkable. I consider it preferable to
> adding yet another big ball of C code to the stdlib in the absence of
> a PEP addressing the concerns already raised.
>
> >> My perspective is that it's now too late to make a change that big for
> >> 3.3, but the in principle approval holds for anyone that wants to work
> with
> >> MRAB and get the idea written up as a PEP for 3.4.
> >
> > Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
> > according to PEP 398) it can go in.
> >
> > I don't like to claim that a PEP for this one is strictly necessary
>
> Why not? Requiring a PEP is the norm, not the exception. Even when
> there's agreement that something *should* be done, there's plenty of
> details to be thrashed out in turning in principle agreement into a
> concrete plan of action.
>
> > but Nick
> > raises good questions to be answered and has good suggestions for what to
> > write up in the PEP in his earlier response that I certainly would
> prefer to
> > have gathered up and documented so that is the route I suggest.
> >
> > The issue seems to be primarily one of "who is volunteering to do it?"
>
> Correct, both in figuring out the integration details and in agreeing
> to maintain it in the future.
>
> Remember, now is better than never, but never is often better than
> *right* now :)
>
>
heh.  indeed.  regardless, the module is available on pypi whether it goes
in or not so we do at least have something to point people to when they
need more than the existing undermaintained re (sre) module.

There are also other options with different properties such as
http://pypi.python.org/pypi/re2/.

-gps
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/

Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Nick Coghlan
On Mon, Jun 4, 2012 at 6:25 AM, Gregory P. Smith  wrote:
>
> On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:
>>
>> ipaddress really made it in because I personally ran into the limitations
>> of not having IP address support in the stdlib. I ended up doing quite a bit
>> of prompting to ensure the process of cleaning up the API to modern stdlib
>> standards didn't stall (even now, generating a module reference from the
>> docstrings is still a pending task)
>>
>> With regex, the pain isn't there, since re already covers such a large
>> subset of what regex provides.
>
> That last statement basically suggests that something like regex would never
> be accepted until a CPython core developer was actually running into pain
> with the many flaws in the re module (especially when it comes to Unicode).
>  I disagree with that.

No, that's not really what I meant. Driving integration of a module
takes *time* and *effort*. The decision to commit that effort has to
be driven by something, and personal annoyance is a great motivator.
In the case of PEP 3144, I happened to be in a position to do
something about a gap in the standard library after the omission was
made glaringly obvious [1].

Getting this done was a combined effort from Peter (in getting the
module API updated), myself and others (esp. Antoine) in reviewing the
reference implementation's API and requesting changes and more
recently Sandro Tosi has been doing most of the heavy lifting in
getting the docs up to scratch.

> Per the language summit, I think we need to just do it.  Put it in as re and
> rename the existing re module to sre.

No. We almost burned Jesse out dropping multiprocessing into 2.6 at
the last minute, and many longstanding issues with that module are
only being addressed now that Richard has the time to be involved
again. SRE already suffers from a lack of maintenance, and we've had
zero indication that regex will make that situation better (and
several indications that it will actually make it worse. Matthew's
silence on the topic is *not* encouraging, and nobody else has even
volunteered to write a PEP, let alone agree to maintain the module).

> We could pull the plug on it and leave it out if substantial as yet unknown
> problems that can't be fixed in time for release crop up during the beta 1
> or 2 (release manager's decision).

Unwinding changes to the build process is yet more work that may not
be needed. We need to remember the purpose of the standard library:
most of the time, it is *not* intended to be all things to all people.
The status quo is that, if you're doing basic, primarily ASCII,
regular expression processing, then "import re" will serve you just
fine. If you're doing more than that, then you'll probably need to do
"pip install regex" (or platform specific equivalent) and change your
import to "import regex as re".

That's not *great* (as the number of open Unicode bugs against SRE can
attest), but it's far from unworkable. I consider it preferable to
adding yet another big ball of C code to the stdlib in the absence of
a PEP addressing the concerns already raised.

>> My perspective is that it's now too late to make a change that big for
>> 3.3, but the in principle approval holds for anyone that wants to work with
>> MRAB and get the idea written up as a PEP for 3.4.
>
> Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
> according to PEP 398) it can go in.
>
> I don't like to claim that a PEP for this one is strictly necessary

Why not? Requiring a PEP is the norm, not the exception. Even when
there's agreement that something *should* be done, there's plenty of
details to be thrashed out in turning in principle agreement into a
concrete plan of action.

> but Nick
> raises good questions to be answered and has good suggestions for what to
> write up in the PEP in his earlier response that I certainly would prefer to
> have gathered up and documented so that is the route I suggest.
>
> The issue seems to be primarily one of "who is volunteering to do it?"

Correct, both in figuring out the integration details and in agreeing
to maintain it in the future.

Remember, now is better than never, but never is often better than
*right* now :)

Cheers,
Nick.

[1] 
http://git.fedorahosted.org/git/?p=pulpdist.git;a=blob;f=src/pulpdist/core/validation.py;h=ebccf354c5bbec376258681a345fb73129eeeb95;hb=736250d85b758a11e1d09f70ec3877d1c022aa9a#l77


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Gregory P. Smith
On Fri, Jun 1, 2012 at 5:37 PM, Nick Coghlan  wrote:

> ipaddress really made it in because I personally ran into the limitations
> of not having IP address support in the stdlib. I ended up doing quite a
> bit of prompting to ensure the process of cleaning up the API to modern
> stdlib standards didn't stall (even now, generating a module reference from
> the docstrings is still a pending task)
>
> With regex, the pain isn't there, since re already covers such a large
> subset of what regex provides.
>
That last statement basically suggests that something like regex would
never be accepted until a CPython core developer was actually running into
pain with the many flaws in the re module (especially when it comes to
Unicode).  I disagree with that.

Per the language summit, I think we need to just do it.  Put it in as re
and rename the existing re module to sre.

We could pull the plug on it and leave it out if substantial as yet unknown
problems that can't be fixed in time for release crop up during the beta 1
or 2 (release manager's decision).

> My perspective is that it's now too late to make a change that big for
> 3.3, but the in principle approval holds for anyone that wants to work with
> MRAB and get the idea written up as a PEP for 3.4.
>
Nonsense, as long as its in before 3.3 Beta 1 (scheduled for June 23rd
according to PEP 398) it can go in.

I don't like to claim that a PEP for this one is *strictly* necessary but
Nick raises good questions to be answered and has good suggestions for what
to write up in the PEP in his earlier response that I certainly would
prefer to have gathered up and documented so that is the route I suggest.

The issue seems to be primarily one of "who is volunteering to do it?"

-gps
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-03 Thread Martin v. Löwis

On 02.06.2012 02:37, Mark Lawrence wrote:

On 01/06/2012 18:27, Brett Cannon wrote:

About the only thing I can think of from the language summit that we
discussed doing for Python 3.3 that has not come about is accepting the
regex module and getting it into the stdlib. Is this still being worked
towards?



Umpteen versions of regex have been available on pypi for years. Umpteen
bugs against the original re module have been fixed. If regex can't now
go into the standard library, what on earth can?


Something that isn't that big so that a maintainer can really read all 
of it.


I really wish the bug fixes had been made to SRE, instead of rewriting 
it all. So I'm -0 on this regex module. If this isn't added to 3.3,

I'll start encouraging people to contribute changes to SRE for 3.4, and
just ignore the regex module.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Nick Coghlan
On Sat, Jun 2, 2012 at 10:37 AM, Mark Lawrence  wrote:
> On 01/06/2012 18:27, Brett Cannon wrote:
>>
>> About the only thing I can think of from the language summit that we
>> discussed doing for Python 3.3 that has not come about is accepting the
>> regex module and getting it into the stdlib. Is this still being worked
>> towards?
>>
>
> Umpteen versions of regex have been available on pypi for years. Umpteen
> bugs against the original re module have been fixed.  If regex can't now go
> into the standard library, what on earth can?

That's why it's approved *in principle* already. However, it's not a
simple matter of dropping something into the standard library and
calling it done, especially an extension module as complex as regex.
Even integrating a simple pure Python module like ipaddr took
substantial effort:

1. The API had to be reviewed to see if it was suitable for someone
that was *not* familiar with the problem domain, but was instead
learning about it from the standard library documentation. This isn't
a big concern for regex, since it is replacing the existing re module,
but this is the main reason ipaddr became ipaddress before PEP 3144
was approved (The ipaddr API plays fast and loose with network
terminology in a way that someone that already *knows* that
terminology can easily grasp, but would have been incredibly confusing
to someone that is discovering those terms for the first time).

2. The code had to actually be added to the standard library (not a
big effort for PEP 3144 - saving ipaddress.py into Lib/ and
test_ipaddress.py into Lib/test/ pretty much covered it)

3. Redundant 2.x cruft needed to be weeded out (ongoing)

4. The howto guide needed to be incorporated into the documentation
(and rewritten to be more suitable for genuine beginners)

5. An API module reference still needs to be incorporated into the
standard library reference

The effort to integrate regex is going to be substantially higher,
since it's a significantly more complicated module:

1. A new, non-trivial C extension needs to be incorporated into both
the autotools and Windows build processes
2. Due to PEP 393, there's a major change to the string implementation
in 3.3. Does regex still build against that? Even if it builds, it
should probably be ported to the new API for performance reasons.
3. Does regex build cleanly on all platforms supported by CPython? If
not, do we need to keep the existing re module around as a fallback
mechanism?
4. How do we merge the test suites? Do we keep the existing test
suite, add the regex test suite, then filter for duplication
afterwards?
5. What, precisely, *are* the backwards incompatibilities between
regex and re? Does the standard library trigger any of them? Does the
test suite?
6. How will the PyPI backport be maintained in the future? The amount
of backwards compatibility cruft in standard library code should be
minimised, but that potentially makes backports more difficult.

ipaddress is in the 3.3 standard library because Peter Moody cared
enough about the concept to initially submit it for inclusion, and
because I volunteered to drive the review and integration process
forward and to be the final arbiter of what counted as "good enough"
for inclusion. That hasn't happened yet for regex - either nobody has
cared enough to write a PEP for it, or the bystander effect has kicked
in and everyone that cares is assuming *someone else* will take up the
burden of being the PEP champion.

So that's the first step: someone needs to take
http://bugs.python.org/issue2636 and turn it into a PEP (searching the
python-dev and python-ideas archives for references to previous
discussions of the topic would also be good, along with summarising
the open Unicode related re bugs reported by Tom Christensen where the
answer is currently "use regex from PyPI instead of the standard
library's re module" [1]).

[1] 
http://bugs.python.org/issue?%40search_text=&ignore=file%3Acontent&title=&%40columns=title&id=&%40columns=id&stage=&creation=&creator=tchrist&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Brian Curtin
On Fri, Jun 1, 2012 at 7:37 PM, Mark Lawrence  wrote:
> Umpteen versions of regex have been available on pypi for years. Umpteen
> bugs against the original re module have been fixed.  If regex can't now go
> into the standard library, what on earth can?

Reviewing a 4000 line patch is probably the main roadblock here.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Mark Lawrence

On 01/06/2012 18:27, Brett Cannon wrote:

About the only thing I can think of from the language summit that we
discussed doing for Python 3.3 that has not come about is accepting the
regex module and getting it into the stdlib. Is this still being worked
towards?



Umpteen versions of regex have been available on pypi for years. 
Umpteen bugs against the original re module have been fixed.  If regex 
can't now go into the standard library, what on earth can?


--
Cheers.

Mark Lawrence.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Nick Coghlan
ipaddress really made it in because I personally ran into the limitations
of not having IP address support in the stdlib. I ended up doing quite a
bit of prompting to ensure the process of cleaning up the API to modern
stdlib standards didn't stall (even now, generating a module reference from
the docstrings is still a pending task)

With regex, the pain isn't there, since re already covers such a large
subset of what regex provides.

My perspective is that it's now too late to make a change that big for 3.3,
but the in principle approval holds for anyone that wants to work with MRAB
and get the idea written up as a PEP for 3.4.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Brian Curtin
On Fri, Jun 1, 2012 at 1:57 PM, Terry Reedy  wrote:
> On 6/1/2012 1:27 PM, Brett Cannon wrote:
>>
>> About the only thing I can think of from the language summit that we
>> discussed doing for Python 3.3 that has not come about is accepting the
>> regex module and getting it into the stdlib. Is this still being worked
>> towards?
>
>
> Since there is no PEP to define the details* of such an addition, I assume
> that no particular core developer focused on this. There have been a lot of
> other additions to take people's attention. Also, I do not remember seeing
> anything from Mathew Barnett about his views on the proposal.
>
> * Some details:
>
> Replacement of or addition to re.

At the language summit it was proposed that this regex project would
enter as re, and the current re moves to sre. Everyone seemed to
agree.

> Relation to continued external project and 'ownership' of code.

As with anything else, no more external.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what is happening with the regex module going into Python 3.3?

2012-06-01 Thread Terry Reedy

On 6/1/2012 1:27 PM, Brett Cannon wrote:

About the only thing I can think of from the language summit that we
discussed doing for Python 3.3 that has not come about is accepting the
regex module and getting it into the stdlib. Is this still being worked
towards?


Since there is no PEP to define the details* of such an addition, I 
assume that no particular core developer focused on this. There have 
been a lot of other additions to take people's attention. Also, I do not 
remember seeing anything from Mathew Barnett about his views on the 
proposal.


* Some details:

Replacement of or addition to re.

Relation to continued external project and 'ownership' of code.

Relation, if any, to the Unicode Regular Expression Technical Standard, 
and its levels of conformance.

http://unicode.org/reports/tr18/


The addition of ipaddress was not a drop-in process. There have been 
some docstrings changes and clarifications, some code changes and 
cleanups, and removal of stuff only needed for 2.x. I suspect that regex 
would get at least as much tuning once seriously looked at.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com