Re: [Python-Dev] Generator objects and list comprehensions?

[Craig Rodrigues]

On Thu, Jan 26, 2017 at 4:09 AM, Ivan Levkivskyi 
wrote:

>
>
> Concerning list/set/dict comprehensions, I am much more in favor of making
> comprehensions simply equivalent to for-loops (more or less like you
> proposed using yield from). The only reason to introduce auxiliary function
> scope was to prevent the loop variables from leaking outside comprehensions.
> Formally, this is indeed backward incompatible, but I doubt many people
> depend on the current counter-intuitive behavior.
>
> Concerning generator expressions, probably it is indeed better to simply
> prohibit yield inside them.
>
>
Thank you to everyone who responded to my post and provided excellent
analysis.

For Python, I don't know what the best way to proceed is:

OPTION 1


Make a SyntaxError:  [(yield 1) for x in range(10)]
and update the documentation to explain that this is an invalid construct.

This would have certainly helped me identify the source of the problem as I
tried porting buildbot 0.9 to Python 3.

However, while not very common, there is Python 2.x code that uses that.
I found these cases in the buildbot code which I changed so as to work on
Python 2 and 3:

https://github.com/buildbot/buildbot/pull/2661
https://github.com/buildbot/buildbot/pull/2673


 OPTION 2
=
Make this return a list on Python 3, like in Python 2:  [
(yield 1) for x in range(10)]

As pointed out by others on the this mailing list, there are some
problems associated with that.  I don't know if there are many Python 2
codebases out there
with this construct, but it would be nice to have one less Python 2 -> 3
porting gotcha.


I'm OK with either approach.  Leaving things the way they are in Python 3
is no good, IMHO.
--
Craig
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Greg Ewing]

Armin Rigo wrote:


The theoretical kind of regexp is about giving a "yes/no" answer,
whereas the concrete "re" or "regexp" modules gives a match object,
which lets you ask for the subgroups' location, for example.

Another issue is that the theoretical engine has no notion of
greedy/non-greedy matching.


These things aren't part of the classical theory of REs that is
usually taught, but it should be possible to do them in linear time.
They can be done for context-free languages using e.g. an LALR parser,
and regular languages are a subset of context-free languages.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Serhiy Storchaka]

On 29.01.17 12:18, Jakub Wilk wrote:

* Armin Rigo , 2017-01-28, 12:44:

The theoretical kind of regexp is about giving a "yes/no" answer,
whereas the concrete "re" or "regexp" modules gives a match object,
which lets you ask for the subgroups' location, for example. Strange
at it may seem, I am not aware of a way to do that using the
linear-time approach of the theory---if it answers "yes", then you
have no way of knowing *where* the subgroups matched.

Another issue is that the theoretical engine has no notion of
greedy/non-greedy matching.


RE2 has linear execution time, and it supports both capture groups and
greedy/non-greedy matching.

The implementation is explained in this article:
https://swtch.com/~rsc/regexp/regexp3.html


Not all features of Python regular expressions can be implemented with 
linear complexity. It is possible to compile the part of regular 
expressions to the implementation with linear complexity. Patches are 
welcome.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Serhiy Storchaka]

On 29.01.17 22:30, Steve Holden wrote:

Why not declare re deprecated and remove it in Python 4? I am pretty
sure everyone wants to keep re in all 3.x releases, but that support
need not extend beyond. So Py4 would have no battery for re, but it
would (should!) be common knowledge that regex was the go-to module for
general-purpose pattern matching. If re has advantages in certain
situations someone might upgrade the 3.x implementation and provide it
as a 3rd-party module, though the effort involved would be significant,
so someone would have to be motivated to keep it.


Regular expressions are used in a number of standard modules and 
scripts. Excluding them from the stdlib would require excluding or 
rewriting the large part of the stdlib.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Paul Moore]

On 29 January 2017 at 20:30, Steve Holden  wrote:
> Why not declare re deprecated and remove it in Python 4? I am pretty sure
> everyone wants to keep re in all 3.x releases, but that support need not
> extend beyond. So Py4 would have no battery for re, but it would (should!)
> be common knowledge that regex was the go-to module for general-purpose
> pattern matching. If re has advantages in certain situations someone might
> upgrade the 3.x implementation and provide it as a 3rd-party module, though
> the effort involved would be significant, so someone would have to be
> motivated to keep it.

Not having regex capability distributed with Python is a pretty big
regression. There are still a lot of users who don't have access to
3rd party modules.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Steve Holden]

Why not declare re deprecated and remove it in Python 4? I am pretty sure
everyone wants to keep re in all 3.x releases, but that support need not
extend beyond. So Py4 would have no battery for re, but it would (should!)
be common knowledge that regex was the go-to module for general-purpose
pattern matching. If re has advantages in certain situations someone might
upgrade the 3.x implementation and provide it as a 3rd-party module, though
the effort involved would be significant, so someone would have to be
motivated to keep it.

regards
 Steve

Steve Holden

On Sun, Jan 29, 2017 at 4:13 PM, Antoine Pitrou  wrote:

> On Sat, 28 Jan 2017 12:07:05 -0500
> Barry Warsaw  wrote:
> > On Jan 28, 2017, at 03:43 PM, Nick Coghlan wrote:
> >
> > >I still think it could be a good candidate for a first "bundled"
> > >module, where we don't migrate it fully into the CPython development
> > >process, but *do* officially bless it and provide it by default in the
> > >form of a bundled wheel file (similar to what we do with pip).
> >
> > How would that work exactly.  I.e. is there a PEP?
> >
> > While I think it could be a good idea to bundle (more?) third party
> packages
> > for a variety of reasons, I want to make sure it's done in a way that's
> still
> > friendly to downstream repackagers, as I'm sure you do to. :)
>
> That sounds like a lot of effort and maintenance...  Don't we bundle pip
> *exactly* so that we don't have to bundle other third-party packages
> and instead tell users to "just use `pip install `"?
>
> To sum it up, how about we simply add an official suggestion to use
> regex in the docs?
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> steve%40holdenweb.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Antoine Pitrou]

On Sat, 28 Jan 2017 12:07:05 -0500
Barry Warsaw  wrote:
> On Jan 28, 2017, at 03:43 PM, Nick Coghlan wrote:
> 
> >I still think it could be a good candidate for a first "bundled"
> >module, where we don't migrate it fully into the CPython development
> >process, but *do* officially bless it and provide it by default in the
> >form of a bundled wheel file (similar to what we do with pip).  
> 
> How would that work exactly.  I.e. is there a PEP?
> 
> While I think it could be a good idea to bundle (more?) third party packages
> for a variety of reasons, I want to make sure it's done in a way that's still
> friendly to downstream repackagers, as I'm sure you do to. :)

That sounds like a lot of effort and maintenance...  Don't we bundle pip
*exactly* so that we don't have to bundle other third-party packages
and instead tell users to "just use `pip install `"?

To sum it up, how about we simply add an official suggestion to use
regex in the docs?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Jakub Wilk]

* Armin Rigo , 2017-01-28, 12:44:
The theoretical kind of regexp is about giving a "yes/no" answer, whereas the 
concrete "re" or "regexp" modules gives a match object, which lets you ask for 
the subgroups' location, for example. Strange at it may seem, I am not aware 
of a way to do that using the linear-time approach of the theory---if it 
answers "yes", then you have no way of knowing *where* the subgroups matched.


Another issue is that the theoretical engine has no notion of 
greedy/non-greedy matching.


RE2 has linear execution time, and it supports both capture groups and 
greedy/non-greedy matching.


The implementation is explained in this article:
https://swtch.com/~rsc/regexp/regexp3.html

--
Jakub Wilk
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com