John added the comment:
Consider the following generator function similar to the one that started this
issue:
from __future__ import generator_stop
def myzip(*seqs):
its = (iter(seq) for seq in seqs)
while True:
try:
yield
John added the comment:
Please ignore my previous comment - now I understood what is going on. The
`its` generator is a one-shot iterator so is exhausted the first time the while
loop is run. Therefore, on subsequent loops, only empty tuples are yielded.
Still learning about generators... :)
Roundup Robot added the comment:
New changeset 2771a0ac806b by Yury Selivanov in branch 'default':
Issue 24237: Raise PendingDeprecationWarning per PEP 479
https://hg.python.org/cpython/rev/2771a0ac806b
--
___
Python tracker rep...@bugs.python.org
Roundup Robot added the comment:
New changeset c8a3e49f35e7 by Yury Selivanov in branch 'default':
docs: Mention PEP 479 in whatsnew.
https://hg.python.org/cpython/rev/c8a3e49f35e7
--
___
Python tracker rep...@bugs.python.org
Nick Coghlan added the comment:
We missed the deprecation warning part of the PEP (for when the future import
is *not* in effect, but the default behaviour will change in 3.7), but rather
than reopening this one, I filed a new issue: issue 24237
--
Roundup Robot added the comment:
New changeset 787cc3d1d3af by Yury Selivanov in branch 'default':
Issue #22906: Do incref before SetCause/SetContext
https://hg.python.org/cpython/rev/787cc3d1d3af
--
___
Python tracker rep...@bugs.python.org
Nick Coghlan added the comment:
Since it took me a moment to figure out why the extra incref was needed:
* both PyException_SetCause and PyException_SetContext steal a reference to
their second argument
* hence we need the second incref, rather than relying solely on the reference
received
Yury Selivanov added the comment:
Thanks Nick and Berker for the reviews!
--
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Changes by Yury Selivanov yseliva...@gmail.com:
--
stage: commit review - resolved
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Berker Peksag added the comment:
Buildbots are not happy:
[ 63/393] test_contextlib
Fatal Python error: Objects/frameobject.c:429 object at 0x200041abc28 has
negative ref count -2604246222170760230
Current thread 0x022c2500 (most recent call first):
File
Chris Angelico added the comment:
Weird. Tests ran fine on my machine too. Interestingly, that number is
0xdbdbdbdbdbdbdbda - does that mean anything? (It's negative
0x2424242424242426, for what that's worth.)
--
___
Python tracker
Roundup Robot added the comment:
New changeset 36a8d935c322 by Yury Selivanov in branch 'default':
PEP 479: Change StopIteration handling inside generators.
https://hg.python.org/cpython/rev/36a8d935c322
--
___
Python tracker rep...@bugs.python.org
Chris Angelico added the comment:
Thanks everyone for all the help getting this to land! This is going to be a
part of my active python3 binary from now on :)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Yury Selivanov added the comment:
Strange, the test suite was running just fine on my machine. I'll take a closer
look later today.
--
nosy: +Yury.Selivanov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Yury Selivanov added the comment:
I think it crashes in debug mode or something. Somewhere we did too many
decrefs.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Yury Selivanov added the comment:
Berker, buildbots should be happy now.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Roundup Robot added the comment:
New changeset 5d8bc813d270 by Yury Selivanov in branch 'default':
Issue 22906: Increment refcount after PyException_SetContext
https://hg.python.org/cpython/rev/5d8bc813d270
--
___
Python tracker
Roundup Robot added the comment:
New changeset d15c26085591 by Yury Selivanov in branch 'default':
Issue 22906: Add test file.
https://hg.python.org/cpython/rev/d15c26085591
--
___
Python tracker rep...@bugs.python.org
Berker Peksag added the comment:
Thanks!
--
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Python-bugs-list
Nick Coghlan added the comment:
A couple of minor comments:
* self.fail with an appropriate error message is a clearer way to
indicate an incorrect logic path has been hit in a test case
* the details of the exception chaining don't quite look right, as I
believe raise X from Y sets both the
Yury Selivanov added the comment:
Nick, Berker,
Please see the updated patch.
--
Added file: http://bugs.python.org/file39323/pep0479.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Nick Coghlan added the comment:
Latest patch LGTM, although I believe the new chaining behaviour checks
would be clearer with the 3 try/except blocks merged into a single block
where all 3 behaviours are checked in the same except clause, and the else
clause complains that StopIteration was
Berker Peksag added the comment:
In Lib/__future__.py:
+generator_stop = _Feature((3, 5, 0, alpha, 1),
alpha needs to be changed to beta.
--
stage: test needed - commit review
___
Python tracker rep...@bugs.python.org
Guido van Rossum added the comment:
I think you could help by (a) reviewing what's there, and (b) helping with
the implementation of __future__.
On Thu, May 7, 2015 at 1:16 PM, Yury Selivanov rep...@bugs.python.org
wrote:
Yury Selivanov added the comment:
You sure can! Take it, deploy it,
Yury Selivanov added the comment:
You sure can! Take it, deploy it, run the test suite, and then start writing
real code that uses it. When you find a problem, that's what needs help! :)
Thank you for this generic answer, Chris.
The reason I was asking is because issue #24017 depends on
Yury Selivanov added the comment:
Yury's patch mostly looks good to me, except:
Thanks!
* the check in contextlib should be against __cause__ rather than
__context__, and there should be a new test for this code handling path
Done. I've also added one test for correct handling of
Berker Peksag added the comment:
A minor comment about the __future__ changes: 3.5.0a1 should probably be
3.5.0b1.
--
nosy: +berker.peksag
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Chris Angelico added the comment:
The comment was general because I honestly had no idea what was needed still.
All I knew was that the patch seemed to work for me, all tests passing
(including the new one). Thanks for uploading the new patch; it compiles
happily, and I'm running tests now,
Yury Selivanov added the comment:
Hi,
Please find attached an updated patch.
Summary of changes:
1. Most of feedback from Nick Coghlan and Serhiy Storchaka is applied;
2. Changes in difflib.py were reverted (unless we add the __future__ import
there right now there is no need to fix it);
Changes by Yury Selivanov yseliva...@gmail.com:
--
assignee: - yselivanov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Nick Coghlan added the comment:
Yury's patch mostly looks good to me, except:
* the check in contextlib should be against __cause__ rather than
__context__, and there should be a new test for this code handling path
* there should be a new test for the __future__ flag itself (independently
of
Guido van Rossum added the comment:
Or, if it's perfect (or good enough :-), just check it in.
On Thu, May 7, 2015 at 1:18 PM, Guido van Rossum rep...@bugs.python.org
wrote:
Guido van Rossum added the comment:
I think you could help by (a) reviewing what's there, and (b) helping with
the
Yury Selivanov added the comment:
Hi Chris! Can I somehow help with the patch?
--
nosy: +yselivanov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Chris Angelico added the comment:
You sure can! Take it, deploy it, run the test suite, and then start writing
real code that uses it. When you find a problem, that's what needs help! :)
--
___
Python tracker rep...@bugs.python.org
Guido van Rossum added the comment:
Well that would break a lot of code...
On May 5, 2015 5:18 PM, STINNER Victor rep...@bugs.python.org wrote:
STINNER Victor added the comment:
Would it be possible to push the first part of the implementation (without
__future__) just to unblock the
STINNER Victor added the comment:
Would it be possible to push the first part of the implementation (without
__future__) just to unblock the implementation of the PEP 492 (issue #24017:
async/await)?
Later push the second part for __future__.
--
Chris Angelico added the comment:
Stinner, not sure what you mean by first part / second part. Is there a way for
me to withdraw the first two versions of the patch and just keep #37646?
--
___
Python tracker rep...@bugs.python.org
Chris Angelico added the comment:
Had a peek at the 2.7 branch in the web
(https://hg.python.org/cpython/file/4234b0dd2a54/Lib/test) and all the tests
appear to be testing the behaviour *with* the future directive, not bothering
to test the old behaviour. It makes sense - that way, when the
Nick Coghlan added the comment:
Unfortunately we didn't find the time to review the currently flagged release
blockers at the sprints :(
Chris, for the mechanics of testing the future flag, it's potentially worth
looking at the 2.7 branch, as that should have tests for the print_function and
Chris Angelico added the comment:
Simple test case for the future directive. Needs expansion.
--
Added file: http://bugs.python.org/file39079/test_pep479.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Guido van Rossum added the comment:
Did anyone look at this yet? The ultimate deadline would be May 24 (beta 1).
But making alpha 4 (April 19) would be better.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Changes by Nick Coghlan ncogh...@gmail.com:
--
priority: normal - release blocker
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Nick Coghlan added the comment:
Bumped to release blocker so we make sure to look at this at the PyCon sprints,
if not before. (I'm assuming Larry's not taking release blocker literally for
the 3.5 alphas)
As Neil notes, this patch needs some explicit tests for the new behaviour when
the
Changes by Ethan Furman et...@stoneleaf.us:
--
stage: patch review - test needed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Ethan Furman added the comment:
Thanks, Neil, for catching that.
I did run the entire test suite with the patch, and nothing new broke, so it
would seem the patch is at least benign. :)
--
nosy: +ethan.furman
___
Python tracker
Ethan Furman added the comment:
Oh, and my tests ran on Ubuntu 13.04 (GNU/Linux 3.8.0-22-generic x86_64).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Neil Girdhar added the comment:
FWIW I looked at the changes. Does it make sense to run tests before there are
actual tests in lib/Test? I'll happily run all tests when some new ones are
added.
--
nosy: +neil.g
___
Python tracker
Guido van Rossum added the comment:
Looking for code reviewers!
--
stage: - patch review
type: - enhancement
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Chris Angelico added the comment:
Nick, any particular reason for pointing to
https://hg.python.org/cpython/annotate/bbf16fd024df/Lib/__future__.py rather
than https://hg.python.org/cpython/annotate/tip/Lib/__future__.py ? I'm looking
at both, anyhow.
--
Nick Coghlan added the comment:
Looking more closely at the patch:
* for the missing tracebacks, you're probably hitting the note in
https://docs.python.org/3/c-api/exceptions.html#c.PyErr_NormalizeException and
need an explicit call to PyErr_SetTraceback (My recollection is that I made
this
Chris Angelico added the comment:
PyErr_Restore doesn't seem to trigger exception chaining. But thanks for the
tip about explicitly setting the traceback; not sure how I missed that, but now
the StopIteration traceback is visible.
Minor point: The previous patch was setting the __context__ of
Changes by Chris Angelico ros...@gmail.com:
Added file: http://bugs.python.org/file37641/stopiter.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Chris Angelico added the comment:
Okay! I think I have something here. DEFINITELY needs more eyeballs, but all
tests pass, including a new one that tests StopIteration leakage both with and
without the __future__ directive. Some docs changes have been made (I grepped
for 'stopiteration' and
Nick Coghlan added the comment:
As Stefan noted, the leading underscore on _PyErr_ChainExceptions is just a
warning to third party users that we don't yet offer any API stability
guarantees for it. Using it in CPython itself is entirely appropriate - that's
the whole reason for exposing it to
Nick Coghlan added the comment:
Heh, I guess that should have been The main thing missing other than docs and
tests is ... :)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
Chris Angelico added the comment:
I can have a poke at the __future__ import tonight, but my main concern is
memory management - I'm not sufficiently familiar with the exception handling
calls to be sure that I'm neither leaking nor over-freeing anything. There's
also a secondary concern that
Stefan Behnel added the comment:
Regarding the patch below, isn't most of this redundant? ISTM that simply
calling PyErr_SetString(...) should do all of this, including the exception
chaining.
diff -r 23ab1197df0b Objects/genobject.c
--- a/Objects/genobject.c Wed Nov 19 13:21:40 2014
Chris Angelico added the comment:
Stefan, I'm not sure - I don't know the details of the C API here. But I tried
commenting out everything but that one line, and while it does result in
RuntimeError, it doesn't do the exception chaining. Currently, I believe the
exception isn't being caught
Stefan Behnel added the comment:
Ah, right - chaining only happens automatically when the exception has already
been caught and moved to sys.exc_info.
There's a _PyErr_ChainExceptions(), though, which does it for you. You should
be able to say
PyErr_Fetch(x,y,z)
PyErr_SetString()
Chris Angelico added the comment:
Yeah, I saw that. Since that function begins with an underscore, I thought it
best to replicate its behaviour rather than to call it. Either way ought to
work though.
--
___
Python tracker rep...@bugs.python.org
Stefan Behnel added the comment:
Public underscore C functions are there for exactly the purpose of not
duplicating functionality across *internal* core runtime code, without making
them an official part of the C-API for *external* code. (I understand that it's
a POC patch, though, so whoever
Stefan Behnel added the comment:
FYI, here's the set of tests that I've written for my implementation in Cython:
https://github.com/cython/cython/blob/b4383a540a790a5553f19438b3fc22b11858d665/tests/run/generators_pep479.pyx
--
___
Python tracker
Changes by Alexander Belopolsky alexander.belopol...@gmail.com:
--
nosy: +belopolsky
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Raymond Hettinger added the comment:
FYI: I applied these two changes right after Guido pronounced on PEP 479:
https://mail.python.org/pipermail/python-checkins/2014-November/133252.html
https://mail.python.org/pipermail/python-checkins/2014-November/133253.html
Also, I'm submitting a patch
STINNER Victor added the comment:
FYI: I applied these two changes right after Guido pronounced on PEP 479:
Extract of emails:
changeset: 93542:9eb0d0eb0992
parent: 93540:23f8a511050a
user:Raymond Hettinger python at rcn.com
date:Sat Nov 22 21:56:23 2014 -0800
PEP
Guido van Rossum added the comment:
Here's a doc patch for itertools.
--
Added file: http://bugs.python.org/file37297/itertoolsdoc.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
Chris Angelico added the comment:
Known issues with the current patch, if anyone feels like playing with this who
better knows the code:
1) Needs a __future__ directive to control behaviour
2) test_generators needs to be heavily reworked
3) The test of what exception was thrown needs to also
Marc Schlaich added the comment:
AFAIS this would break all existing code for yield-based coroutine schedulers
(Tornado, Twisted, Trollius, monocle, ...) when a coroutine is exited with
`raise StopIteration` in client code.
And this seems like a lot, a quick GitHub code search gives various
Chris Angelico added the comment:
Marc, are those all cases where the raise StopIteration is actually inside a
generator? If so, it can be trivially replaced with return. Yes, it'll break
that way of spelling it, but it's a completely mechanical transformation, and
making the change won't
Marc Schlaich added the comment:
Yes, all yield-based coroutines are generators. I know that there is a backward
compatible upgrade path, but this might have a huge impact on existing code.
Interestingly, I didn't know before researching this PEP that you can actually
use `return` without
Chris Angelico added the comment:
Yep, the question is whether any of the raise StopIteration lines are
actually non-local flow control. If they're local, then it's easy: mechanical
replacement with return and it becomes compatible with all versions (unless
it has a value attached to it, as
Guido van Rossum added the comment:
If all examples were just using raise StopIteration instead of return
in a generator I would be very happy. Such code can be trivially fixed by
using return and the __future__ import will help the eventual transition.
It's sad that apparently this use of
Chris Angelico added the comment:
Sadly, I don't know of a way to check if that's the case, other than by
manually going through and eyeballing the code - if there's raise
StopIteration, see if there's also yield in the same function. The three
cited examples are (I checked those straight
Changes by R. David Murray rdmur...@bitdance.com:
--
nosy: +r.david.murray
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Serhiy Storchaka added the comment:
I suggest to apply right now those changes which are compatible with current
behavior and don't make code more cumbersome. E.g.
-while True:
-yield next(line_pair_iterator)
+yield from line_pair_iterator
and
-raise
New submission from Chris Angelico:
See PEP for full details. Attached is POC patch: behaviour is altered globally
(rather than respecting a __future__ directive), and minimal changes are made
elsewhere to make the test suite mostly pass (test_generators does not - it'll
need more
Changes by Guido van Rossum gu...@python.org:
--
nosy: +gvanrossum
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22906
___
___
Python-bugs-list
Roundup Robot added the comment:
New changeset dd19add74b21 by Guido van Rossum in branch 'default':
PEP 479: Add link to issue 22906.
https://hg.python.org/peps/rev/dd19add74b21
--
nosy: +python-dev
___
Python tracker rep...@bugs.python.org
78 matches
Mail list logo
