[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Roundup Robot added the comment: New changeset defd09a5339a by Victor Stinner in branch '3.4': asyncio: sync with Tulip http://hg.python.org/cpython/rev/defd09a5339a New changeset 8dc8c93e74c9 by Victor Stinner in branch 'default': asyncio: sync with Tulip http://hg.python.org/cpython/rev/8dc8c9

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: IMO the comment is too aggressive. I want the workaround to stay in the codebase so CPython asyncio ans Tulip asyncio (== upstream) don't diverge. -- ___ Python tracker

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Arfrever Frehtes Taifersar Arahesis added the comment: The added comment contains "This workaround should be removed in 3.5.0.". Since default branch now contains Python 3.5, maybe it is time to remove workaround on default branch? -- nosy: +Arfrever __

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: > I should check, but I think that Python create a tuple for you if you don't > pass directly a tuple, so it's not very different. That's what I thought, but still, better to have the code clearly expressing what it does, than relying on obscure implementation

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Roundup Robot added the comment: New changeset 2729823525fe by Yury Selivanov in branch '3.4': asyncio.tasks: Make sure CoroWrapper.send proxies one argument correctly http://hg.python.org/cpython/rev/2729823525fe New changeset 552ee474f3e7 by Yury Selivanov in branch 'default': asyncio.tasks: M

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: I agree with Yuri and I approve of the patch. -- ___ Python tracker ___ ___ Python-bugs-list maili

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

STINNER Victor added the comment: "I think we should adjust the solution, to avoid having arguments to 'gen.send' packed in two nested tuples." I should check, but I think that Python create a tuple for you if you don't pass directly a tuple, so it's not very different. Anyway, it is only use

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: Guido, I'm feeling a bit uncomfortable with the patch I pushed. I think we should adjust the solution, to avoid having arguments to 'gen.send' packed in two nested tuples. Please take a look at the new patch (corowrapper_03.patch). It adds some amount of uglin

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Changes by Yury Selivanov : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Roundup Robot added the comment: New changeset 0c35d3616df5 by Yury Selivanov in branch '3.4': asyncio.tasks: Fix CoroWrapper to workaround yield-from bug in CPython < 3.4.1 http://hg.python.org/cpython/rev/0c35d3616df5 New changeset 13ff8645be57 by Yury Selivanov in branch 'default': syncio.tas

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: > [...] CoroWrapper.send() signature is different from a real generator's > send() method, but I think that send() to a coroutine is an internal detail > anyway [...] Yeah, and since it's used in debug mode only, I think we should be safe. > When you commit,

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: OK, looks good. I tried your test with my earlier workaround and the wrapper got deallocated too early, proving that my workaround was indeed wrong and your test is useful. I am still concerned theoretically that the CoroWrapper.send() signature is different

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: Please see the corowrapper_02.patch. I've removed the version check, now it's much simpler. -- Added file: http://bugs.python.org/file34850/corowrapper_02.patch ___ Python tracker

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: Yuri, thanks for the test, but why would the patch need a version check? Shouldn't the work-around work equally well in Python versions that don't need it? Maybe all we need is a comment explaining that it is a work-around and a hint that eventually we shoul

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Changes by Yury Selivanov : -- resolution: fixed -> status: closed -> open ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: Guido: please take a look at the patch "corowrapper_01.patch". -- Added file: http://bugs.python.org/file34832/corowrapper_01.patch ___ Python tracker _

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: I think I have to add a work-around to Tulip anyway, because I don't want to have to tell people "you must upgrade your Python 3.3 otherwise this problem can happen" (if upgrading was easy for them they would be on 3.4 :-). So I don't care much if the 3.3 ba

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: > 3.3 is in security-fix only mode. Yeah, but this is a core language bug. I believe some people may be stuck on 3.3 with broken 'yield from' for whatever reason, which will cause hard to find bugs in 3.3 compatible libraries (like asyncio/tulip). I think we

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Benjamin Peterson added the comment: On Mon, Apr 14, 2014, at 10:20, Yury Selivanov wrote: > > Yury Selivanov added the comment: > > > 3.3 is in security-fix only mode. > > Yeah, but this is a core language bug. I believe some people may be stuck > on 3.3 with broken 'yield from' for whatever

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Benjamin Peterson added the comment: 3.3 is in security-fix only mode. -- nosy: +benjamin.peterson ___ Python tracker ___ ___ Python-b

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: This is nice (a backport 3.3 would be even nicer) but at least for the PyPI repo version of Tulip I'd like to have work-around so people won't run into this when they are using a slightly outdated Python version. I'll think about which of my work-arounds is saf

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Yury Selivanov added the comment: Hm... Can we also commit this to 3.3? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Roundup Robot added the comment: New changeset 05b3a23b3836 by Benjamin Peterson in branch '3.4': fix sending tuples to custom generator objects with yield from (closes #21209) http://hg.python.org/cpython/rev/05b3a23b3836 New changeset d1eba2645b80 by Benjamin Peterson in branch 'default': merg

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

STINNER Victor added the comment: > Are you going to be at the CPython sprint tomorrow? I'll be there in the > morning but my plane leaves in the afternoon. I organize a "Port OpenStack to Python3" sprint, but I may come also to the CPython sprint. -- _

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: Wow. So many fixes! :-) Are you going to be at the CPython sprint tomorrow? I'll be there in the morning but my plane leaves in the afternoon. -- ___ Python tracker

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

STINNER Victor added the comment: I don't think that the bug comes from asyncio, but it looks like a bug in the implementation of "yield from" in CPython directly! Try with ceval.patch. ceval.c has a fast path if the object is a generator. With PYTHONASYNCIODEBUG=1, the object is a CoroWrapper

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

STINNER Victor added the comment: gen_send.diff doesn't look like a fix but a workaround. gen_send_2.diff lacks a unit test. -- ___ Python tracker ___ __

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

STINNER Victor added the comment: The error occurs at line "v = yield from q.get()": Traceback (most recent call last): File "/home/haypo/prog/python/default/Lib/asyncio/events.py", line 39, in _run self._callback(*self._args) File "/home/haypo/prog/python/default/Lib/asyncio/tasks.py",

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: Heh. METH_O was *also* a red herring. But upstream (Tulip) issue 163 *was* a good clue. I now believe that the real bug is that CoroWrapper.__iter__() has "return self" rather than "return iter(self.gen)". That fix is in the 2nd attachment. -- Adde

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Guido van Rossum added the comment: I'll be darned. It appears that generator's send() method uses METH_O, which means that it really expects a single argument, but if you pass it a tuple, it assumes that you meant each item in the tuple as a separate argument. I think a more correct fix is at

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Changes by Ned Deily : -- nosy: +giampaolo.rodola, gvanrossum, haypo, pitrou, yselivanov ___ Python tracker ___ ___ Python-bugs-list m

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

Richard Kiss added the comment: For a reason that I don't understand, this patch to asyncio fixes the problem: --- a/asyncio/tasks.py Mon Mar 31 11:31:16 2014 -0700 +++ b/asyncio/tasks.py Sat Apr 12 20:37:02 2014 -0700 @@ -49,7 +49,8 @@ def __next__(self): return next(self.gen)

[issue21209] q.put(some_tuple) fails when PYTHONASYNCIODEBUG=1

New submission from Richard Kiss: import asyncio import os def t1(q): yield from asyncio.sleep(0.5) q.put_nowait((0, 1, 2, 3, 4, 5)) def t2(q): v = yield from q.get() print(v) q = asyncio.Queue() asyncio.get_event_loop().run_until_complete(asyncio.wait([t1(q), t2(q)])) When