[issue25601] test_cpickle failure on the ware-gentoo-x86 builbot

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
components: +Tests
type:  -> behavior
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 builbot

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +alexandre.vassalotti, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

Sorry for being obscure before, it was hard to pinpoint.  I think I just 
figured it out!  I had code like this in a subprocess:

def worker():
while True:
obj = self.queue.get()
# do work with obj using asyncio http module

def producer():
nonlocal self
obj2 = self.queue.get()
return obj2


workers = []
for i in range(FILE_OP_WORKERS):
t = asyncio.ensure_future(worker())
t.add_done_callback(op_finished)
workers.append(t)

while True:
f = loop.run_in_executor(None, producer)
obj = loop.run_until_complete(f)

t = async_queue.put(obj)
loop.run_until_complete(t)

loop.run_until_complete(asyncio.wait(workers))

where self.queue is a multiprocessing.Queue, and async_queue is an asyncio 
queue.  The idea is that I have a process populating a multiprocessing queue, 
and I want to transfer it to an syncio queue while letting the workers do their 
thing.

Without knowing the underlying behavior, my theory is that when python blocks 
on the multiprocessing queue lock, it releases socket events to the async http 
module's selectors, and then when the async loop gets to the selectors they're 
released again.

If I switch the producer to instead use a queue.get_nowait and busy wait with 
asyncio.sleep I don't get the error...however this is not ideal is we're busy 
waiting.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25598] Fix memory_hex (#9951) for non-contiguous buffers

[Stefan Krah]

Changes by Stefan Krah :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7759] mhlib fails on Btrfs filesystem (test_mhlib failure)

[R. David Murray]

R. David Murray added the comment:

Sure, why not.  Having buildbots be green is good, and I doubt anyone is using 
mhlib any more even in python2.  And if they are the chances this will break 
something seems extremely small.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 buildbot

[Zachary Ware]

Zachary Ware added the comment:

This also happened on my Windows buildbot: 
http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Non-Debug%202.7/builds/195

That build is running again as build 196.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7759] mhlib fails on Btrfs filesystem (test_mhlib failure)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The test now is passed.

--
resolution: wont fix -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25599] asyncio.iscoroutinefunction returns unexpected results when presented with unittest.mock.Mock

[Theron Luhn]

Theron Luhn added the comment:

For me, the context is a test I was writing that went something like this:

>>> import asyncio
>>> from unittest.mock import Mock
>>> loop = asyncio.get_event_loop()
>>> blocking_func = Mock()
>>> loop.run_in_executor(None, blocking_func)
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
 line 497, in run_in_executor
raise TypeError("coroutines cannot be used with run_in_executor()")
TypeError: coroutines cannot be used with run_in_executor()

I understand that the nature of Mock makes its behaviors ambiguous.  However, 
there are a few reasons I think asyncio.iscoroutinefunction(Mock()) should be 
false:

1) inspect.iscoroutinefunction reports false.  asyncio.iscoroutinefunction 
should be consistent with this.
2) A coroutine function should return a coroutine object.  Mock's default 
behavior won't return a coroutine object, so it shouldn't be identified as a 
coroutine function by default.
3) It's tidier to make a non-coroutine function Mock into a coroutine function 
(asyncio.coroutine(Mock())) than it is to make a coroutine function Mock into a 
non-coroutine function Mock (mock._is_coroutine is implementation-specific 
hack).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

clarification, adding the fut.done() check, or monkey patching:
orig_sock_connect_cb = 
asyncio.selector_events.BaseSelectorEventLoop._sock_connect_cb
def _sock_connect_cb(self, fut, sock, address):
if fut.done(): return
return orig_sock_connect_cb(self, fut, sock, address)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 buildbot

[Zachary Ware]

Zachary Ware added the comment:

I also have no idea how that happened, but seems to have been a fluke. Build 
101 is fine, and nothing has changed on that box.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 buildbot

[Zachary Ware]

Zachary Ware added the comment:

Build 196 on the Windows bot is fine.  Could be a bizarre test ordering issue?  
Nothing else should have changed between builds 195 and 196 on the Windows bot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

Actually, I just realized I had fixed it locally by changing the callback to 
the following:
429 def _sock_connect_cb(self, fut, sock, address):
430 if fut.cancelled() or fut.done():
431 return
 
so a fix is still needed, and I also verified this happens with python3.4 as 
well.

--
status: closed -> open
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7759] mhlib fails on Btrfs filesystem (test_mhlib failure)

[Roundup Robot]

Roundup Robot added the comment:

New changeset 37431d9abbcd by Serhiy Storchaka in branch '2.7':
Issue #7759: Fixed the mhlib module on filesystems that doesn't support
https://hg.python.org/cpython/rev/37431d9abbcd

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

I'm going to close this as I've found a work-around, if I find a better 
test-case I'll open a new bug.

--
resolution:  -> later
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 buildbot

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
title: test_cpickle failure on the ware-gentoo-x86 builbot -> test_cpickle 
failure on the ware-gentoo-x86 buildbot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25601] test_cpickle failure on the ware-gentoo-x86 builbot

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

http://buildbot.python.org/all/builders/x86%20Gentoo%20Non-Debug%20with%20X%202.7/builds/100/steps/test/logs/stdio

There are a lot of failures, all look as:
==
ERROR: test_simple_newobj (test.test_cpickle.FileIOCPicklerFastTests)
--
Traceback (most recent call last):
  File 
"/buildbot/buildarea/2.7.ware-gentoo-x86/build/Lib/test/pickletester.py", line 
1145, in test_simple_newobj
s = self.dumps(x, proto)
  File 
"/buildbot/buildarea/2.7.ware-gentoo-x86/build/Lib/test/test_cpickle.py", line 
141, in dumps
p.dump(arg)
AttributeError: 'module' object has no attribute '_reduce_ex'

--

It looks as the _reduce_ex() function is missed in the copy_reg module. Have no 
ideas how it is possible.

--
messages: 254483
nosy: serhiy.storchaka, zach.ware
priority: normal
severity: normal
status: open
title: test_cpickle failure on the ware-gentoo-x86 builbot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25600] argparse, argument_default=argparse.SUPPRESS seems to have no effect

[R. David Murray]

R. David Murray added the comment:

If you specify a default for an argument, that overrides the global default.

--
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25602] Add support for EVFILT_USER kqueue filter

[Jakub Klama]

New submission from Jakub Klama:

It's useful for doing signaling between threads (especially I/O-bound threads).

Related github pull request: https://github.com/python/cpython/pull/23

--
components: Library (Lib)
files: 0001-Add-support-for-EVFILT_USER-kqueue-filter.patch
keywords: patch
messages: 254485
nosy: jceel
priority: normal
severity: normal
status: open
title: Add support for EVFILT_USER kqueue filter
type: enhancement
versions: Python 3.6
Added file: 
http://bugs.python.org/file41014/0001-Add-support-for-EVFILT_USER-kqueue-filter.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Guido van Rossum]

Guido van Rossum added the comment:

I'm not an expert on this terminology but don't you have that backwards?
Assume we're using select() for a second. If you ask select() "is this FD
ready" several times in a row without doing something to the FD it will
answer yes every time once the FD is ready. IIUC that's what
level-triggered means, and that's what causes the bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Berker Peksag]

Berker Peksag added the comment:

I like Martin's support.expected_module_api() suggestion in msg247167. I still 
find passing self to a function just to use assertCountEqual a bit weird, but I 
can live with that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25603] spelling mistake - 26.1 typing

[Roundup Robot]

Roundup Robot added the comment:

New changeset 81cc0cea2323 by Zachary Ware in branch '3.5':
Issue #25603: Add missing parenthesis.
https://hg.python.org/cpython/rev/81cc0cea2323

New changeset 1af59662f6d5 by Zachary Ware in branch 'default':
Closes #25603: Merge with 3.5
https://hg.python.org/cpython/rev/1af59662f6d5

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

I don't believe this is a case of nonidempotent callbacks, unless you are 
referring to Future.set_result(), which by design can't be called twice.  The 
callbacks are given an inconsistent opportunity to modify the poll set because 
of indeterminacy in the ioloop.  That being said I understand your reluctance 
given the amount of turmoil this has but would argue that consistency with 
tornado is a powerful ally and that a model where any callback using call_soon 
will be guaranteed the opportunity to modify the poll set is a good thing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

Guido,

Shouldn't this not be the case for level triggered polling?  From looking at 
selectors it looks like these are always level triggered which means they 
should only event once.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks, but I don't like the idea of that patch. It feels like a hack that 
makes it less likely that the issue occurs, but I don't feel we should rely on 
the callbacks being called before checking the selector again. There may be 
other reasons (perhaps a future modification to the code) why we might 
occasionally check the selector redundantly. IOW I think we should really 
ensure that all I/O callbacks are properly idempotent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

Attached server side of repro.

--
Added file: http://bugs.python.org/file41017/Issue25593_repro_server.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

Just reproduced on Linux, Fedora Core 23.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

attaching my simplified testcase and logged an aiohttp bug: 
https://github.com/KeepSafe/aiohttp/issues/633

--
Added file: http://bugs.python.org/file41018/test_app.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Mike Gilbert]

Changes by Mike Gilbert :


--
nosy: +floppymaster

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

Attaching simplified test setup.   It does take some doing to repro so the 
local async server is required to make it happen (for me).  When I tried just 
pointing to python.org it would not repro in 100 iterations, but using a local 
dummy server repros 100% for me.

--
Added file: http://bugs.python.org/file41016/Issue25593_repro_client.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25603] spelling mistake - 26.1 typing

[Matthias welp]

New submission from Matthias welp:

Almost at the end of the page, under Usage of Typing.NamedTuple(...), this code 
snippet occurs: `Employee = typing.NamedTuple('Employee', [('name', str), 'id', 
int)])`. Unfortunately, this has an error in its parenthesis. 

This can easily be fixed by adding an opening bracket before the `'id'`-part, 
as seen here: `Employee = typing.NamedTuple('Employee', [('name', str), ('id', 
int)])`.

--
assignee: docs@python
components: Documentation
messages: 254511
nosy: Matthias welp, docs@python
priority: normal
severity: normal
status: open
title: spelling mistake - 26.1 typing
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

Nevermind, in the case of writeablity it won't matter either way.

--

So in looking at tornado's ioloop they run the ready callbacks before calling 
poll().  So the callbacks can modify the poll set.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25569] Memory leak in SSLSocket.getpeercert()

[Roundup Robot]

Roundup Robot added the comment:

New changeset 10c3646b2d59 by Benjamin Peterson in branch '2.7':
fix memory leak in _get_crl_dp (closes #25569)
https://hg.python.org/cpython/rev/10c3646b2d59

New changeset aabe273b20ab by Benjamin Peterson in branch '3.4':
fix memory leak in _get_crl_dp (closes #25569)
https://hg.python.org/cpython/rev/aabe273b20ab

New changeset 07a298572d93 by Benjamin Peterson in branch '3.5':
merge 3.5 (#25569)
https://hg.python.org/cpython/rev/07a298572d93

New changeset fb55b1ab43fc by Benjamin Peterson in branch 'default':
merge 3.5 (#25569)
https://hg.python.org/cpython/rev/fb55b1ab43fc

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25603] spelling mistake - 26.1 typing

[Zachary Ware]

Zachary Ware added the comment:

Thanks for the report!

--
nosy: +zach.ware
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25530] ssl: OP_NO_SSLv3 should always be set unless a user specifically asks for it

[Roundup Robot]

Roundup Robot added the comment:

New changeset d80954d941c7 by Benjamin Peterson in branch '2.7':
always set OP_NO_SSLv3 by default (closes #25530)
https://hg.python.org/cpython/rev/d80954d941c7

New changeset 56f64ec9259f by Benjamin Peterson in branch '3.4':
always set OP_NO_SSLv3 by default (closes #25530)
https://hg.python.org/cpython/rev/56f64ec9259f

New changeset d1737db0f1b2 by Benjamin Peterson in branch '3.5':
merge 3.4 (#25530)
https://hg.python.org/cpython/rev/d1737db0f1b2

New changeset 2899acbd2b46 by Benjamin Peterson in branch 'default':
merge 3.5 (#25530)
https://hg.python.org/cpython/rev/2899acbd2b46

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Guido van Rossum]

Guido van Rossum added the comment:

I wonder if the bug is in aiohttp? The code you show is still too complex
to debug for me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21748] glob.glob does not sort its results

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

This code repros without aiohttp when pitted against the previously attached 
web server (again on OSX 10.11, mid-2012 MBPr).

Admittedly this may seem very arbitrary but I have better reasons in my 
production code for stopping an IOLoop and starting it again (which seems to be 
important to the reproduction steps).


import asyncio

loop = asyncio.get_event_loop()

def batch_open():
for i in range(100):
c = asyncio.ensure_future(asyncio.open_connection('127.0.0.1', 8080))
c.add_done_callback(on_resp)

def on_resp(task):
task.result()
loop.stop()

loop.call_soon(batch_open)
while True:
loop.run_forever()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Guido van Rossum]

Guido van Rossum added the comment:

Justin's repro provides a clue: when the event loop is stopped before all
callbacks have been processed, when the loop is restarted the I/O selector
is asked again to do its work, and it will report all the same sockets as
ready. So then the callback will be put into the ready queue again (even
though it's already there). Then the second call will find the future
already done.

I'm not sure how this explains Alexander's issue but it's probably
something similar. We should carefully review the other I/O callbacks too
-- most of them look like they don't mind being called spuriously, but
there are a few others (_sock_recv, _sock_sendall, _sock_accept) that look
like they check for fut.cancelled() and might be susceptible to the same
bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

I'm attaching a patch that runs `_ready` callbacks at the start of `_run_once`. 
 The style and implications are ranging so I leave it to you at this point.

--
keywords: +patch
Added file: 
http://bugs.python.org/file41019/run_once_testfix_for_Issue25593.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Guido van Rossum]

Guido van Rossum added the comment:

Sorry, the code you posted is still incomprehensible. E.g. I suppose your
worker doesn't really have
```
obj = self.queue.get()
```
but rather something like
```
obj = yield from async_queue.get()
```
But in the end, even with that hypothesis, I can't explain what you're
seeing, and I believe there is a bug related to bad mixing multiprocessing
and asyncio in some code you're not showing, and your "fix" just masks the
problem. Note that the code you posted doesn't touch sockets in any way,
while the issue you're seeing is related to sockets. So there *must* be
more to it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

self.queue is not an async queue, as I stated above its a multiprocessing 
queue.  This code is to multiplex a multiprocessing queue to a async queue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18689] add argument for formatter to logging.Handler and subclasses in logging module

[Derek Wilson]

Derek Wilson added the comment:

> It's not an ideal world. Sorry, but I think this change is too invasive to 
> consider.

Obviously its not ideal, which is why my suggestion doesn't require intelligent 
3rd party libraries and is explicitly not invasive. As I said in my previous 
comment, using a keyword only argument means: "If [3rd party libraries] haven't 
[properly handled **kwargs] then nothing changes for them and they just support 
exactly the same features they supported previously."

The upshot is it is not invasive and no one needs to care unless they want to 
use the new functionality.

As far as this change being needed or not, nothing "needs" to be made easier to 
use if it is possible to use it. But that isn't really a good reason not to 
improve things.

I honestly think that part of the reason this hasn't come up is because the 
advanced features of logging are so difficult to use that people just don't use 
it to its fullest extent. On top of that, when learning python, logging is way 
harder to grok than it should be for someone new to python.

Logging and unittest are two of the most important libraries for new 
pythonistas to learn, but they are also some of the most nebulous, stateful, 
magical, java-like, complicated, verbose, and difficult to master packages in 
python.

They've been around for a while for sure - but doesn't that rather mean that 
they could use an update?

I'm willing to submit a patch if it has the smallest chance of being considered?

--
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Justin Mayfield]

Justin Mayfield added the comment:

I believe I'm seeing this bug in a non-threaded and non-forked env.  

System:
 OSX 10.11.1 (15B42)
 Python 3.5.0 (from brew install)

I'm using aiohttp to create several dozens of HTTP connections to the same 
server (an async tornado web server).  Nothing special is being done around the 
event loop creation (standard get_event_loop()).  However in my system the 
event loop is frequently stopped, via ioloop.stop(), and restarted via 
ioloop.run_forever().  I'm not sure this is related to the issue yet, but it's 
worth mentioning.

I can't provide simplified test code just yet, but I can reproduce in my env 
with nearly 100% odds when doing a full system test.  Attached is a sample 
backtrace.

--
nosy: +Justin Mayfield
Added file: http://bugs.python.org/file41015/asyncio_invalid_state_bt.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25593] _sock_connect_cb can be called twice resulting in InvalidStateError

[Alexander Mohr]

Alexander Mohr added the comment:

Perhaps I'm doing something really stupid, but I was able to reproduce the two 
issues I'm having with the following sample script. If you leave the monkey 
patch disabled, you get the InvalidStateError, if you enable it, you get the 
ServerDisconnect errors that I'm currently seeing which I work-around with 
retries.  Ideas?

import asyncio
import aiohttp
import multiprocessing
import aiohttp.server
import logging
import traceback

# Monkey patching
import asyncio.selector_events

# http://bugs.python.org/issue25593
if False:
orig_sock_connect_cb = 
asyncio.selector_events.BaseSelectorEventLoop._sock_connect_cb
def _sock_connect_cb(self, fut, sock, address):
if fut.done(): return
return orig_sock_connect_cb(self, fut, sock, address)
asyncio.selector_events.BaseSelectorEventLoop._sock_connect_cb = 
_sock_connect_cb


class HttpRequestHandler(aiohttp.server.ServerHttpProtocol):
@asyncio.coroutine
def handle_request(self, message, payload):
response = aiohttp.Response(self.writer, 200, 
http_version=message.version)
response.add_header('Content-Type', 'text/html')
response.add_header('Content-Length', '18')
response.send_headers()
yield from asyncio.sleep(0.5)
response.write(b'It Works!')
yield from response.write_eof()


def process_worker(q):
loop = asyncio.get_event_loop()
#loop.set_debug(True)
connector = aiohttp.TCPConnector(force_close=False, keepalive_timeout=8, 
use_dns_cache=True)
session = aiohttp.ClientSession(connector=connector)
async_queue = asyncio.Queue(100)

@asyncio.coroutine
def async_worker(session, async_queue):
while True:
try:
print("blocking on asyncio queue get")
url = yield from async_queue.get()
print("unblocking on asyncio queue get")
print("get aqueue size:", async_queue.qsize())
response = yield from session.request('GET', url)
try:
data = yield from response.read()
print(data)
finally:
yield from response.wait_for_close()
except:
traceback.print_exc()

def producer(q):
print("blocking on multiprocessing queue get")
obj2 = q.get()
print("unblocking on multiprocessing queue get")
print("get qempty:", q.empty())
return obj2

def worker_done(f):
try:
f.result()
print("worker exited")
except:
traceback.print_exc()

workers = []
for i in range(100):
t = asyncio.ensure_future(async_worker(session, async_queue))
t.add_done_callback(worker_done)
workers.append(t)

@asyncio.coroutine
def doit():
print("start producer")
obj = yield from loop.run_in_executor(None, producer, q)
print("finish producer")

print("blocking on asyncio queue put")
yield from async_queue.put(obj)
print("unblocking on asyncio queue put")
print("put aqueue size:", async_queue.qsize())

while True:
loop.run_until_complete(doit())


def server():
loop = asyncio.get_event_loop()
#loop.set_debug(True)

f = loop.create_server(lambda: HttpRequestHandler(debug=True, 
keep_alive=75), '0.0.0.0', '8080')

srv = loop.run_until_complete(f)
loop.run_forever()


if __name__ == '__main__':
q = multiprocessing.Queue(100)

log_proc = multiprocessing.log_to_stderr()
log_proc.setLevel(logging.DEBUG)

p = multiprocessing.Process(target=process_worker, args=(q,))
p.start()

p2 = multiprocessing.Process(target=server)
p2.start()

while True:
print("blocking on multiprocessing queue put")
q.put("http://0.0.0.0:8080;)
print("unblocking on multiprocessing queue put")

print("put qempty:", q.empty())

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7759] mhlib fails on Btrfs filesystem (test_mhlib failure)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is even simpler and more reliable patch. It works even if the subfolder is 
a symlink to the directory on the filesystem that doesn't support links 
counting for directories.

--
Added file: http://bugs.python.org/file41011/mhlib_nlinks_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25263] test_tkinter fails randomly on the buildbots "AMD64 Windows10" (3.4, 3.5, 3.x)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Tests are fixed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 2.7, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Changes by Jacek Kołodziej :


Added file: http://bugs.python.org/file41012/Issue23883_all.v6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Jacek Kołodziej added the comment:

Serhiy, thank you for the review. I've made proposed changes (along with 
rebasing Issue23883_all patch; Issue23883_test_gettext.v3.patch still applies 
cleanly).

--
Added file: 
http://bugs.python.org/file41013/Issue23883_support_check__all__.v6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com