Kyle Stanley added the comment:
> since the new threads are spawned in ThreadPoolExecutor *after*
> executor.submit() is called
It's also worth mentioning that ThreadPoolExecutor only spawns up to one
additional thread at a time for each executor.submit()
Kyle Stanley added the comment:
> Hey, I've done the change and opened a pull request for it (I'm working with
> Ben and I've let him know)
Make sure to change the title of the PR to "bpo-: ",
this will automatically attach the PR to the associated bpo issue. For example,
Kyle Stanley added the comment:
> Nice work! This is a great excercise, but we can really just use
> concurrent.futures.ThreadPool as is. Spawning threads is fast. As I mentioned
> before all we need to do is to design *our* API to NOT initialize pools in
> __init__, that's it
Kyle Stanley added the comment:
> The asynchronous spawning of threads or processes would also not be
> compatible with the executor subclasses as far as I can tell.
> Thus, it seemed to make more sense to me to actually build up a new Pool
> class from scratch that was l
Kyle Stanley added the comment:
> No, that would be too much work. Writing a thread pool or process pool from
> scratch is an extremely tedious task and it will take us years to stabilize
> the code. It's not simple.
> We should design *our* API correctly though. And that m
Kyle Stanley added the comment:
> And that's why I like it. If we add ProcessPool it will have the same
> argument: concurrency.
> max_workers isn't correct, as we want to spawn all threads and all processes
> when we start. Thus btw makes me think that initializing threa
Kyle Stanley added the comment:
> def __init__(self, concurrency=None):
Minor clarification: the default should probably be None, which would
effectively set the default maximum number of threads to min(32, os.cpu_count()
+ 4), once it's passed to ThreadPoolExecu
Kyle Stanley added the comment:
> Number of OS threads to spawn.
Ah I see, so this would correspond with the "max_workers" argument of
ThreadPoolExecutor then, correct? If so, we could pass this in the __init__ for
ThreadPool:
def __init__(self, concurrency):
...
s
Kyle Stanley added the comment:
> I believe behavior occurs within shutdown_default_executor(), correct?
> Specifically, within for ThreadPoolExecutor when executor.shutdown(wait=True)
> is called and all of the threads are joined without a timeout, it simply
> waits for
Kyle Stanley added the comment:
> async with asyncio.ThreadPool(concurrency=10) as pool:
I'm definitely on board with the usage of an async context manager and the
functionality shown in the example, but I'm not sure that I entirely understand
what the "concurrency" kwarg in &q
Kyle Stanley added the comment:
> Also in this case run awaits and returns the result. Yury suggested earlier
> just to return the future and not await.
Yeah that's roughly what my initial version was doing. I'm personally leaning a
bit more towards returning the future rathe
Kyle Stanley added the comment:
Actually, I think it would be better to move the functionality of
loop.shutdown_default_executor() to a new private method
loop._shutdown_executor() that takes an executor argument rather than shutting
down the default one. This could be used in both
Kyle Stanley added the comment:
> thread = threading.Thread(target=loop._do_shutdown, args=(executor,future))
Correction:
> thread = threading.Thread(target=_do_shutdown, args=(loop, executor,future))
Also, it might make more sense to rename _do_shutdown() to
_do_executor_sh
Kyle Stanley added the comment:
Also, I agree with Paul's idea of initializing the ThreadPoolExecutor in the
__init__ instead of __aenter__, that makes more sense now that I think about it.
--
___
Python tracker
<https://bugs.python.
Kyle Stanley added the comment:
> I don't think changing the default executor is a good approach. What happens,
> if two or more thread pools are running at the same time? In that case they
> will use the same default executor anyway, so creating a new executor each
> tim
Kyle Stanley added the comment:
So, here's a prototype implementation of asyncio.ThreadPool that would function
exactly as Yury described, but I'm not convinced about the design. Usually, it
seems preferred to separate the context manager from the rest of the class (as
was done
Kyle Stanley added the comment:
> end up adding two high-level functions
Clarification: asyncio.run_in_executor() would be a function, but
asyncio.ThreadPool would be a context manager class.
--
___
Python tracker
<https://bugs.pyth
Kyle Stanley added the comment:
> I don't like the low-level API of run_in_executor. "executor" being the
> first argument, the inability to pass **kwargs, etc.
> I mean it's great that we can use 'concurrent.futures' in asyncio, but having
> native asyncio pool
Kyle Stanley added the comment:
> I'm a newcomer, would it be ok for me to take on this propsal?
It will need approval from Yury and/or Andrew before it can be merged, but I
think this is relatively uncontroversial. You can definitely work on it tho
Change by Kyle Stanley :
--
components: +asyncio
___
Python tracker
<https://bugs.python.org/issue38652>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> I would propose to changing it to:
propose changing it to*
Also, I think this issue would be a good candidate for "newcomer friendly",
since it's a simple and well-defined documentation update. It could provide a
decent introduction for
New submission from Kyle Stanley :
In the documentation
(https://docs.python.org/3.9/library/asyncio-protocol.html#buffered-streaming-protocols)
for asyncio.BufferedProtocol there is a provisional API note. This should be
updated/removed for 3.8 and 3.9 since it became a part of the stable
Kyle Stanley added the comment:
> Is this whatsnew/3.8.rst entry correct and complete?
> The :func:`asyncio.coroutine` :term:`decorator` is deprecated and will be
> removed in version 3.10. Instead of ``@asyncio.coroutine``, use
> :keyword:`async def` instead. (Contribut
Kyle Stanley added the comment:
GH-16977 Implements the deprecation warning, adds tests, and adds the 3.9
whatsnew entry. Once this PR is finalized, I think this issue can be closed.
--
stage: patch review ->
___
Python tracker
<
Change by Kyle Stanley :
--
pull_requests: +16503
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16977
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
GH-16975 Is a simple fix for the asyncio.wait() whatsnew entry for 3.8. I'll
implement the deprecation warning and add a 3.9 whatsnew entry in a separate
PR, since those changes won't be backported.
--
stage: patch review
Change by Kyle Stanley :
--
pull_requests: +16502
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16975
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
> Regarding 3.8 release notes update -- not sure if it is needed flr docs-only
> change.
I'm not certain if the entry is necessary; my main concern is just that it's
already present in the 3.8 release notes/whatsnew without anywhere to look for
f
Kyle Stanley added the comment:
Sounds good, I'll work on opening a PR.
--
___
Python tracker
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list m
Kyle Stanley added the comment:
> Is anyone able to make a call on whether this issue should be closed, or
> alternatively give some guidance on what work this issue should encompass?
Added Serhiy to the nosy list, since he's an active maintainer for the re
module.
--
nosy:
Change by Kyle Stanley :
--
nosy: +serhiy.storchaka
___
Python tracker
<https://bugs.python.org/issue21002>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> It was the first time I saw a deprecation clearly documented in a final
> release that didn't have an associated deprecation warning.
I want to clarify that this may be a more common occurrence than I'm realizing,
I'm not entirely certain. Als
Kyle Stanley added the comment:
> I think we can add `DeprecationWarning` for 3.9.
If we add the deprecation warning just for 3.9, would the removal release also
be pushed forward?
> Honestly, we just missed the issue when were prepared for 3.8
Yeah that's definitely understa
Kyle Stanley added the comment:
> Actually, since Andrew also agrees that we need to deprecate passing
> coroutines to wait(), I'll keep this issue open until we add an actual
> DeprecationWarning in 3.8.
Since 3.8 has been released and the deprecation notice is in the 3.8
Kyle Stanley added the comment:
> But it spawns a new Python thread per process which can be a blocker issue if
> a server memory is limited.
I understand that there's *some* overhead associated with spawning a new
thread, but from my impression it's not substantial enough t
Kyle Stanley added the comment:
> > If asyncio is only run from the main thread, FastChildWatcher is safe, fast
> > and has low memory footprint, no?
> Unfortunately, no. FastChildWatcher is safe if you can guarantee that no code
> executed in asyncio main thread AND
Kyle Stanley added the comment:
> I think FastChildWatcher and SafeChildWatcher should go, ThreadedChildWatcher
> should be kept default and MultiLoopChildWatcher is an option where
> ThreadedChildWatcher is not satisfactory.
Okay, I think I can understand the reasoning here. Do
Kyle Stanley added the comment:
> Kyle, why are you resetting the status to "Pending"?
That was an accident, I think I had that set already on the page and submitted
my comment just after you did yours.
I'm going to change the title of the issue to "Deprecate Process C
Change by Kyle Stanley :
--
title: Deprecating MultiLoopChildWatcher -> Deprecate Process Child Watchers
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
> Speaking of watchers -- big +1 from me to drop them all at some point. I
> would start as early as 3.9.
Yeah that was my initial plan, to start the deprecation in 3.9 and finalize the
removal in 3.11. We might be able to make an exc
Kyle Stanley added the comment:
> Didn't we just add MultiLoopChildWatcher in 3.8?
Yep, that's correct:
https://docs.python.org/3/library/asyncio-policy.html#asyncio.MultiLoopChildWatcher
I wasn't aware of that, but it would explain the current lack of usage. I'm
generally in fa
Change by Kyle Stanley :
--
assignee: -> aeros
nosy: +vstinner
status: open -> pending
___
Python tracker
<https://bugs.python.org/issue38591>
___
___
Pyth
New submission from Kyle Stanley :
Proposal:
Deprecate the alternative process watcher implementation to
ThreadedChildWatcher, MultiLoopChildWatcher.
Motivation:
The idea for this proposal came from a comment from Andrew Svetlov in GH-16552:
"I believe after polishing ThreadedChildWatch
Kyle Stanley added the comment:
> IHMO a test must not depend on time.
I would agree, the tests should optimally not depend on time. My above comment
was primarily to confirm that the failure was able to be consistently
reproduced, along with the minimum conditions to do so.
Yury had b
Kyle Stanley added the comment:
> I'm able to reproduce the issue locally using the command:
./python -m test test_asyncio
--match=test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests.test_close_kill_running
-v -F -j20 --timeout=30.0
I was unable to reproduce this locally on
Kyle Stanley added the comment:
I can confirm Victor's method of reproducing the failure consistently, by using
asyncio.sleep(1e-9) within `RunCoroutineThreadsafeTests.add()` instead of the
current asyncio.sleep(0.05).
I also experimented with adjusting the sleep time, to figure out
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue38323>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> Is it still open? What else needs to be done?
Yes, this patch needs to be translated into a GitHub PR. See
https://devguide.python.org/pullrequest/ for more information on our PR
workflow if you're not already familiar with it. Since naught101 wrote a pa
es
I have no personal expeience with this package.
--
Stanley C. Kitching
Human Being
Phoenix, Arizona
--
https://mail.python.org/mailman/listinfo/python-list
Cousin Stanley wrote:
> There is also a useful python package
> called sunset which I fouund a reference to
> on stackoverflow
Maybe I'll wake up some time today :-)
The python package is called suntime
not sunset
# pip3 show suntime
--
Stanley C. Kitching
Hu
Cousin Stanley wrote:
>
> d_sse = sse_sunset - sse_sunrise # seconds of daylight
>
I think it might be required to convert
utc time to local time for the difference
in sunrise and sunset times to make sense
in local time ....
--
Stanley C. Kitching
Human Bein
longitude are known
sunrise and sunset times are available
$ cat daylight_phoenix.py
#!/usr/bin/env python3
'''
NewsGroup comp.lang.python
Subject .. question about maiking an App for Android
Date . 2019-10-10
Post_By .. pyotr filipivich
Code
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue28533>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue31387>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> Kyle Stanley proposed a fix: PR 16293.
Note that I'm not confident about the fix I proposed in GH-16293, it was more
of an idea to fix the intermittent failures more than anything. It definitely
needs review from someone knowledgeable about sub-interpret
Change by Kyle Stanley :
--
stage: backport needed -> commit review
___
Python tracker
<https://bugs.python.org/issue38355>
___
___
Python-bugs-list mai
Change by Kyle Stanley :
--
stage: backport needed -> commit review
___
Python tracker
<https://bugs.python.org/issue38359>
___
___
Python-bugs-list mai
Change by Kyle Stanley :
--
stage: patch review -> commit review
___
Python tracker
<https://bugs.python.org/issue38344>
___
___
Python-bugs-list mai
Kyle Stanley added the comment:
> For the most part, templating examples can be switched to the .format() style
> but shouldn't be switched to f-strings.
Is there no specific use case for the older "%s" % sub template that .format()
doesn't have?
> The former technique
Change by Kyle Stanley :
--
keywords: +patch
pull_requests: +16140
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16552
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
> Another consideration is if we want this method to join the threads to be
> called in `ThreadedChildWatcher.close()`.
An additional benefit of having the method called from `close()` is that it
means we don't have to modify the tests directly
Kyle Stanley added the comment:
First I'll work on adding a new method. Here's a few potential names, ordered
roughly by my preferences:
1) join_threads()
2) shutdown_threads()
3) shutdown_threadpool()
The first and second options are roughly equal, but I think join_threads() is a
bit
Kyle Stanley added the comment:
> I definitely think we should not modify any code in the stdlib just to switch
> to f-strings.
Does this also apply to updating code to use f-strings in an area that's
already being modified for a functional purpose?
I agree that that we shouldn't
Kyle Stanley added the comment:
I can try to work on fixing this.
--
nosy: +aeros167
___
Python tracker
<https://bugs.python.org/issue38356>
___
___
Python-bug
Kyle Stanley added the comment:
> so it would be a good candidate for the "newcomer friendly" label
Never mind, just noticed this was already labeled as newcomer friendly. I only
saw the "easy" label at first. (:
If consensus is reached for this, we can open a separa
Kyle Stanley added the comment:
Welcome back from the OOOS break Mariatta!
> My question (and it's just that) is whether we've made a decision to prefer
> one formatting syntax over others (outside of examples discussing the
> formatting approaches themselves).
I agree that we sho
Kyle Stanley added the comment:
Closed by the new asyncio stream API reversion in GH-16485 and GH-16482.
--
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/i
Change by Kyle Stanley :
--
status: closed -> open
___
Python tracker
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Kyle Stanley :
--
stage: resolved -> commit review
___
Python tracker
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Un
Kyle Stanley added the comment:
> I've reverted the code. Andrew, would really appreciate if you could quickly
> do a post commit review.
Oops, I'll reopen it.
--
___
Python tracker
<https://bugs.python.org/i
Change by Kyle Stanley :
--
stage: commit review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue38242>
___
___
Pyth
Kyle Stanley added the comment:
This should no longer be a release blocker for 3.8 with the reversion of the
new asyncio streaming API in GH-16455.
--
nosy: +aeros167
___
Python tracker
<https://bugs.python.org/issue38
Change by Kyle Stanley :
--
pull_requests: +16035
pull_request: https://github.com/python/cpython/pull/16455
___
Python tracker
<https://bugs.python.org/issue38
Change by Kyle Stanley :
--
keywords: +patch
pull_requests: +16024
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16445
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
Currently focusing on the Lib/asyncio/* and Lib/test/* changes. Working on doc
changes next, but that should be significantly easier.
In addition to
https://github.com/python/cpython/commit/23b4b697e5b6cc897696f9c0288c187d2d24bff2
(main commit from Andrew
Kyle Stanley added the comment:
> You'll need to be careful to only revert the new functions & the
> asyncio.Stream class.
So far the trickiest part has proven to be the tests (specifically
test_streams.py) and keeping the deprecation warning for passing explicit loop
arguments
Kyle Stanley added the comment:
> Andrew, do you want me to submit a PR or you can do it?
Since this has been elevated to a release blocker, I wouldn't mind helping to
revert this ASAP. I can open a PR to fix it today.
--
___
Python trac
Change by Kyle Stanley :
--
keywords: +patch
pull_requests: +15941
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16360
___
Python tracker
<https://bugs.python.org/issu
New submission from Kyle Stanley :
Currently, for the recently added coroutine `loop.shutdown_default_executor()`,
the executor shutdown can wait indefinitely for the threads to join. Under
normal circumstances, waiting on the threads is appropriate, but there should
be a timeout duration
Kyle Stanley added the comment:
> We have to document that Process objects use a third kind of stream object
> that doesn't match either the old or new APIs, and how this one works
>From my perspective, this point would have the largest user learning cost due
>to the stream o
Change by Kyle Stanley :
--
nosy: +aeros167
___
Python tracker
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> Is there an "aesthetic code cleanup" patch that's ever turned out well? ;-)
>From what I can tell, any remotely extensive aesthetic code patch I've seen
>has been highly controversial. I think they can have value in some cases, but
Kyle Stanley added the comment:
> You can find my email in Git, and I'm on Zulip and Discourse; and I'd be
> happy to start or follow a thread in a forum you think appropriate. Or if
> you'd rather drop it entirely, that's fine too.
I think opening a thread in https://discuss.pyt
Kyle Stanley added the comment:
For an updated explanation of the PR, see https://bugs.python.org/msg352835 or
the comments in the PR itself.
--
stage: patch review ->
___
Python tracker
<https://bugs.python.org/issu
Change by Kyle Stanley :
--
keywords: +patch
pull_requests: +15879
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16293
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
Is there a currently reliable way of accessing the GIL functions within the
sub-interpreters, without causing deadlock issues? I was trying to follow the
advice in the documentation
(https://docs.python.org/3/c-api/init.html?highlight=global%20interpreter
Kyle Stanley added the comment:
> I'm one of the first to advocate to replace ugly macros with clean static
> inline functions. Macros are evil and can be too easily misused.
As someone who has only more recently started learning the C-API (and C in
general), I'm certainly in
Kyle Stanley added the comment:
> Thanks, Kyle!
No problem, and thanks for all of the help from Andrew, Yury, and Victor!
> IMHO it will make asyncio more reliable, especially for tests on the CI.
Awesome, that was my primary intention. (:
> If it becomes an issue in Python 3.9
Kyle Stanley added the comment:
Upon further consideration, I don't think this will address the issue. If the
RuntimeError was not being raised, this failure would be consistent rather than
intermittent.
I think I have have gotten a bit mixed up, even if the return value of
PyErr_Format
Kyle Stanley added the comment:
I believe I found a potential fix, see
https://bugs.python.org/issue37224?@ok_message=msg%20352516%20created%0Aissue%2037224%20message_count%2C%20messages%20edited%20ok&@template=item#msg352514.
Should I attach the PR to that issue or this
Kyle Stanley added the comment:
Clarification:
"If I'm not mistaken doesn't mean that the `return -1` within ..."
Should instead be:
"If I'm not mistaken doesn't this mean that the `return -1` within ..."
(I am really looking forward to moving issue over to GitHub at so
Kyle Stanley added the comment:
Upon further reading of the documentation and some experimentation, it would
definitely not make sense to call `PyInterpreterState_Delete` here (since we're
only closing the sub-interpreter in the current thread), so that's not the
issue. I still have no idea
Kyle Stanley added the comment:
Clarification: In the above comment when I was referring to the commit made by
Eric Snow, I meant to link to
https://github.com/python/cpython/commit/7f8bfc9b9a8381ddb768421b5dd5cbd970266190.
--
___
Python tracker
Kyle Stanley added the comment:
Note that I'm not particularly experienced with the c-api, I'm just trying to
take a stab at understanding why the buildbot failure is occuring.
--
___
Python tracker
<https://bugs.python.org/issue37
Kyle Stanley added the comment:
Upon digging through Modules/_xxsubinterpretersmodule.c, I noticed that on line
2059, `PyInterpreterState_Delete(interp);` is commented out
(https://github.com/python/cpython/blob/bf169915ecdd42329726104278eb723a7dda2736/Modules/_xxsubinterpretersmodule.c
Kyle Stanley added the comment:
> FWIW, I've confirmed again that the exact same script on the same machine
> seems fine under Python 3.x. Given the imminent demise of Python 2, perhaps
> this issue is just destined to be an unsolved historical oddity.
Since it doesn't seem to be
Kyle Stanley added the comment:
> Thanks you Kyle!
No problem, thanks for merging it Antoine!
What do you think of the followup PR to make use of the SEEK_* constants listed
in the documentation? I think it would be useful to at least mention them in
the tutorial, or even make use of t
Kyle Stanley added the comment:
Created GH-16147 for replacing the *from_what* argument with *whence* in the IO
tutorial.
I would like to consider following up on this with another PR that adds the IO
constants `SEEK_SET`, `SEEK_CUR`, and `SEEK_END` to the tutorial. Those
constants would
Change by Kyle Stanley :
--
keywords: +patch
pull_requests: +15757
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16147
___
Python tracker
<https://bugs.python.org/issu
Kyle Stanley added the comment:
I've opened PR-15735 which applies the same functionality as Victor's PR-13786,
but adds the public getter and setter methods (for both AbstractEventLoop and
BaseEventLoop) as requested by Andrew.
Since this is still causing intermittent CI failures from
301 - 400 of 651 matches
Mail list logo
