[issue29595] Expose max_queue_size in ThreadPoolExecutor

2021-10-14 Thread Jack Wong 


Change by Jack Wong :


--
nosy: +iforapsy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2020-12-19 Thread tianc777 


Change by tianc777 <18702287...@163.com>:


--
nosy: +tianc777
nosy_count: 9.0 -> 10.0
pull_requests: +22727
pull_request: https://github.com/python/cpython/pull/23865

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2020-12-19 Thread Roundup Robot 


Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 8.0 -> 9.0
pull_requests: +22726
pull_request: https://github.com/python/cpython/pull/23864

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2018-03-31 Thread Antoine Pitrou 

Antoine Pitrou  added the comment:

Please note the PR here has some review comments that need addressing.
Also, it needs its conflicts with git master resolved.

I'm cc'ing Thomas Moreau, who has done a lot of work recently on the 
concurrent.futures internals.

--
assignee: davin -> 
nosy: +tomMoral
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2018-03-30 Thread Stephen ONeal 

Stephen ONeal  added the comment:

My project we're going into the underlying _work_queue and blocking adding more 
elements based on unfinished_tasks to accomplish this, bubbling this up to the 
API would be a welcome addition.

--
nosy: +stephen.oneal...@gmail.com

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-03-31 Thread Raymond Hettinger 

Raymond Hettinger added the comment:

Prayslayer, please don't shove.  Your PR request was responded to by Mariatta 
so it wasn't ignored.  

Making decisions about API expansions takes a while (making sure it fits the 
intended use, that it isn't a bug factory itself, that it is broadly useful, 
that it is the best solution to the problem, that is doesn't complicate the 
implementation or limit future opportunities, that there are unforeseen 
problems).   Among the core developers, there are only a couple part-time 
contributors who are qualified to make these assessments for the 
multi-processing module (those devs don't include me).

--
assignee:  -> davin
nosy: +davin, pitrou, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-03-31 Thread Mariatta Wijaya 

Changes by Mariatta Wijaya :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-03-24 Thread prayerslayer 

prayerslayer added the comment:

Ping. That's really a two-line change, can be easily reviewed in 15 minutes :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-03-06 Thread prayerslayer 

prayerslayer added the comment:

Hello again, there's a reviewed PR open for this issue and it hasn't even 
received authoritative feedback yet (ie whether or not you intend to support 
this feature at all). I would be very happy if a core dev could look over this 
change before everyone forgets about it :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-02-20 Thread Xiang Zhang 

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-02-18 Thread Jim Fasarakis-Hilliard 

Changes by Jim Fasarakis-Hilliard :


--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-02-18 Thread Mariatta Wijaya 

Changes by Mariatta Wijaya :


--
components: +Library (Lib)
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29595] Expose max_queue_size in ThreadPoolExecutor

2017-02-17 Thread prayerslayer 

New submission from prayerslayer:

Hi!

I think the ThreadPoolExecutor should allow to set the maximum size of the 
underlying queue.

The situation I ran into recently was that I used ThreadPoolExecutor to 
parallelize AWS API calls; I had to move data from one S3 bucket to another 
(~150M objects). Contrary to what I expected the maximum size of the underlying 
queue doesn't have a non-zero value by default. Thus my process ended up 
consuming gigabytes of memory, because it put more items into the queue than 
the threads were able to work off: The queue just kept growing. (It ran on K8s 
and the pod was rightfully killed eventually.)

Of course there ways to work around this. One could use more threads, to some 
extent. Or you could use your own queue with a defined maximum size. But I 
think it's more work for users of Python than necessary.

--
messages: 288043
nosy: prayerslayer
priority: normal
pull_requests: 104
severity: normal
status: open
title: Expose max_queue_size in ThreadPoolExecutor
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com