tvalentyn commented on code in PR #38407:
URL: https://github.com/apache/beam/pull/38407#discussion_r3223776540
##########
sdks/python/apache_beam/runners/worker/data_plane.py:
##########
@@ -455,11 +456,14 @@ class _GrpcDataChannel(DataChannel):
def __init__(self, data_buffer_time_limit_ms=0):
# type: (int) -> None
+
self._data_buffer_time_limit_ms = data_buffer_time_limit_ms
- self._to_send = queue.Queue() # type: queue.Queue[DataOrTimers]
+ self._to_send = ByteLimitedQueue(
+ maxsize=10000,
Review Comment:
one interesting consideration is comparing the overheads of spinning up a
thread to wait on the condition vs just letting an element land on the queue if
its size is small... it might be worth introducing min_size of maybe 32kb or
whatever the minimum overheads are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
