AMOOOMA commented on code in PR #37103:
URL: https://github.com/apache/beam/pull/37103#discussion_r2620944543
##########
sdks/python/apache_beam/utils/multi_process_shared.py:
##########
@@ -105,25 +113,47 @@ def __dir__(self):
dir = self._SingletonProxy_entry.obj.__dir__()
dir.append('singletonProxy_call__')
dir.append('singletonProxy_release')
+ dir.append('unsafe_hard_delete')
return dir
+def _run_with_oom_protection(func, *args, **kwargs):
Review Comment:
Yep! will definitely split this change up. The OOM protection here is
tricky, originally it's at the caller but I noticed the CUDA usage not being
dropped down because the process will be left in a broken state but yeah now
that I think about it I can probably pass it in as part of the constructor so
it's more natural. will update!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
