HyukjinKwon opened a new pull request, #46993: URL: https://github.com/apache/spark/pull/46993
### What changes were proposed in this pull request? This PR propose to avoid having `ExecutePlanResponseReattachableIterator._release_thread_pool` to initialize `ThreadPool`. ### Why are the changes needed? This instance might be dragged in during pickle because it's statically initialized. ``` _release_thread_pool: Optional[ThreadPool] = ThreadPool(os.cpu_count() if os.cpu_count() else 8) File "/usr/lib/python3.10/multiprocessing/pool.py", line 930, in __init__ Pool.__init__(self, processes, initializer, initargs) File "/usr/lib/python3.10/multiprocessing/pool.py", line 196, in __init__ self._change_notifier = self._ctx.SimpleQueue() File "/usr/lib/python3.10/multiprocessing/context.py", line 113, in SimpleQueue return SimpleQueue(ctx=self.get_context()) File "/usr/lib/python3.10/multiprocessing/queues.py", line 341, in __init__ self._rlock = ctx.Lock() File "/usr/lib/python3.10/multiprocessing/context.py", line 68, in Lock return Lock(ctx=self.get_context()) File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__ SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx) File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__ sl = self._semlock = _multiprocessing.SemLock( PermissionError: [Errno 13] Permission denied ``` which requires to change in OS level. ### Does this PR introduce _any_ user-facing change? Yeah, potentially this could trigger some random job failures in some environment like Ubuntu ### How was this patch tested? Manually tested. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org