osscm commented on issue #15031: URL: https://github.com/apache/iceberg/issues/15031#issuecomment-3764189084
>> I think the best we can do is to (1) add a shutdown method for ThreadPools > We have some static pools in the ThreadPools, but ThreadPools.newExitingWorkerPool is used in 4 other places where it is used in static context, which has the exact same issue. Maybe we can create a variable which tracks the pools created by the newExitingWorkerPool method, and create a static method in ThreadPools which shuts down all of these pools. Option1 - This seems to be a safer and feasible option. > Finally, I wonder whether depending on static thread pools in Iceberg is a sound design pattern. We could at least add a way to disable creating them in a static way, such that methods like ThreadPools.getWorkerPool() can optionally return new thread pools which are managed by the user. >> It is possible to use the ThreadPools.newFixedThreadPool if we decide to do so. My understanding is that these use-cases specifically want a shareable, reusable worker pool which "globally" limit the number of threads for specific cases. + 1 Option2 - Creating a new thread-pool for based on a flag is also a safer option, but can start creating threads which are not controlled by Iceberg, and rely on the client app to control. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
