osscm commented on issue #15031: URL: https://github.com/apache/iceberg/issues/15031#issuecomment-3740350346
+1 to @RussellSpitzer's question. In systems like Tomcat or OSGi, applications are loaded and unloaded using different classloaders, so libraries are generally expected to clean up resources when an app is stopped. From a quick look at Flink (not a Flink expert), it seems Flink does something similar in session clusters. Different jobs run with their own classloaders, and Flink can unload them without restarting the JVM. This works fine as long as there are no running threads or other references that keep the classloader alive. This looks related to: https://github.com/apache/iceberg/issues/13058 — where threads kept growing unless Flink was restarted https://github.com/apache/iceberg/issues/12220 — about thread pools that rely on JVM shutdown hooks In a long-running Flink session cluster, the JVM usually does not restart when jobs stop. So relying only on JVM shutdown hooks to clean up thread pools may not be enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
