kamijin-fanta commented on issue #13738: URL: https://github.com/apache/iceberg/issues/13738#issuecomment-3157520729
Thanks for the detailed analysis. I totally agree. In our environment we run Iceberg in low-memory containers with very high append rates, so the cumulative GC impact made this leak immediately noticeable. Even if the per-entry overhead is modest, long-lived applications running for months can easily accumulate millions of entries and trigger OOMs, so it’s definitely worth removing the deleteOnExit() calls. I’ve also reviewed the Aliyun module and see the same pattern there—thank you for pointing it out. I checked our existing tests around currentStagingFile.deleteOnExit() and noticed that some temporary files still linger after test execution (likely because Gradle runs tasks in a way that doesn’t trigger the JVM shutdown hook). In any case, leaving a few stray files in test or crash scenarios seems like a minor trade-off compared to unbounded memory growth in production. I’ll prepare a PR to remove the deleteOnExit() calls from both AWS and Aliyun output streams. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
