findepi commented on code in PR #8315: URL: https://github.com/apache/iceberg/pull/8315#discussion_r1650952000
########## aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java: ########## @@ -619,4 +635,17 @@ public void setConf(Configuration conf) { protected Map<String, String> properties() { return catalogProperties == null ? ImmutableMap.of() : catalogProperties; } + + private Cache<TableOperations, FileIO> newFileIOCloser() { + return Caffeine.newBuilder() + .weakKeys() + .removalListener( + (RemovalListener<TableOperations, FileIO>) + (ops, fileIO, cause) -> { + if (null != fileIO) { + fileIO.close(); Review Comment: This is not a regular cache, this could be better served with `java.lang.ref.Cleaner`. However, none of these addresses the main problem that we use Java GC as a proxy to discover and close resource leaks. I understand that fileIO instances can hold on to resources that require closing (like file descriptors or sockets inside AWS S3 client). If i have abundance of memory, the weakly held instances may not get collected, so the app can run out of descriptors. Can you please help me understand why this isn't a concern? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org