lirui-apache commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1316470746
@pvary That's a good question. Unfortunately the test case you mentioned won't work. I looked into how UGI implements equals/hashCode and found the behavior is intentional [1]. Hadoop folks did mention that optimizations could be possible for proxy users, but it's not safe to do so on their end because UGI instances are mutable [2]. Instead they intend to leave that optimization to engines/services who have better knowledge about the UGI instances [3]. There were also reports about the UGI behavior causing memory issues in Hadoop FileSystem cache (which also uses UGI in the key) [4][5]. But as we have TTL on our cache entries, I guess we're fine in that regard. What do you think? [1] https://issues.apache.org/jira/browse/HADOOP-6670 [2] https://issues.apache.org/jira/browse/HADOOP-12529 [3] https://issues.apache.org/jira/browse/HADOOP-12529?focusedCommentId=14984176&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14984176 [4] https://issues.apache.org/jira/browse/HIVE-3098 [5] https://issues.apache.org/jira/browse/YARN-58 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
