gaborkaszab commented on PR #15954: URL: https://github.com/apache/iceberg/pull/15954#issuecomment-4259266611
Hey @manuzhang , Thanks for the PR. I started taking a look as the owner of the original code, but frankly I found the code somewhat complicated to understand. So before taking a deeper look, may I ask the motivation and the problem being solved in more details. What is the exact issue, what engine, what use-case, what error, what breaks, etc. Is this a real issue you see in some test/prod environment or a theoretical one that you think might break one time? The reason I ask is that most of the engines use RESTSessionCatalog through a single RESTCatalog and as a result through the same session (and even many different users share the same session). I've checked Spark, Hive, Impala does this if I haven't missed anything, while Trino creates SessionContexts on its own. Also, the idea to balance between tables being cached and credentials expiring, the general rule of thumb is to set the expiration on the cache way lower than the expiration of the credentials to guarantee that whenever getting a table from the cache, its creds will live for enough time to finish the query. Isn't this something that helps with you use-case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
