flyrain commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1320364877
#5378 is a similar solution. One of reasons we didn't continue is that we also think about removing the `CachedClientPool`, which might be a better solution. The `CachedClientPool` was introduce by #2325, the idea to share the client pool for the same HMS makes sense at that time, but it seems no strong reason to do that since I think there won't be many catalogs within an app. For Spark, I never see any job uses more than two catalogs. Is it necessary to have a cache between them? Let me know if I missing something. cc @pvary. Also the `CachedClientPool` becomes less effective when we add more and more things into the keys. Each catalog within one application will have its own HMS client pool effectively. Considering the complex it adds to the system, we may remove it and let each catalog has its own HMS client pool. Debugging a concurrent bug is never a pleasant job. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
