rdblue commented on issue #1502: URL: https://github.com/apache/iceberg/issues/1502#issuecomment-698635234
Yeah, I think that the lifecycle of the client pool should be the same as the catalog. We want to reuse a client pool to avoid opening too many connections to the Hive MetaStore the catalog is configured to use. The most reasonable place to do that is with the catalog. We could separate them, so the catalog is passed a client pool, but it isn't clear to me what that would accomplish: why would we create identical catalogs that use the same external pool? That would also be difficult to manage because we would have to make sure that the pool's connections are made to the same HMS that the catalog is configured to use. If it doesn't make sense to have an external client pool and we want the client pool to be reused and have a long lifecycle, then it makes sense to keep the catalog around to manage that lifecycle. Sounds like the problem here is that code is creating and discarding a catalog before the tables that were created by it are no longer needed. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
