Bharath Vissapragada has posted comments on this change. Change subject: IMPALA-2347: Reuse metastore client connections in Catalog ......................................................................
Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/3381/3//COMMIT_MSG Commit Message: PS3, Line 13: 10 > You may want to set the load_catalog_in_background to true and measure the In my local setting I don't see much difference in the startup time. One thing I noticed though is that there were >1000 thread pool hits (instead of creating a new connection). I think its much more effective in larger catalogs and kerberized environments where a new connection to the metastore is costly and involves a roundtrip to the KDC, although I don't have the numbers to back this up. As discussed in the chat, please send me that zip so that I can profile it further (probably on a per connection basis). http://gerrit.cloudera.org:8080/#/c/3381/3/fe/src/main/java/com/cloudera/impala/catalog/MetaStoreClientPool.java File fe/src/main/java/com/cloudera/impala/catalog/MetaStoreClientPool.java: PS3, Line 50: MetaStoreClient > If you're up to it it would be nice if this class implemented the AutoClose Nice idea. Done. -- To view, visit http://gerrit.cloudera.org:8080/3381 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I517c0e1efef2584cd8d34017b33574f2ad69bd52 Gerrit-PatchSet: 3 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Huaisi Xu <h...@cloudera.com> Gerrit-HasComments: Yes
