[ https://issues.apache.org/jira/browse/LENS-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935717#comment-14935717 ]
Deepak Barr commented on LENS-796: ---------------------------------- https://reviews.apache.org/r/38857/ > Slow response times for /metastore/nativetables API > --------------------------------------------------- > > Key: LENS-796 > URL: https://issues.apache.org/jira/browse/LENS-796 > Project: Apache Lens > Issue Type: Improvement > Reporter: Deepak Barr > Assignee: Deepak Barr > > To obtain list of native tables, CubeMetastoreService does the following - > 1. Fetches the list of tables ( one MetastoreClient call) > 2. Filters out the cube tables from the list. The filtering happens by > looking at the table properties from the Table object. This table object is > obtained with another Metastore call. So, If there are 'n' tables, there will > be 'n' metastore calls. > Here is the code snippet : > private List<String> getTablesFromDB(LensSessionHandle sessionid, > String dbName, boolean prependDbName) > throws MetaException, UnknownDBException, HiveSQLException, TException, > LensException { > List<String> tables = > getSession(sessionid).getMetaStoreClient().getAllTables( > dbName); > List<String> result = new ArrayList<String>(); > if (tables != null && !tables.isEmpty()) { > Iterator<String> it = tables.iterator(); > while (it.hasNext()) { > String tblName = it.next(); > org.apache.hadoop.hive.metastore.api.Table tbl = > getSession(sessionid).getMetaStoreClient().getTable(dbName, > tblName); > if (tbl.getParameters().get(MetastoreConstants.TABLE_TYPE_KEY) == > null) { > if (prependDbName) { > result.add(dbName + "." + tblName); > } else { > result.add(tblName); > } > } > } > } > return result; > } > Instead of this, we can directly fetch the list of table objects for our list > of table names in a single API call using > getMetaStoreClient().getTableObjectsByName() method. > Currently, one of our databases contain 8000+ tables which leads to very very > long response times. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)