[
https://issues.apache.org/jira/browse/LENS-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943089#comment-14943089
]
Deepak Barr commented on LENS-796:
----------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules .............................. SUCCESS [ 3.229 s]
[INFO] Lens ............................................... SUCCESS [ 4.324 s]
[INFO] Lens API ........................................... SUCCESS [ 29.459 s]
[INFO] Lens API for server and extensions ................. SUCCESS [ 26.582 s]
[INFO] Lens Cube .......................................... SUCCESS [07:00 min]
[INFO] Lens DB storage .................................... SUCCESS [ 27.595 s]
[INFO] Lens Query Library ................................. SUCCESS [ 23.146 s]
[INFO] Lens Hive Driver ................................... SUCCESS [04:15 min]
[INFO] Lens Driver for JDBC ............................... SUCCESS [ 49.816 s]
[INFO] Lens Elastic Search Driver ......................... SUCCESS [ 22.917 s]
[INFO] Lens Server ........................................ SUCCESS [11:09 min]
[INFO] Lens client ........................................ SUCCESS [ 45.979 s]
[INFO] Lens CLI ........................................... SUCCESS [03:45 min]
[INFO] Lens Examples ...................................... SUCCESS [ 11.075 s]
[INFO] Lens Distribution .................................. SUCCESS [ 12.269 s]
[INFO] Lens ML Lib ........................................ SUCCESS [01:58 min]
[INFO] Lens ML Ext Distribution ........................... SUCCESS [ 2.843 s]
[INFO] Lens Regression .................................... SUCCESS [ 15.780 s]
[INFO] Lens UI ............................................ SUCCESS [ 51.368 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:36 min
[INFO] Finished at: 2015-10-02T00:17:01+05:30
[INFO] Final Memory: 175M/3142M
[INFO] ------------------------------------------------------------------------
> Slow response times for /metastore/nativetables API
> ---------------------------------------------------
>
> Key: LENS-796
> URL: https://issues.apache.org/jira/browse/LENS-796
> Project: Apache Lens
> Issue Type: Improvement
> Reporter: Deepak Barr
> Assignee: Deepak Barr
> Attachments: LENS-796.patch
>
>
> To obtain list of native tables, CubeMetastoreService does the following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by
> looking at the table properties from the Table object. This table object is
> obtained with another Metastore call. So, If there are 'n' tables, there will
> be 'n' metastore calls.
> Here is the code snippet :
> private List<String> getTablesFromDB(LensSessionHandle sessionid,
> String dbName, boolean prependDbName)
> throws MetaException, UnknownDBException, HiveSQLException, TException,
> LensException {
> List<String> tables =
> getSession(sessionid).getMetaStoreClient().getAllTables(
> dbName);
> List<String> result = new ArrayList<String>();
> if (tables != null && !tables.isEmpty()) {
> Iterator<String> it = tables.iterator();
> while (it.hasNext()) {
> String tblName = it.next();
> org.apache.hadoop.hive.metastore.api.Table tbl =
> getSession(sessionid).getMetaStoreClient().getTable(dbName,
> tblName);
> if (tbl.getParameters().get(MetastoreConstants.TABLE_TYPE_KEY) ==
> null) {
> if (prependDbName) {
> result.add(dbName + "." + tblName);
> } else {
> result.add(tblName);
> }
> }
> }
> }
> return result;
> }
> Instead of this, we can directly fetch the list of table objects for our list
> of table names in a single API call using
> getMetaStoreClient().getTableObjectsByName() method.
> Currently, one of our databases contain 8000+ tables which leads to very very
> long response times.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
