[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096822#comment-17096822 ]
Hive QA commented on HIVE-23282: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/13001631/HIVE-23282.5.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 17190 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testPartitionFilter (batchId=155) org.apache.hadoop.hive.metastore.TestMetastoreExpr.testPartitionExpr (batchId=244) org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartitionFilter (batchId=157) org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreCustomAuth.testPartitionFilter (batchId=163) org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartitionFilter (batchId=156) org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartitionFilter (batchId=163) org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartitionFilter (batchId=153) org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartitionFilter (batchId=151) org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter (batchId=162) org.apache.hadoop.hive.ql.exec.TestMsckDropPartitionsInBatches.testBatchingWhenException (batchId=277) org.apache.hadoop.hive.ql.exec.TestMsckDropPartitionsInBatches.testEqualNumberOfPartitions (batchId=277) org.apache.hadoop.hive.ql.exec.TestMsckDropPartitionsInBatches.testNumberOfDropPartitionsCalls (batchId=277) org.apache.hadoop.hive.ql.exec.TestMsckDropPartitionsInBatches.testSmallNumberOfPartitions (batchId=277) org.apache.hadoop.hive.ql.exec.TestMsckDropPartitionsInBatches.testUnevenNumberOfCreatePartitionCalls (batchId=277) org.apache.hive.minikdc.TestRemoteHiveMetaStoreDualAuthCustom.testPartitionFilter (batchId=235) org.apache.hive.minikdc.TestRemoteHiveMetaStoreDualAuthKerb.testPartitionFilter (batchId=235) org.apache.hive.minikdc.TestRemoteHiveMetaStoreKerberos.testPartitionFilter (batchId=235) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/22037/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22037/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22037/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 17 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 13001631 - PreCommit-HIVE-Build > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > --------------------------------------------------------------------- > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore > Reporter: Rajesh Balamohan > Assignee: Attila Magyar > Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)