Re: Review Request 45911: HIVE-13341: Stats state is not captured correctly: differentiate load table and create table
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45911/ --- (Updated April 25, 2016, 5:53 a.m.) Review request for hive and Ashutosh Chauhan. Repository: hive-git Description --- HIVE-13341: Stats state is not captured correctly: differentiate load table and create table Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 41d150c itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java c046708 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 76220f4 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 54592cc ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java 87a7667 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4c9acce ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 549d24f ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 03b4d8b ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java eaeb66b ql/src/test/queries/clientpositive/insert_values_orig_table_use_metadata.q PRE-CREATION ql/src/test/queries/clientpositive/stats20.q 59701bd ql/src/test/results/clientnegative/alter_file_format.q.out 96f1bfb ql/src/test/results/clientnegative/stats_partialscan_autogether.q.out 4faf327 ql/src/test/results/clientnegative/unset_table_property.q.out 0510788 ql/src/test/results/clientpositive/allcolref_in_udf.q.out eda49ed ql/src/test/results/clientpositive/alter_file_format.q.out 5d83b23 ql/src/test/results/clientpositive/alter_skewed_table.q.out a1caa99 ql/src/test/results/clientpositive/alter_table_not_sorted.q.out 6e1ec59 ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out f768ea4 ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out f4bbfd0 ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out e5ff904 ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out 0de079d ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out a161f1c ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out 3421ab1 ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out 9c2663c ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out a25db01 ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out d22ba3b ql/src/test/results/clientpositive/binary_output_format.q.out 32b2feb ql/src/test/results/clientpositive/bucket1.q.out 0680176 ql/src/test/results/clientpositive/bucket2.q.out 958e556 ql/src/test/results/clientpositive/bucket4.q.out dd9ef00 ql/src/test/results/clientpositive/bucket5.q.out fa926aa ql/src/test/results/clientpositive/bucket_many.q.out a9e2f0b ql/src/test/results/clientpositive/bucket_map_join_1.q.out 05fb434 ql/src/test/results/clientpositive/bucket_map_join_2.q.out 1173c93 ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out 3b1912a ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 9a6aef7 ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out 1d2166f ql/src/test/results/clientpositive/bucketcontext_1.q.out 78a0c8c ql/src/test/results/clientpositive/bucketcontext_2.q.out bc08fd2 ql/src/test/results/clientpositive/bucketcontext_3.q.out 0cba2ca ql/src/test/results/clientpositive/bucketcontext_4.q.out 652b377 ql/src/test/results/clientpositive/bucketcontext_5.q.out ee4532a ql/src/test/results/clientpositive/bucketcontext_6.q.out e280b95 ql/src/test/results/clientpositive/bucketcontext_7.q.out b44ae95 ql/src/test/results/clientpositive/bucketcontext_8.q.out eee7b57 ql/src/test/results/clientpositive/bucketmapjoin1.q.out ac8a76f ql/src/test/results/clientpositive/bucketmapjoin10.q.out 041f2d0 ql/src/test/results/clientpositive/bucketmapjoin11.q.out 5e9fbe4 ql/src/test/results/clientpositive/bucketmapjoin12.q.out 3a38d7b ql/src/test/results/clientpositive/bucketmapjoin2.q.out d9c4463 ql/src/test/results/clientpositive/bucketmapjoin3.q.out 77b19ba ql/src/test/results/clientpositive/bucketmapjoin4.q.out 5858160 ql/src/test/results/clientpositive/bucketmapjoin5.q.out 728c0b7 ql/src/test/results/clientpositive/bucketmapjoin7.q.out 7f96a23 ql/src/test/results/clientpositive/bucketmapjoin8.q.out 804c882 ql/src/test/results/clientpositive/bucketmapjoin9.q.out cc0672e ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 36b1cfe ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 21138d6 ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out c95681a ql/src/test/results/clientpositive/cbo_SortUnionTransposeRule.q.out 7542882 ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 89f1746 ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out 8c9664d
[jira] [Created] (HIVE-13603) Fix ptest unit tests broken by HIVE13505
Siddharth Seth created HIVE-13603: - Summary: Fix ptest unit tests broken by HIVE13505 Key: HIVE-13603 URL: https://issues.apache.org/jira/browse/HIVE-13603 Project: Hive Issue Type: Task Affects Versions: 2.1.0 Reporter: Siddharth Seth Assignee: Siddharth Seth HIVE-13505 broke some unit tests in the ptest2 framework, which need to be fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13602) TPCH q16 return wrong result when CBO is on
Nemon Lou created HIVE-13602: Summary: TPCH q16 return wrong result when CBO is on Key: HIVE-13602 URL: https://issues.apache.org/jira/browse/HIVE-13602 Project: Hive Issue Type: Bug Components: CBO, Logical Optimizer Affects Versions: 1.2.1 Reporter: Nemon Lou Running tpch with factor 2, q16 returns 1,160 rows when CBO is on, while returns 59,616 rows when CBO is off. See attachment for detail . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13601) Deadline is not registered at some places in HiveMetastore before the ObjectStore/MetaStoreDirectSQL APIs are invoked
Hari Sankar Sivarama Subramaniyan created HIVE-13601:
Summary: Deadline is not registered at some places in
HiveMetastore before the ObjectStore/MetaStoreDirectSQL APIs are invoked
Key: HIVE-13601
URL: https://issues.apache.org/jira/browse/HIVE-13601
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
HIVE-13542 exposed this issue as shown below when making Metastore API calls
via CBO code path.
{code}
16-04-21T11:45:55,201 DEBUG [f2995fd1-e4fc-4783-86cb-9264f715e540 main[]]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(864)) -
Direct SQL query in 14.825303ms + 0.086336ms, the query is [select
"COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE",
"DOUBLE_LOW_VALU\
E", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE",
"NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES",
"NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and
"TABLE_NAME" = ? and "COLUMN_NAME" in (...)]
2016-04-21T11:45:55,242 WARN [f2995fd1-e4fc-4783-86cb-9264f715e540 main[]]:
metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(2602)) - Direct
SQL failed, falling back to ORM
org.apache.hadoop.hive.metastore.api.MetaException: The threadlocal Deadline is
null, please register it first.
at
org.apache.hadoop.hive.metastore.Deadline.newMetaException(Deadline.java:178)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.Deadline.checkTimeout(Deadline.java:149)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.makeColumnStats(MetaStoreDirectSql.java:1717)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getTableStats(MetaStoreDirectSql.java:1201)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$7.getSqlResult(ObjectStore.java:6849)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$7.getSqlResult(ObjectStore.java:6846)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2567)
[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatisticsInternal(ObjectStore.java:6845)
[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatistics(ObjectStore.java:6839)
[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_statistics_req(HiveMetaStore.java:4361)
[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableColumnStatistics(HiveMetaStoreClient.java:1613)
[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTableColumnStatistics(SessionHiveMetaStoreClient.java:347)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:3317)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:758)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.updateColStats(RelOptHiveTable.java:302)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:417)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:410)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan.getColStat(HiveTableScan.java:158)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:77)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:64)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.7.0_79]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[?:1.7.0_79]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
