[
https://issues.apache.org/jira/browse/HIVE-13542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247892#comment-15247892
]
Jesus Camacho Rodriguez commented on HIVE-13542:
------------------------------------------------
Thanks [~hsubramaniyan].
I have been running some tests using your patch e.g. query7.q. Examining the
logs I see the following message that seems to indicate we still have some kind
of problem with the stats:
{noformat}
Caused by: org.apache.hadoop.hive.metastore.DeadlineException: The threadlocal
Deadline is null, please register it first.
at
org.apache.hadoop.hive.metastore.Deadline.checkTimeout(Deadline.java:149)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$7.getJdoResult(ObjectStore.java:6686)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$7.getJdoResult(ObjectStore.java:6664)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2550)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatisticsInternal(ObjectStore.java:6663)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatistics(ObjectStore.java:6657)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_statistics_req(HiveMetaStore.java:4327)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableColumnStatistics(HiveMetaStoreClient.java:1570)
~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTableColumnStatistics(SessionHiveMetaStoreClient.java:347)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:3317)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
... 178 more
2016-04-19T07:23:58,718 ERROR [ee2214e0-ff62-4617-a51c-45dfba3ae1c0 main[]]:
calcite.RelOptHiveTable (RelOptHiveTable.java:updateColStats(395)) - No Stats
for default@customer_demographics, Columns: cd_demo_sk
2016-04-19T07:23:58,719 WARN [ee2214e0-ff62-4617-a51c-45dfba3ae1c0 main[]]:
parse.CalcitePlanner (CalcitePlanner.java:apply(993)) - Missing column stats
(see previous messages), skipping join reordering in CBO
{noformat}
In particular, that column is present in
{{data/files/tpcds-perf/metastore_export/csv/TAB_COL_STATS.txt}}.
I could avoid the DeadlineException by setting {{hive.metastore.fastpath}} to
false in {{data/conf/perf-reg/hive-site.xml}}, thus avoiding bypassing the raw
store proxy as we are using the ObjectStore in the PerfCliDriver. However, even
after doing that, I still see the missing stats message.
> Missing stats for tables in TPCDS performance regression suite
> --------------------------------------------------------------
>
> Key: HIVE-13542
> URL: https://issues.apache.org/jira/browse/HIVE-13542
> Project: Hive
> Issue Type: Bug
> Reporter: Hari Sankar Sivarama Subramaniyan
> Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13542.1.patch
>
>
> These are the tables whose stats are missing in
> data/files/tpcds-perf/metastore_export/csv/TAB_COL_STATS.txt:
> * catalog_returns
> * catalog_sales
> * inventory
> * store_returns
> * store_sales
> * web_returns
> * web_sales
> Thanks to [~jcamachorodriguez] for discovering this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
