[
https://issues.apache.org/jira/browse/HIVE-30238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30238:
----------------------------------
Description:
{{GetHelper.run()}} uses direct SQL only when {{metastore.try.direct.sql}} is
true, and runs the JDO path only when {{canUseJdoQuery()}} is true. For a
helper whose only implementation is direct SQL ({{canUseJdoQuery()}} returns
false), turning direct SQL off makes {{run()}} execute neither path and return
null. The helpers' {{getJdoResult()}} errors ("make sure direct SQL is
enabled") are never reached.
Affected operations:
* alter_table with {{expected_parameter_key}}/{{expected_parameter_value}}
({{TableStoreImpl.updateParameterWithExpectedValue}}): the null is unboxed to
long, so the alter fails with
{noformat}java.lang.NullPointerException: Cannot invoke
"java.lang.Long.longValue()" because the return value of
"org.apache.hadoop.hive.metastore.metastore.impl.TableStoreImpl$18.run(boolean)"
is null{noformat}
Commits to Iceberg tables with {{engine.hive.lock-enabled=false}} fail with
{{CommitStateUnknownException}}, and Hive's Iceberg stats updates (HIVE-30229)
fail.
* update_table_params ({{TableStoreImpl.updateTableParams}}): writes nothing
but reports success, so {{HiveTxnCoordinator}} multi-table commits are silently
lost.
* {{ColStatsStoreImpl}} {{getTableNamesWithStats}},
{{getPartitionColsWithStats}}, {{get_aggr_stats_for}} and
{{getPartitionColStatsForDatabase}} return null.
* {{updatePartitionColumnStatisticsInBatch}} is not reached: its caller
({{SetAggrStatsHandler}}) takes the JDO path when direct SQL is off.
*Reproduce*: run an HMS with {{metastore.try.direct.sql=false}} (or {{SET
metaconf:metastore.try.direct.sql=false}} in a session), then alter a table
with an expected parameter, or insert into an Iceberg table with
{{engine.hive.lock-enabled=false}}.
*Proposed fix*: an operation without a JDO path runs with direct SQL regardless
of {{metastore.try.direct.sql}}. The setting chooses between direct SQL and JDO
only where both exist, and the notification log already uses direct SQL
regardless of it. In {{GetHelper}}: {{tryDirectSql = !allowJdo ||
TRY_DIRECT_SQL}}. On a datastore that direct SQL doesn't support, these
operations fail with "SQL is not operational".
was:
{{GetHelper.run()}} uses direct SQL only when {{metastore.try.direct.sql}} is
true, and runs the JDO path only when {{canUseJdoQuery()}} is true. For a
helper whose only implementation is direct SQL ({{canUseJdoQuery()}} returns
false), turning direct SQL off makes {{run()}} execute neither path and return
null. The helpers' {{getJdoResult()}} errors ("make sure direct SQL is
enabled") are never reached.
Affected operations:
* alter_table with {{expected_parameter_key}}/{{expected_parameter_value}}
({{TableStoreImpl.updateParameterWithExpectedValue}}): the null is unboxed to
long, so the alter fails with
{noformat}java.lang.NullPointerException: Cannot invoke
"java.lang.Long.longValue()" because the return value of
"org.apache.hadoop.hive.metastore.metastore.impl.TableStoreImpl$18.run(boolean)"
is null{noformat}
Commits to Iceberg tables with {{engine.hive.lock-enabled=false}} fail with
{{CommitStateUnknownException}}, and Hive's Iceberg stats updates (HIVE-30229)
fail.
* update_table_params ({{TableStoreImpl.updateTableParams}}): writes nothing
but reports success, so {{HiveTxnCoordinator}} multi-table commits are silently
lost.
* {{ColStatsStoreImpl}} {{getTableNamesWithStats}},
{{getPartitionColsWithStats}}, {{get_aggr_stats_for}} and
{{getPartitionColStatsForDatabase}} return null.
* {{updatePartitionColumnStatisticsInBatch}} is not reached: its caller
({{SetAggrStatsHandler}}) takes the JDO path when direct SQL is off.
* Reproduce*: run an HMS with {{metastore.try.direct.sql=false}} (or {{SET
metaconf:metastore.try.direct.sql=false}} in a session), then alter a table
with an expected parameter, or insert into an Iceberg table with
{{engine.hive.lock-enabled=false}}.
* Proposed fix*: an operation without a JDO path runs with direct SQL
regardless of {{metastore.try.direct.sql}}. The setting chooses between direct
SQL and JDO only where both exist, and the notification log already uses direct
SQL regardless of it. In {{GetHelper}}: {{tryDirectSql = !allowJdo ||
TRY_DIRECT_SQL}}. On a datastore that direct SQL doesn't support, these
operations fail with "SQL is not operational".
> HMS: operations that support only direct SQL do nothing when
> metastore.try.direct.sql=false
> -------------------------------------------------------------------------------------------
>
> Key: HIVE-30238
> URL: https://issues.apache.org/jira/browse/HIVE-30238
> Project: Hive
> Issue Type: Bug
> Reporter: Denys Kuzmenko
> Priority: Major
>
> {{GetHelper.run()}} uses direct SQL only when {{metastore.try.direct.sql}}
> is true, and runs the JDO path only when {{canUseJdoQuery()}} is true. For a
> helper whose only implementation is direct SQL ({{canUseJdoQuery()}} returns
> false), turning direct SQL off makes {{run()}} execute neither path and
> return null. The helpers' {{getJdoResult()}} errors ("make sure direct SQL is
> enabled") are never reached.
>
>
>
>
>
> Affected operations:
>
>
> * alter_table with {{expected_parameter_key}}/{{expected_parameter_value}}
> ({{TableStoreImpl.updateParameterWithExpectedValue}}): the null is unboxed to
> long, so the alter fails with
> {noformat}java.lang.NullPointerException: Cannot invoke
> "java.lang.Long.longValue()" because the return value of
>
>
> "org.apache.hadoop.hive.metastore.metastore.impl.TableStoreImpl$18.run(boolean)"
> is null{noformat}
>
> Commits to Iceberg tables with {{engine.hive.lock-enabled=false}} fail with
> {{CommitStateUnknownException}}, and Hive's Iceberg stats updates
> (HIVE-30229) fail.
> * update_table_params ({{TableStoreImpl.updateTableParams}}): writes
> nothing but reports success, so {{HiveTxnCoordinator}} multi-table commits
> are silently lost.
> * {{ColStatsStoreImpl}} {{getTableNamesWithStats}},
> {{getPartitionColsWithStats}}, {{get_aggr_stats_for}} and
> {{getPartitionColStatsForDatabase}} return null.
>
> * {{updatePartitionColumnStatisticsInBatch}} is not reached: its caller
> ({{SetAggrStatsHandler}}) takes the JDO path when direct SQL is off.
>
>
>
>
> *Reproduce*: run an HMS with {{metastore.try.direct.sql=false}} (or {{SET
> metaconf:metastore.try.direct.sql=false}} in a session), then alter a table
> with an expected parameter, or insert into an Iceberg table with
> {{engine.hive.lock-enabled=false}}.
>
>
>
>
>
> *Proposed fix*: an operation without a JDO path runs with direct SQL
> regardless of {{metastore.try.direct.sql}}. The setting chooses between
> direct SQL and JDO only where both exist, and the notification log already
> uses direct SQL regardless of it. In {{GetHelper}}: {{tryDirectSql =
> !allowJdo || TRY_DIRECT_SQL}}. On a datastore that direct SQL doesn't
> support, these operations fail with "SQL is not operational".
>
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
