[jira] [Commented] (HIVE-28182) Logs page doesn't load in Hive UI
[ https://issues.apache.org/jira/browse/HIVE-28182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837932#comment-17837932 ] Zhihua Deng commented on HIVE-28182: It works for me as well... > Logs page doesn't load in Hive UI > - > > Key: HIVE-28182 > URL: https://issues.apache.org/jira/browse/HIVE-28182 > Project: Hive > Issue Type: Sub-task >Reporter: Dmitriy Fingerman >Priority: Major > Attachments: Screen Shot 2024-04-05 at 5.16.26 PM.png, > image-2024-04-06-02-51-24-363.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-28198) Trino table is recognized as EXTERNAL_TABLE regardless of external_location parameter
[
https://issues.apache.org/jira/browse/HIVE-28198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837931#comment-17837931
]
Zhihua Deng commented on HIVE-28198:
In Hive 4.0, we treat the Hive acid table as the managed table by default, for
the legacy managed table, it's translated to an external table with parameter:
TRANSLATED_TO_EXTERNAL=true, external.table.purge=true, the legacy external
table works as the same.
> Trino table is recognized as EXTERNAL_TABLE regardless of external_location
> parameter
> -
>
> Key: HIVE-28198
> URL: https://issues.apache.org/jira/browse/HIVE-28198
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Mladjan Gadzic
>Priority: Major
>
> {code:java}
> trino > create table hive.default.test_table(id int);{code}
> {code:java}
> trino> delete from hive.default.test_table;
> Query 20240402_103228_00042_hm8m3, FAILED, 1 node Splits: 1 total, 0 done
> (0.00%) 0.08 [0 rows, 0B] [0 rows/s, 0B/s]
> Query 20240402_103228_00042_hm8m3 failed: Cannot delete from non-managed Hive
> table{code}
> This behavior is tested and works as expected in Hive 3. Table type is stored
> in HMS DB in {{TBLS}} table {{TBL_TYPE}} field. For Hive 3 value is
> MANAGED_TABLE and EXTERNAL_TABLE for Hive 4.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-28199) Docker quickstart does not work for Hive 3.1.3 on Mac M2
[
https://issues.apache.org/jira/browse/HIVE-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng resolved HIVE-28199.
Fix Version/s: 4.1.0
Resolution: Fixed
Fix has been merged. Thank you for the PR [~ryandgoldenberg]!
> Docker quickstart does not work for Hive 3.1.3 on Mac M2
>
>
> Key: HIVE-28199
> URL: https://issues.apache.org/jira/browse/HIVE-28199
> Project: Hive
> Issue Type: Bug
>Reporter: Ryan Goldenberg
>Assignee: Ryan Goldenberg
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> Quickstart:
> [https://hive.apache.org/developement/quickstart/#--hiveserver2-metastore]
> On Mac M2, {{docker-compose up}} for {{HIVE_VERSION=3.1.3}} gives the
> following errors
> * {{/home/hive/.beeline}} directory issue
> {quote}metastore | *** schemaTool failed ***
> metastore | [
> metastore | WARN] Failed to create directory:
> metastore | /home/hive/.beeline
> metastore | No such file or directory
> {quote} * Underscore in network name, from {{/tmp/hive/hive.log}} on
> {{{}hiveserver2{}}}:
> {quote}2024-04-02T16:26:24,867 ERROR [main] utils.MetaStoreUtils: Got
> exception: java.net.URISyntaxException Illegal character in hostname at index
> 25: thrift://metastore.docker_default:9083
> java.net.URISyntaxException: Illegal character in hostname at index 25:
> thrift://metastore.docker_default:9083
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28200) Improve get_partitions_by_filter/expr when partition limit enabled
[
https://issues.apache.org/jira/browse/HIVE-28200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28200:
--
Labels: pull-request-available (was: )
> Improve get_partitions_by_filter/expr when partition limit enabled
> --
>
> Key: HIVE-28200
> URL: https://issues.apache.org/jira/browse/HIVE-28200
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
>
> When {{hive.metastore.limit.partition.request}} is configured, HMS would get
> the matching partition counts before get the real partition objects. The
> count could be a slow query if the input filter or expr is too complex.
> In this case, such slow filter will be executed in both counting partition
> numbers and fetching real partition objects, which harms the performance and
> backend DBMS.
> We can make an improvement by getting matched partition names firstly, and
> then check limit through the size of partition names, and finally get the
> partitions by the partition names.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28200) Improve get_partitions_by_filter/expr when partition limit enabled
Wechar created HIVE-28200:
-
Summary: Improve get_partitions_by_filter/expr when partition
limit enabled
Key: HIVE-28200
URL: https://issues.apache.org/jira/browse/HIVE-28200
Project: Hive
Issue Type: Improvement
Components: Hive
Reporter: Wechar
Assignee: Wechar
When {{hive.metastore.limit.partition.request}} is configured, HMS would get
the matching partition counts before get the real partition objects. The count
could be a slow query if the input filter or expr is too complex.
In this case, such slow filter will be executed in both counting partition
numbers and fetching real partition objects, which harms the performance and
backend DBMS.
We can make an improvement by getting matched partition names firstly, and then
check limit through the size of partition names, and finally get the partitions
by the partition names.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26537) Deprecate older APIs in the HMS
[ https://issues.apache.org/jira/browse/HIVE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-26537: -- Labels: hive-4.0.1-must pull-request-available (was: hive-4.1.0-must pull-request-available) > Deprecate older APIs in the HMS > --- > > Key: HIVE-26537 > URL: https://issues.apache.org/jira/browse/HIVE-26537 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2 >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Critical > Labels: hive-4.0.1-must, pull-request-available > Time Spent: 7h 10m > Remaining Estimate: 0h > > This Jira is to track the clean-up(deprecate older APIs and point the HMS > client to the newer APIs) work in the hive metastore server. > More details will be added here soon. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HIVE-28198) Trino table is recognized as EXTERNAL_TABLE regardless of external_location parameter
[
https://issues.apache.org/jira/browse/HIVE-28198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837622#comment-17837622
]
Butao Zhang edited comment on HIVE-28198 at 4/16/24 9:28 AM:
-
{code:java}
metastore.metadata.transformer.class
{code}
You can disable the HMS transformer feature by set the
*metastore.metadata.transformer.class* value to {*}empty{*}.
was (Author: zhangbutao):
{code:java}
metastore.metadata.transformer.class
{code}
You can disable the HMS transformer feature by set the
*metastore.metadata.transformer.class* value to {*}empty{*}.
> Trino table is recognized as EXTERNAL_TABLE regardless of external_location
> parameter
> -
>
> Key: HIVE-28198
> URL: https://issues.apache.org/jira/browse/HIVE-28198
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Mladjan Gadzic
>Priority: Major
>
> {code:java}
> trino > create table hive.default.test_table(id int);{code}
> {code:java}
> trino> delete from hive.default.test_table;
> Query 20240402_103228_00042_hm8m3, FAILED, 1 node Splits: 1 total, 0 done
> (0.00%) 0.08 [0 rows, 0B] [0 rows/s, 0B/s]
> Query 20240402_103228_00042_hm8m3 failed: Cannot delete from non-managed Hive
> table{code}
> This behavior is tested and works as expected in Hive 3. Table type is stored
> in HMS DB in {{TBLS}} table {{TBL_TYPE}} field. For Hive 3 value is
> MANAGED_TABLE and EXTERNAL_TABLE for Hive 4.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HIVE-28198) Trino table is recognized as EXTERNAL_TABLE regardless of external_location parameter
[
https://issues.apache.org/jira/browse/HIVE-28198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837622#comment-17837622
]
Butao Zhang edited comment on HIVE-28198 at 4/16/24 9:27 AM:
-
{code:java}
metastore.metadata.transformer.class
{code}
You can disable the HMS transformer feature by set the
*metastore.metadata.transformer.class* value to {*}empty{*}.
was (Author: zhangbutao):
{code:java}
metastore.metadata.transformer.class
{code}
You can disable the HMS transformer feature by set the
*metastore.metadata.transformer.class* value to \{*}empty{*}.
> Trino table is recognized as EXTERNAL_TABLE regardless of external_location
> parameter
> -
>
> Key: HIVE-28198
> URL: https://issues.apache.org/jira/browse/HIVE-28198
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Mladjan Gadzic
>Priority: Major
>
> {code:java}
> trino > create table hive.default.test_table(id int);{code}
> {code:java}
> trino> delete from hive.default.test_table;
> Query 20240402_103228_00042_hm8m3, FAILED, 1 node Splits: 1 total, 0 done
> (0.00%) 0.08 [0 rows, 0B] [0 rows/s, 0B/s]
> Query 20240402_103228_00042_hm8m3 failed: Cannot delete from non-managed Hive
> table{code}
> This behavior is tested and works as expected in Hive 3. Table type is stored
> in HMS DB in {{TBLS}} table {{TBL_TYPE}} field. For Hive 3 value is
> MANAGED_TABLE and EXTERNAL_TABLE for Hive 4.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-28198) Trino table is recognized as EXTERNAL_TABLE regardless of external_location parameter
[
https://issues.apache.org/jira/browse/HIVE-28198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837622#comment-17837622
]
Butao Zhang commented on HIVE-28198:
{code:java}
metastore.metadata.transformer.class
{code}
You can disable the HMS transformer feature by set the
*metastore.metadata.transformer.class* value to \{*}empty{*}.
> Trino table is recognized as EXTERNAL_TABLE regardless of external_location
> parameter
> -
>
> Key: HIVE-28198
> URL: https://issues.apache.org/jira/browse/HIVE-28198
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Mladjan Gadzic
>Priority: Major
>
> {code:java}
> trino > create table hive.default.test_table(id int);{code}
> {code:java}
> trino> delete from hive.default.test_table;
> Query 20240402_103228_00042_hm8m3, FAILED, 1 node Splits: 1 total, 0 done
> (0.00%) 0.08 [0 rows, 0B] [0 rows/s, 0B/s]
> Query 20240402_103228_00042_hm8m3 failed: Cannot delete from non-managed Hive
> table{code}
> This behavior is tested and works as expected in Hive 3. Table type is stored
> in HMS DB in {{TBLS}} table {{TBL_TYPE}} field. For Hive 3 value is
> MANAGED_TABLE and EXTERNAL_TABLE for Hive 4.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-28127) Exception when rebuilding materialized view with calculated columns on iceberg sources
[
https://issues.apache.org/jira/browse/HIVE-28127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-28127.
---
Fix Version/s: 4.1.0
Resolution: Fixed
Merged to master. Thanks [~dkuzmenko] for review.
> Exception when rebuilding materialized view with calculated columns on
> iceberg sources
> --
>
> Key: HIVE-28127
> URL: https://issues.apache.org/jira/browse/HIVE-28127
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create external table tbl_ice(a int, b string, c int) stored by iceberg
> stored as orc tblproperties ('format-version'='1');
> insert into tbl_ice values (1, 'one', 50), (4, 'four', 53), (5, 'five', 54);
> create materialized view iceberg_mat2 stored by iceberg stored as orc
> tblproperties ('format-version'='2') as
> select tbl_ice.b, sum(tbl_ice.c), count(tbl_ice.c), avg(tbl_ice.c)
> from tbl_ice
> group by tbl_ice.b;
> insert into tbl_ice values (1, 'one', 50), (2, 'two', 51), (3, 'three', 52),
> (4, 'four', 53), (5, 'five', 54);
> alter materialized view iceberg_mat2 rebuild;
> {code}
> {code}
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid column
> reference '_c3'
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:13598)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:13540)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4931)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4719)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11554)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:11496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12432)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:12298)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:634)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13162)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:465)
> at
> org.apache.hadoop.hive.ql.ddl.view.materialized.alter.rebuild.AlterMaterializedViewRebuildAnalyzer.analyzeInternal(AlterMaterializedViewRebuildAnalyzer.java:178)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:732)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:702)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:115)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestIcebergLlapLocalCliDriver.testCliDriver(TestIcebergLlapLocalCliDriver.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.
