[jira] [Commented] (HIVE-26473) Upgrade to Java17

2023-11-08 Thread Brahma Reddy Battula (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-26473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784277#comment-17784277
 ] 

Brahma Reddy Battula commented on HIVE-26473:
-

[~dingwei2019] thanks for reporting this.. any further updates on this.?

> Upgrade to Java17
> -
>
> Key: HIVE-26473
> URL: https://issues.apache.org/jira/browse/HIVE-26473
> Project: Hive
>  Issue Type: Improvement
>Reporter: dingwei2019
>Priority: Major
>
> we know that jdk11 is a LTS version, but the technical support will be end in 
> September 2023. JDK17 is the next generation LTS version, and will support a 
> least to 2026. 
> for G1GC, Java17 will get 8.66% faster than  Java11, for ParallelGC, the 
> percent will be 6.54%. If we upgrade to java17, we will get more performance 
> improvementthan Java11.
>  
> I suggest, we upgrade hive version to support java17.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HIVE-27555) Upgrade issues with Kudu table on backend db

2023-11-08 Thread Zhihua Deng (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihua Deng reassigned HIVE-27555:
--

Assignee: Zhihua Deng

> Upgrade issues with Kudu table on backend db
> 
>
> Key: HIVE-27555
> URL: https://issues.apache.org/jira/browse/HIVE-27555
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
>
> In HIVE-27457, we try to update the serde lib, (input/output)format of the 
> kudu table in back db. In the upgrade scripts, we join the  "SDS"."SD_ID" 
> with "TABLE_PARAMS"."TBL_ID", 
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/sql/mysql/upgrade-4.0.0-alpha-2-to-4.0.0-beta-1.mysql.sql#L37-L39
> as "SD_ID" is the primary key of SDS, and "TBL_ID" is the primary key of 
> TBLS, we can't join the two tables using these two columns.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27863) Explore ways to inspect the inner fields of the json property

2023-11-08 Thread Zhihua Deng (Jira) 
Zhihua Deng created HIVE-27863:
--

 Summary: Explore ways to inspect the inner fields of the json 
property
 Key: HIVE-27863
 URL: https://issues.apache.org/jira/browse/HIVE-27863
 Project: Hive
  Issue Type: Improvement
Reporter: Zhihua Deng






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27862) Map propertyContent to a wrong column in package.jdo

2023-11-08 Thread Zhihua Deng (Jira) 
Zhihua Deng created HIVE-27862:
--

 Summary: Map propertyContent to a wrong column in package.jdo
 Key: HIVE-27862
 URL: https://issues.apache.org/jira/browse/HIVE-27862
 Project: Hive
  Issue Type: Bug
Reporter: Zhihua Deng






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27861) Property schema is lost after HMS restart

2023-11-08 Thread Zhihua Deng (Jira) 
Zhihua Deng created HIVE-27861:
--

 Summary: Property schema is lost after HMS restart
 Key: HIVE-27861
 URL: https://issues.apache.org/jira/browse/HIVE-27861
 Project: Hive
  Issue Type: Bug
Reporter: Zhihua Deng






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HIVE-27860) Provide a way for declaring the property in property store

2023-11-08 Thread Zhihua Deng (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihua Deng updated HIVE-27860:
---
Issue Type: Improvement  (was: Bug)

> Provide a way for declaring the property in property store
> --
>
> Key: HIVE-27860
> URL: https://issues.apache.org/jira/browse/HIVE-27860
> Project: Hive
>  Issue Type: Improvement
>  Components: Standalone Metastore
>Reporter: Zhihua Deng
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27860) Provide a way for declaring the property in property store

2023-11-08 Thread Zhihua Deng (Jira) 
Zhihua Deng created HIVE-27860:
--

 Summary: Provide a way for declaring the property in property store
 Key: HIVE-27860
 URL: https://issues.apache.org/jira/browse/HIVE-27860
 Project: Hive
  Issue Type: Bug
  Components: Standalone Metastore
Reporter: Zhihua Deng






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HIVE-27859) Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1

2023-11-08 Thread Yuming Wang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuming Wang updated HIVE-27859:
---
Affects Version/s: 2.3.9
   (was: 2.3.8)

> Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
> 
>
> Key: HIVE-27859
> URL: https://issues.apache.org/jira/browse/HIVE-27859
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.9
>Reporter: Yuming Wang
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HIVE-27859) Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1

2023-11-08 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27859:
--
Labels: pull-request-available  (was: )

> Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
> 
>
> Key: HIVE-27859
> URL: https://issues.apache.org/jira/browse/HIVE-27859
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.8
>Reporter: Yuming Wang
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27859) Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1

2023-11-08 Thread Yuming Wang (Jira) 
Yuming Wang created HIVE-27859:
--

 Summary: Backport HIVE-27817: Disable ssl hostname verification 
for 127.0.0.1
 Key: HIVE-27859
 URL: https://issues.apache.org/jira/browse/HIVE-27859
 Project: Hive
  Issue Type: Improvement
Affects Versions: 2.3.8
Reporter: Yuming Wang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HIVE-27788) Exception in Sort Merge join with Group By + PTF Operator

2023-11-08 Thread Krisztian Kasa (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa reassigned HIVE-27788:
-

Assignee: Krisztian Kasa

> Exception in Sort Merge join with Group By + PTF Operator
> -
>
> Key: HIVE-27788
> URL: https://issues.apache.org/jira/browse/HIVE-27788
> Project: Hive
>  Issue Type: Bug
>  Components: Operators
>Affects Versions: 4.0.0-beta-1
>Reporter: Riju Trivedi
>Assignee: Krisztian Kasa
>Priority: Major
>  Labels: pull-request-available
> Attachments: auto_sortmerge_join_17.q
>
>
> Sort- merge join with Group By + PTF operator leads  to Runtime exception 
> {code:java}
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:313)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:291)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:293)
>   ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:387)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:303)
>   ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:392)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:372)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:316)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:127)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.handleOutputRows(PTFOperator.java:337)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:325)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:139)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:372)
>   ... 18 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:534)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchNextGroup(CommonMergeJoinOperator.java:488)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:390)
>   ... 31 more
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:313)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:522)
>   ... 33 more {code}
> Issue can be reproduced with [^auto_sortmerge_join_17.q]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HIVE-27788) Exception in Sort Merge join with Group By + PTF Operator

2023-11-08 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27788:
--
Labels: pull-request-available  (was: )

> Exception in Sort Merge join with Group By + PTF Operator
> -
>
> Key: HIVE-27788
> URL: https://issues.apache.org/jira/browse/HIVE-27788
> Project: Hive
>  Issue Type: Bug
>  Components: Operators
>Affects Versions: 4.0.0-beta-1
>Reporter: Riju Trivedi
>Priority: Major
>  Labels: pull-request-available
> Attachments: auto_sortmerge_join_17.q
>
>
> Sort- merge join with Group By + PTF operator leads  to Runtime exception 
> {code:java}
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:313)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:291)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:293)
>   ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:387)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:303)
>   ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:392)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:372)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:316)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:127)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.handleOutputRows(PTFOperator.java:337)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:325)
>   at 
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:139)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:372)
>   ... 18 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:534)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchNextGroup(CommonMergeJoinOperator.java:488)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinOneGroup(CommonMergeJoinOperator.java:390)
>   ... 31 more
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite 
> nextKeyWritables[1]
>   at 
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:313)
>   at 
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:522)
>   ... 33 more {code}
> Issue can be reproduced with [^auto_sortmerge_join_17.q]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HIVE-27593) Iceberg: Keep iceberg properties in sync with hms properties

2023-11-08 Thread Denys Kuzmenko (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-27593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko resolved HIVE-27593.
---
Fix Version/s: 4.0.0
   Resolution: Fixed

> Iceberg: Keep iceberg properties in sync with hms properties
> 
>
> Key: HIVE-27593
> URL: https://issues.apache.org/jira/browse/HIVE-27593
> Project: Hive
>  Issue Type: Improvement
>  Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
>
> In HIVE-26596, as we have not implement all COW mode, we enforced *mor* mode 
> for iceberg v2 table in the tow scenarios:
>  # create a v2 iceberg table, the delete mode will be set *mor* if not 
> specified
>  # upgrage v1 table to v2, and the delete mode will be set mor
>  
> In HS2, we check the mode(cow/mor) from hms table properties instead of 
> *iceberg* {*}properties(metadata json file){*}, and HIVE-26596 only change 
> hms table properties.  Therefore, it is ok for HS2 to operate iceberg table 
> by checking cow/mor mode from hms properties, but for others like Spark, they 
> operate the iceberg table by checking cow/mor from {*}iceberg 
> properties(metadata json file){*}.
> Before we implement all COW mode, we need keep iceberg properties in sync 
> with hms properties to  make the users have the same experience on multiple 
> engines(HS2 & Spark).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HIVE-27593) Iceberg: Keep iceberg properties in sync with hms properties

2023-11-08 Thread Denys Kuzmenko (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-27593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783952#comment-17783952
 ] 

Denys Kuzmenko commented on HIVE-27593:
---

Merged to master.
Thanks for the patch [~zhangbutao]!

> Iceberg: Keep iceberg properties in sync with hms properties
> 
>
> Key: HIVE-27593
> URL: https://issues.apache.org/jira/browse/HIVE-27593
> Project: Hive
>  Issue Type: Improvement
>  Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
>  Labels: pull-request-available
>
> In HIVE-26596, as we have not implement all COW mode, we enforced *mor* mode 
> for iceberg v2 table in the tow scenarios:
>  # create a v2 iceberg table, the delete mode will be set *mor* if not 
> specified
>  # upgrage v1 table to v2, and the delete mode will be set mor
>  
> In HS2, we check the mode(cow/mor) from hms table properties instead of 
> *iceberg* {*}properties(metadata json file){*}, and HIVE-26596 only change 
> hms table properties.  Therefore, it is ok for HS2 to operate iceberg table 
> by checking cow/mor mode from hms properties, but for others like Spark, they 
> operate the iceberg table by checking cow/mor from {*}iceberg 
> properties(metadata json file){*}.
> Before we implement all COW mode, we need keep iceberg properties in sync 
> with hms properties to  make the users have the same experience on multiple 
> engines(HS2 & Spark).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)