[jira] [Updated] (HIVE-27821) Backport of HIVE-21032, HIVE-20590, HIVE-20409, HIVE-20919
[ https://issues.apache.org/jira/browse/HIVE-27821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27821: -- Labels: pull-request-available (was: ) > Backport of HIVE-21032, HIVE-20590, HIVE-20409, HIVE-20919 > -- > > Key: HIVE-27821 > URL: https://issues.apache.org/jira/browse/HIVE-27821 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27821) Backport of HIVE-21032, HIVE-20590, HIVE-20409, HIVE-20919
Aman Raj created HIVE-27821: --- Summary: Backport of HIVE-21032, HIVE-20590, HIVE-20409, HIVE-20919 Key: HIVE-27821 URL: https://issues.apache.org/jira/browse/HIVE-27821 Project: Hive Issue Type: Sub-task Affects Versions: 3.2.0 Reporter: Aman Raj Assignee: Aman Raj -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27820) Backport of HIVE-20430, HIVE-20613, HIVE-20615
Aman Raj created HIVE-27820: --- Summary: Backport of HIVE-20430, HIVE-20613, HIVE-20615 Key: HIVE-27820 URL: https://issues.apache.org/jira/browse/HIVE-27820 Project: Hive Issue Type: Sub-task Reporter: Aman Raj Assignee: Aman Raj -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27776) Iceberg: Upgrade iceberg version to 1.4.0
[ https://issues.apache.org/jira/browse/HIVE-27776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao resolved HIVE-27776. --- Resolution: Duplicate > Iceberg: Upgrade iceberg version to 1.4.0 > - > > Key: HIVE-27776 > URL: https://issues.apache.org/jira/browse/HIVE-27776 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Priority: Major > > [https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-core/1.4.0] > Iceberg1.4.0 has been released out, and we need upgrade iceberg depdency from > 1.3.1 to 1.4.0. Meantime, we should port some Hive catalog changes from > Iceberg repo to Hive repo. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27776) Iceberg: Upgrade iceberg version to 1.4.0
[ https://issues.apache.org/jira/browse/HIVE-27776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779010#comment-17779010 ] zhangbutao commented on HIVE-27776: --- Supersede by https://issues.apache.org/jira/browse/HIVE-27819 > Iceberg: Upgrade iceberg version to 1.4.0 > - > > Key: HIVE-27776 > URL: https://issues.apache.org/jira/browse/HIVE-27776 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Priority: Major > > [https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-core/1.4.0] > Iceberg1.4.0 has been released out, and we need upgrade iceberg depdency from > 1.3.1 to 1.4.0. Meantime, we should port some Hive catalog changes from > Iceberg repo to Hive repo. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.1
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27819: -- Labels: pull-request-available (was: ) > Iceberg: Upgrade iceberg version to 1.4.1 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Iceberg latest version 1.4.1 has been released out. we need upgrade iceberg > depdency from 1.3.1 to 1.4.1. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#141-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.1
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao reassigned HIVE-27819: - Assignee: zhangbutao > Iceberg: Upgrade iceberg version to 1.4.1 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > > Iceberg latest version 1.4.1 has been released out. we need upgrade iceberg > depdency from 1.3.1 to 1.4.1. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#141-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.1
zhangbutao created HIVE-27819: - Summary: Iceberg: Upgrade iceberg version to 1.4.1 Key: HIVE-27819 URL: https://issues.apache.org/jira/browse/HIVE-27819 Project: Hive Issue Type: Improvement Components: Iceberg integration Reporter: zhangbutao Iceberg latest version 1.4.1 has been released out. we need upgrade iceberg depdency from 1.3.1 to 1.4.1. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#141-release] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27734) Add Iceberg's storage-partitioned join capabilities to Hive's [sorted-]bucket-map-join
[
https://issues.apache.org/jira/browse/HIVE-27734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Kovacs updated HIVE-27734:
Summary: Add Iceberg's storage-partitioned join capabilities to Hive's
[sorted-]bucket-map-join (was: Add Icenerg's storage-partitioned join
capabilities to Hive's [sorted-]bucket-map-join)
> Add Iceberg's storage-partitioned join capabilities to Hive's
> [sorted-]bucket-map-join
> --
>
> Key: HIVE-27734
> URL: https://issues.apache.org/jira/browse/HIVE-27734
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Affects Versions: 4.0.0-alpha-2
>Reporter: Janos Kovacs
>Priority: Major
>
> Iceberg's 'data bucketing' is implemented through its rich (function based)
> partitioning feature which helps to optimize join operations - called storage
> partitioned joins.
> doc:
> [https://docs.google.com/document/d/1foTkDSM91VxKgkEcBMsuAvEjNybjja-uHk-r3vtXWFE/edit#heading=h.82w8qxfl2uwl]
> spark impl.: https://issues.apache.org/jira/browse/SPARK-37375
> This feature is not yet leveraged in Hive into its bucket-map-join
> optimization, neither alone nor with Iceberg's SortOrder to
> sorted-bucket-map-join.
> Customers migrating from Hive table format to Iceberg format with storage
> optimized schema will experience performance degradation on large tables
> where Iceberg's gain on no-listing performance improvement is significantly
> smaller than the actual join performance over bucket-join or even
> sorted-bucket-join.
>
> {noformat}
> SET hive.query.results.cache.enabled=false;
> SET hive.fetch.task.conversion = none;
> SET hive.optimize.bucketmapjoin=true;
> SET hive.convert.join.bucket.mapjoin.tez=true;
> SET hive.auto.convert.join.noconditionaltask.size=1000;
> --if you are working with external table, you need this for bmj:
> SET hive.disable.unsafe.external.table.operations=false;
> -- HIVE BUCKET-MAP-JOIN
> DROP TABLE IF EXISTS default.hivebmjt1 PURGE;
> DROP TABLE IF EXISTS default.hivebmjt2 PURGE;
> CREATE TABLE default.hivebmjt1 (id int, txt string) CLUSTERED BY (id) INTO 8
> BUCKETS;
> CREATE TABLE default.hivebmjt2 (id int, txt string);
> INSERT INTO default.hivebmjt1 VALUES
> (1,'1'),(2,'2'),(3,'3'),(4,'4'),(5,'5'),(6,'6'),(7,'7'),(8,'8');
> INSERT INTO default.hivebmjt2 VALUES (1,'1'),(2,'2'),(3,'3'),(4,'4');
> EXPLAIN
> SELECT * FROM default.hivebmjt1 f INNER JOIN default.hivebmjt2 d ON f.id
> = d.id;
> EXPLAIN
> SELECT * FROM default.hivebmjt1 f LEFT OUTER JOIN default.hivebmjt2 d ON f.id
> = d.id;
> -- Both are optimized into BMJ
> -- ICEBERG BUCKET-MAP-JOIN via Iceberg's storage-partitioned join
> DROP TABLE IF EXISTS default.icespbmjt1 PURGE;
> DROP TABLE IF EXISTS default.icespbmjt2 PURGE;
> CREATE TABLE default.icespbmjt1 (txt string) PARTITIONED BY (id int) STORED
> BY ICEBERG ;
> CREATE TABLE default.icespbmjt2 (txt string) PARTITIONED BY (id int) STORED
> BY ICEBERG ;
> INSERT INTO default.icespbmjt1 VALUES ('1',1),('2',2),('3',3),('4',4);
> INSERT INTO default.icespbmjt2 VALUES ('1',1),('2',2),('3',3),('4',4);
> EXPLAIN
> SELECT * FROM default.icespbmjt1 f INNER JOIN default.icespbmjt2 d ON
> f.id = d.id;
> EXPLAIN
> SELECT * FROM default.icespbmjt1 f LEFT OUTER JOIN default.icespbmjt2 d ON
> f.id = d.id;
> -- Only Map-Join optimised
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27818) Fix compilation failure in AccumuloPredicateHandler
[
https://issues.apache.org/jira/browse/HIVE-27818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated HIVE-27818:
---
Description:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hive-accumulo-handler: Compilation failure
[ERROR]
/Users/yumwang/opensource/hive/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java:[263,23]
unreported exception org.apache.hadoop.hive.ql.metadata.HiveException; must be
caught or declared to be thrown
[ERROR]
{noformat}
{noformat}
yumwang@G9L07H60PK hive % java -version
openjdk version "1.8.0_382"
OpenJDK Runtime Environment (Zulu 8.72.0.17-CA-macos-aarch64) (build
1.8.0_382-b05)
OpenJDK 64-Bit Server VM (Zulu 8.72.0.17-CA-macos-aarch64) (build 25.382-b05,
mixed mode)
yumwang@G9L07H60PK hive % mvn -version
Apache Maven 3.9.4 (dfbb324ad4a7c8fb0bf182e6d91b0ae20e3d2dd9)
Maven home: /Users/yumwang/software/apache-maven-3.8.8
Java version: 1.8.0_382, vendor: Azul Systems, Inc., runtime:
/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
Default locale: en_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.6", arch: "aarch64", family: "mac"
{noformat}
was:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hive-accumulo-handler: Compilation failure
[ERROR]
/Users/yumwang/opensource/hive/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java:[263,23]
unreported exception org.apache.hadoop.hive.ql.metadata.HiveException; must be
caught or declared to be thrown
[ERROR]
{noformat}
> Fix compilation failure in AccumuloPredicateHandler
> ---
>
> Key: HIVE-27818
> URL: https://issues.apache.org/jira/browse/HIVE-27818
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Priority: Major
>
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hive-accumulo-handler: Compilation failure
> [ERROR]
> /Users/yumwang/opensource/hive/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java:[263,23]
> unreported exception org.apache.hadoop.hive.ql.metadata.HiveException; must
> be caught or declared to be thrown
> [ERROR]
> {noformat}
> {noformat}
> yumwang@G9L07H60PK hive % java -version
> openjdk version "1.8.0_382"
> OpenJDK Runtime Environment (Zulu 8.72.0.17-CA-macos-aarch64) (build
> 1.8.0_382-b05)
> OpenJDK 64-Bit Server VM (Zulu 8.72.0.17-CA-macos-aarch64) (build 25.382-b05,
> mixed mode)
> yumwang@G9L07H60PK hive % mvn -version
> Apache Maven 3.9.4 (dfbb324ad4a7c8fb0bf182e6d91b0ae20e3d2dd9)
> Maven home: /Users/yumwang/software/apache-maven-3.8.8
> Java version: 1.8.0_382, vendor: Azul Systems, Inc., runtime:
> /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
> Default locale: en_CN, platform encoding: UTF-8
> OS name: "mac os x", version: "13.6", arch: "aarch64", family: "mac"
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27818) Fix compilation failure in AccumuloPredicateHandler
Yuming Wang created HIVE-27818:
--
Summary: Fix compilation failure in AccumuloPredicateHandler
Key: HIVE-27818
URL: https://issues.apache.org/jira/browse/HIVE-27818
Project: Hive
Issue Type: Bug
Reporter: Yuming Wang
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hive-accumulo-handler: Compilation failure
[ERROR]
/Users/yumwang/opensource/hive/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/predicate/AccumuloPredicateHandler.java:[263,23]
unreported exception org.apache.hadoop.hive.ql.metadata.HiveException; must be
caught or declared to be thrown
[ERROR]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27807) Backport of HIVE-20629, HIVE-20705, HIVE-20734
[ https://issues.apache.org/jira/browse/HIVE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27807: -- Labels: pull-request-available (was: ) > Backport of HIVE-20629, HIVE-20705, HIVE-20734 > -- > > Key: HIVE-27807 > URL: https://issues.apache.org/jira/browse/HIVE-27807 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > > Backports - HIVE-20629, HIVE-20705, HIVE-20734 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27807) Backport of HIVE-20629, HIVE-20705, HIVE-20734
[ https://issues.apache.org/jira/browse/HIVE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-27807: Affects Version/s: 3.1.3 (was: 3.2.0) > Backport of HIVE-20629, HIVE-20705, HIVE-20734 > -- > > Key: HIVE-27807 > URL: https://issues.apache.org/jira/browse/HIVE-27807 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > > Backports - HIVE-20629, HIVE-20705, HIVE-20734 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27807) Backport of HIVE-20629, HIVE-20705, HIVE-20734
[ https://issues.apache.org/jira/browse/HIVE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-27807. - Fix Version/s: 3.2.0 Resolution: Fixed > Backport of HIVE-20629, HIVE-20705, HIVE-20734 > -- > > Key: HIVE-27807 > URL: https://issues.apache.org/jira/browse/HIVE-27807 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > > Backports - HIVE-20629, HIVE-20705, HIVE-20734 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27807) Backport of HIVE-20629, HIVE-20705, HIVE-20734
[ https://issues.apache.org/jira/browse/HIVE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-27807: Description: Backports - HIVE-20629, HIVE-20705, HIVE-20734 > Backport of HIVE-20629, HIVE-20705, HIVE-20734 > -- > > Key: HIVE-27807 > URL: https://issues.apache.org/jira/browse/HIVE-27807 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > > Backports - HIVE-20629, HIVE-20705, HIVE-20734 -- This message was sent by Atlassian Jira (v8.20.10#820010)
