[jira] [Resolved] (HIVE-28102) Invoke validateDataFilesExist for RowDelta operations
[ https://issues.apache.org/jira/browse/HIVE-28102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-28102. - Fix Version/s: 4.1.0 Resolution: Fixed > Invoke validateDataFilesExist for RowDelta operations > - > > Key: HIVE-28102 > URL: https://issues.apache.org/jira/browse/HIVE-28102 > Project: Hive > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Ayush Saxena >Priority: Major > Labels: iceberg, pull-request-available > Fix For: 4.1.0 > > > Hive must invoke validateDataFilesExist for RowDelta operations > (DELETE/UPDATE/MERGE). > Without this a concurrent RewriteFiles (compaction) and RowDelta can corrupt > a table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28102) Iceberg: Invoke validateDataFilesExist for RowDelta operations
[ https://issues.apache.org/jira/browse/HIVE-28102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HIVE-28102: Summary: Iceberg: Invoke validateDataFilesExist for RowDelta operations (was: Invoke validateDataFilesExist for RowDelta operations) > Iceberg: Invoke validateDataFilesExist for RowDelta operations > -- > > Key: HIVE-28102 > URL: https://issues.apache.org/jira/browse/HIVE-28102 > Project: Hive > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Ayush Saxena >Priority: Major > Labels: iceberg, pull-request-available > Fix For: 4.1.0 > > > Hive must invoke validateDataFilesExist for RowDelta operations > (DELETE/UPDATE/MERGE). > Without this a concurrent RewriteFiles (compaction) and RowDelta can corrupt > a table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28097) Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, HIVE-24113, HIVE-25458, HI
[ https://issues.apache.org/jira/browse/HIVE-28097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28097: -- Labels: pull-request-available (was: ) > Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, > HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, > HIVE-24113, HIVE-25458, HIVE-25577, HIVE-25403 > -- > > Key: HIVE-28097 > URL: https://issues.apache.org/jira/browse/HIVE-28097 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28097) Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, HIVE-24113, HIVE-25458, H
[ https://issues.apache.org/jira/browse/HIVE-28097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-28097. - Fix Version/s: 3.2.0 Resolution: Fixed > Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, > HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, > HIVE-24113, HIVE-25458, HIVE-25577, HIVE-25403 > -- > > Key: HIVE-28097 > URL: https://issues.apache.org/jira/browse/HIVE-28097 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28098) Fails to copy empty column statistics of materialized CTE
[
https://issues.apache.org/jira/browse/HIVE-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-28098:
--
Status: Patch Available (was: Open)
I created a pull request.
https://github.com/apache/hive/pull/5106
> Fails to copy empty column statistics of materialized CTE
> -
>
> Key: HIVE-28098
> URL: https://issues.apache.org/jira/browse/HIVE-28098
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
>
> HIVE-28080 introduced the optimization of materialized CTEs, but it turned
> out that it failed when statistics were empty.
> This query reproduces the issue.
> {code:java}
> set hive.stats.autogather=false;
> CREATE TABLE src_no_stats AS SELECT '123' as key, 'val123' as value UNION ALL
> SELECT '9' as key, 'val9' as value;
> set hive.optimize.cte.materialize.threshold=2;
> set hive.optimize.cte.materialize.full.aggregate.only=false;
> EXPLAIN WITH materialized_cte1 AS (
> SELECT * FROM src_no_stats
> ),
> materialized_cte2 AS (
> SELECT a.key
> FROM materialized_cte1 a
> JOIN materialized_cte1 b ON (a.key = b.key)
> )
> SELECT a.key
> FROM materialized_cte2 a
> JOIN materialized_cte2 b ON (a.key = b.key); {code}
> It throws an error.
> {code:java}
> Error: Error while compiling statement: FAILED: IllegalStateException The
> size of col stats must be equal to that of schema. Stats = [], Schema = [key]
> (state=42000,code=4) {code}
> Attaching a debugger, FSO of materialized_cte2 has empty stats as
> JoinOperator loses stats.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28098) Fails to copy empty column statistics of materialized CTE
[
https://issues.apache.org/jira/browse/HIVE-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28098:
--
Labels: pull-request-available (was: )
> Fails to copy empty column statistics of materialized CTE
> -
>
> Key: HIVE-28098
> URL: https://issues.apache.org/jira/browse/HIVE-28098
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
>
> HIVE-28080 introduced the optimization of materialized CTEs, but it turned
> out that it failed when statistics were empty.
> This query reproduces the issue.
> {code:java}
> set hive.stats.autogather=false;
> CREATE TABLE src_no_stats AS SELECT '123' as key, 'val123' as value UNION ALL
> SELECT '9' as key, 'val9' as value;
> set hive.optimize.cte.materialize.threshold=2;
> set hive.optimize.cte.materialize.full.aggregate.only=false;
> EXPLAIN WITH materialized_cte1 AS (
> SELECT * FROM src_no_stats
> ),
> materialized_cte2 AS (
> SELECT a.key
> FROM materialized_cte1 a
> JOIN materialized_cte1 b ON (a.key = b.key)
> )
> SELECT a.key
> FROM materialized_cte2 a
> JOIN materialized_cte2 b ON (a.key = b.key); {code}
> It throws an error.
> {code:java}
> Error: Error while compiling statement: FAILED: IllegalStateException The
> size of col stats must be equal to that of schema. Stats = [], Schema = [key]
> (state=42000,code=4) {code}
> Attaching a debugger, FSO of materialized_cte2 has empty stats as
> JoinOperator loses stats.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28097) Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, HIVE-24113, HIVE-25458, HI
[ https://issues.apache.org/jira/browse/HIVE-28097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj updated HIVE-28097: Summary: Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, HIVE-24113, HIVE-25458, HIVE-25577, HIVE-25403 (was: Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, HIVE-24113, HIVE-25458) > Backport of HIVE-20082, HIVE-19564, HIVE-20174, HIVE-20197, HIVE-20207, > HIVE-20245, HIVE-20294, HIVE-26447, HIVE-20315, HIVE-22170, HIVE-22331, > HIVE-24113, HIVE-25458, HIVE-25577, HIVE-25403 > -- > > Key: HIVE-28097 > URL: https://issues.apache.org/jira/browse/HIVE-28097 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
