[jira] [Created] (HUDI-5583) multiple writers dead lock while cleaning

2023-01-18 Thread Jian Feng (Jira)
Jian Feng created HUDI-5583: --- Summary: multiple writers dead lock while cleaning Key: HUDI-5583 URL: https://issues.apache.org/jira/browse/HUDI-5583 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Let take a look at [Consistent Hash Index RFC: Bucket Resizing|

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: *Bucket Resizing (Splitting & Merging)* Considering there is a semantic similarity between

[jira] [Updated] (HUDI-5560) Make Consistent hash index Bucket Resizing more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Summary: Make Consistent hash index Bucket Resizing more available on real cases (was: Make Consistent

[jira] [Updated] (HUDI-5560) Make Consistent hash index more available on real cases

2023-01-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5560: Description: Bucket Resizing (Splitting & Merging) Considering there is a semantic similarity between

[jira] [Created] (HUDI-5560) Make Consistent hash index more available on real cases

2023-01-15 Thread Jian Feng (Jira)
Jian Feng created HUDI-5560: --- Summary: Make Consistent hash index more available on real cases Key: HUDI-5560 URL: https://issues.apache.org/jira/browse/HUDI-5560 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-5509) check if dfs support atomic creation when using filesystem base lock provider

2023-01-05 Thread Jian Feng (Jira)
Jian Feng created HUDI-5509: --- Summary: check if dfs support atomic creation when using filesystem base lock provider Key: HUDI-5509 URL: https://issues.apache.org/jira/browse/HUDI-5509 Project: Apache Hudi

[jira] [Created] (HUDI-5469) Hive doesn't respect the space at the end of partition path, so remove it to avoid duplicate key error

2022-12-22 Thread Jian Feng (Jira)
Jian Feng created HUDI-5469: --- Summary: Hive doesn't respect the space at the end of partition path, so remove it to avoid duplicate key error Key: HUDI-5469 URL: https://issues.apache.org/jira/browse/HUDI-5469

[jira] [Closed] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-17 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-5222. --- Resolution: Fixed > Flink job may read wrong timline service address when enable offline > compaction >

[jira] [Commented] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-17 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635597#comment-17635597 ] Jian Feng commented on HUDI-5222: - should be fixed in hudi-5228 > Flink job may read wrong timline

[jira] [Updated] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5222: Description: For now, Flink job is forced to use embed timeline service, and will create new

[jira] [Updated] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5222: Description: For now, Flink job is forced to use embed timeline service, and will create new

[jira] [Updated] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-15 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-5222: Description: For now, Flink job is forced to use embed timeline service, and will create new

[jira] [Created] (HUDI-5222) Flink job may read wrong timline service address when enable offline compaction

2022-11-15 Thread Jian Feng (Jira)
Jian Feng created HUDI-5222: --- Summary: Flink job may read wrong timline service address when enable offline compaction Key: HUDI-5222 URL: https://issues.apache.org/jira/browse/HUDI-5222 Project: Apache

[jira] [Commented] (HUDI-3304) Support partial update payload

2022-11-07 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629911#comment-17629911 ] Jian Feng commented on HUDI-3304: - [~shenshengli] I mean you can update data to null by specifying

[jira] [Commented] (HUDI-3304) Support partial update payload

2022-11-04 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1762#comment-1762 ] Jian Feng commented on HUDI-3304: - [~shenshengli] This can be achieved by

[jira] [Created] (HUDI-5146) support payload precombine between base file and log

2022-11-01 Thread Jian Feng (Jira)
Jian Feng created HUDI-5146: --- Summary: support payload precombine between base file and log Key: HUDI-5146 URL: https://issues.apache.org/jira/browse/HUDI-5146 Project: Apache Hudi Issue Type: Bug

[jira] [Assigned] (HUDI-4964) Refactor HoodieWriteConfig to make sure we have a uniform and better way to access config value

2022-10-03 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4964: --- Assignee: Jian Feng > Refactor HoodieWriteConfig to make sure we have a uniform and better way to >

[jira] [Updated] (HUDI-4964) Refactor HoodieWriteConfig to make sure we have a uniform and better way to access config value

2022-10-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4964: Description: I feel like this class is a hodgepodge and is ballooning. It does not demonstrate the

[jira] [Updated] (HUDI-4964) Refactor HoodieWriteConfig to make sure we have a uniform and better way to access config value

2022-10-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4964: Description: I feel like this class is a hodgepodge and is ballooning. It does not demonstrate the

[jira] [Created] (HUDI-4964) Refactor HoodieWriteConfig to make sure we have a uniform and better way to access config value

2022-10-01 Thread Jian Feng (Jira)
Jian Feng created HUDI-4964: --- Summary: Refactor HoodieWriteConfig to make sure we have a uniform and better way to access config value Key: HUDI-4964 URL: https://issues.apache.org/jira/browse/HUDI-4964

[jira] [Created] (HUDI-4925) Should Force to use ExpressionPayload in MergeIntoTableCommand

2022-09-26 Thread Jian Feng (Jira)
Jian Feng created HUDI-4925: --- Summary: Should Force to use ExpressionPayload in MergeIntoTableCommand Key: HUDI-4925 URL: https://issues.apache.org/jira/browse/HUDI-4925 Project: Apache Hudi

[jira] [Created] (HUDI-4920) fix PartialUpdatePayload cannot return deleted record in preCombine function issue

2022-09-26 Thread Jian Feng (Jira)
Jian Feng created HUDI-4920: --- Summary: fix PartialUpdatePayload cannot return deleted record in preCombine function issue Key: HUDI-4920 URL: https://issues.apache.org/jira/browse/HUDI-4920 Project: Apache

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: This feature aims to improve PartialUpdatePayload to handle multiple sources properly Let's

[jira] [Assigned] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4882: --- Assignee: Jian Feng > Multiple ordering fields for partial update to handle out-of-order events >

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: Let's give you some background about why we need multiple ordering fields For example, we have

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: Let's give you some background about why we need multiple ordering fields For example, we have

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: Let's give you some background about why we need multiple ordering fields For example, we have

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: For example, we have 2 sources, one target table * source1's fields: *id, ts, name* *

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: we have 2 sources, one target table * source1's fields: *id, ts, name* * source2's fields:*

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update to handle out-of-order events

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Summary: Multiple ordering fields for partial update to handle out-of-order events (was: Multiple ordering

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: let's say Stream or Table A * case 1 * Current data: * id ts1 nameprice

[jira] [Updated] (HUDI-4882) Multiple ordering fields for partial update

2022-09-20 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4882: Description: let's say Stream or Table A This feature will allow users to perform partial updates across

[jira] [Created] (HUDI-4882) Multiple ordering fields for partial update

2022-09-20 Thread Jian Feng (Jira)
Jian Feng created HUDI-4882: --- Summary: Multiple ordering fields for partial update Key: HUDI-4882 URL: https://issues.apache.org/jira/browse/HUDI-4882 Project: Apache Hudi Issue Type: New Feature

[jira] [Closed] (HUDI-4538) FIx hive sync lose parititons issue

2022-09-18 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-4538. --- Resolution: Duplicate > FIx hive sync lose parititons issue > --- > >

[jira] [Assigned] (HUDI-4730) FIX Batch job cannot clean old commits files in clean Function

2022-08-29 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4730: --- Assignee: Jian Feng > FIX Batch job cannot clean old commits files in clean Function >

[jira] [Created] (HUDI-4730) FIX Batch job cannot clean old commits files in clean Function

2022-08-26 Thread Jian Feng (Jira)
Jian Feng created HUDI-4730: --- Summary: FIX Batch job cannot clean old commits files in clean Function Key: HUDI-4730 URL: https://issues.apache.org/jira/browse/HUDI-4730 Project: Apache Hudi

[jira] [Assigned] (HUDI-4066) HiveMetastoreBasedLockProvider can not release lock when writer fails

2022-08-23 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4066: --- Assignee: Jian Feng > HiveMetastoreBasedLockProvider can not release lock when writer fails >

[jira] [Assigned] (HUDI-4677) Snapshot view management

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4677: --- Assignee: Jian Feng > Snapshot view management > > > Key:

[jira] [Resolved] (HUDI-4676) infer cleaner policy when write concurrency mode is OCC

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4676. - > infer cleaner policy when write concurrency mode is OCC >

[jira] [Assigned] (HUDI-4676) infer cleaner policy when write concurrency mode is OCC

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4676: --- Assignee: Jian Feng > infer cleaner policy when write concurrency mode is OCC >

[jira] [Assigned] (HUDI-4681) savepoint lifycycle management

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4681: --- Assignee: Jian Feng > savepoint lifycycle management > -- > >

[jira] [Assigned] (HUDI-4678) initial RFC

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4678: --- Assignee: Jian Feng > initial RFC > --- > > Key: HUDI-4678 >

[jira] [Assigned] (HUDI-4679) release snapshot view

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4679: --- Assignee: Jian Feng > release snapshot view > - > > Key:

[jira] [Updated] (HUDI-4680) support snapshot view on query side(Presto/Trino)

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4680: Summary: support snapshot view on query side(Presto/Trino) (was: support snapshot view on query

[jira] [Created] (HUDI-4682) mor table's snapshot view creation

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4682: --- Summary: mor table's snapshot view creation Key: HUDI-4682 URL: https://issues.apache.org/jira/browse/HUDI-4682 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-4681) savepoint lifycycle management

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4681: --- Summary: savepoint lifycycle management Key: HUDI-4681 URL: https://issues.apache.org/jira/browse/HUDI-4681 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-4680) support snapshot view on query side(Spark/Presto/Trino)

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4680: --- Summary: support snapshot view on query side(Spark/Presto/Trino) Key: HUDI-4680 URL: https://issues.apache.org/jira/browse/HUDI-4680 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-4679) release snapshot view

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4679: --- Summary: release snapshot view Key: HUDI-4679 URL: https://issues.apache.org/jira/browse/HUDI-4679 Project: Apache Hudi Issue Type: New Feature Reporter:

[jira] [Created] (HUDI-4678) initial RFC

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4678: --- Summary: initial RFC Key: HUDI-4678 URL: https://issues.apache.org/jira/browse/HUDI-4678 Project: Apache Hudi Issue Type: New Feature Reporter: Jian Feng

[jira] [Updated] (HUDI-4677) Snapshot view management

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4677: Description: !image-2022-08-22-02-03-31-588.png! for the snapshot view scenario, Hudi already

[jira] [Created] (HUDI-4677) Snapshot view management

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4677: --- Summary: Snapshot view management Key: HUDI-4677 URL: https://issues.apache.org/jira/browse/HUDI-4677 Project: Apache Hudi Issue Type: Epic Reporter: Jian

[jira] [Updated] (HUDI-4676) infer cleaner policy when write concurrency mode is OCC

2022-08-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4676: Summary: infer cleaner policy when write concurrency mode is OCC (was: infer FAILED_WRITES_CLEANER_POLICY

[jira] [Created] (HUDI-4676) infer FAILED_WRITES_CLEANER_POLICY when using Multiple writers

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4676: --- Summary: infer FAILED_WRITES_CLEANER_POLICY when using Multiple writers Key: HUDI-4676 URL: https://issues.apache.org/jira/browse/HUDI-4676 Project: Apache Hudi

[jira] [Created] (HUDI-4675) add unittest for RebalancedSparkHoodieHBaseIndex

2022-08-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4675: --- Summary: add unittest for RebalancedSparkHoodieHBaseIndex Key: HUDI-4675 URL: https://issues.apache.org/jira/browse/HUDI-4675 Project: Apache Hudi Issue Type: Test

[jira] [Assigned] (HUDI-4424) add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

2022-08-09 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4424: --- Assignee: Jian Feng > add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST >

[jira] [Resolved] (HUDI-4424) add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

2022-08-09 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4424. - > add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST >

[jira] [Closed] (HUDI-4402) Incremental query should filter data out by "hoodie.datasource.read.begin.instanttime"

2022-08-08 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-4402. --- Resolution: Won't Fix > Incremental query should filter data out by >

[jira] [Resolved] (HUDI-4516) fix Task not serializable error when run HoodieCleaner after one failure

2022-08-08 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4516. - > fix Task not serializable error when run HoodieCleaner after one failure >

[jira] [Resolved] (HUDI-4464) fix CI warning

2022-08-08 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4464. - > fix CI warning > -- > > Key: HUDI-4464 > URL:

[jira] [Assigned] (HUDI-4464) fix CI warning

2022-08-08 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4464: --- Assignee: Jian Feng > fix CI warning > -- > > Key: HUDI-4464 >

[jira] [Assigned] (HUDI-4572) fix 'Not a valid schema field: ts' error in HoodieFlinkCompactor, ifprecombine field is not ts

2022-08-08 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4572: --- Assignee: Jian Feng > fix 'Not a valid schema field: ts' error in HoodieFlinkCompactor, >

[jira] [Created] (HUDI-4572) fix 'Not a valid schema field: ts' error in HoodieFlinkCompactor, ifprecombine field is not ts

2022-08-08 Thread Jian Feng (Jira)
Jian Feng created HUDI-4572: --- Summary: fix 'Not a valid schema field: ts' error in HoodieFlinkCompactor, ifprecombine field is not ts Key: HUDI-4572 URL: https://issues.apache.org/jira/browse/HUDI-4572

[jira] [Created] (HUDI-4571) fix default value of partition_extractor_class may wrong bug if hoodie.datasource.write.partitionpath.field and hoodie.table.partition.fields not aligned

2022-08-08 Thread Jian Feng (Jira)
Jian Feng created HUDI-4571: --- Summary: fix default value of partition_extractor_class may wrong bug if hoodie.datasource.write.partitionpath.field and hoodie.table.partition.fields not aligned Key: HUDI-4571 URL:

[jira] [Closed] (HUDI-4553) fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: ALREADY_ACQUIRED

2022-08-05 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-4553. --- Resolution: Duplicate > fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: >

[jira] [Created] (HUDI-4553) fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: ALREADY_ACQUIRED

2022-08-05 Thread Jian Feng (Jira)
Jian Feng created HUDI-4553: --- Summary: fix HiveMetastoreBasedLockProvider java.lang.IllegalArgumentException: ALREADY_ACQUIRED Key: HUDI-4553 URL: https://issues.apache.org/jira/browse/HUDI-4553 Project:

[jira] [Created] (HUDI-4538) FIx hive sync lose parititons issue

2022-08-04 Thread Jian Feng (Jira)
Jian Feng created HUDI-4538: --- Summary: FIx hive sync lose parititons issue Key: HUDI-4538 URL: https://issues.apache.org/jira/browse/HUDI-4538 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-4516) fix Task not serializable error when run HoodieCleaner after one failure

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4516: Summary: fix Task not serializable error when run HoodieCleaner after one failure (was: fix Task not

[jira] [Assigned] (HUDI-4516) fix Task not serializable error when run HoodieCleaner from one failure

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4516: --- Assignee: Jian Feng > fix Task not serializable error when run HoodieCleaner from one failure >

[jira] [Created] (HUDI-4516) fix Task not serializable error when run HoodieCleaner from one failure

2022-08-01 Thread Jian Feng (Jira)
Jian Feng created HUDI-4516: --- Summary: fix Task not serializable error when run HoodieCleaner from one failure Key: HUDI-4516 URL: https://issues.apache.org/jira/browse/HUDI-4516 Project: Apache Hudi

[jira] [Updated] (HUDI-4424) add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4424: Status: Patch Available (was: In Progress) > add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

[jira] [Updated] (HUDI-4424) add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4424: Status: In Progress (was: Open) > add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST >

[jira] [Updated] (HUDI-4210) Create custom hbase index to solve data skew issue on hbase regions

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4210: Status: In Progress (was: Open) > Create custom hbase index to solve data skew issue on hbase regions >

[jira] [Resolved] (HUDI-4210) Create custom hbase index to solve data skew issue on hbase regions

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4210. - > Create custom hbase index to solve data skew issue on hbase regions >

[jira] [Updated] (HUDI-4210) Create custom hbase index to solve data skew issue on hbase regions

2022-08-01 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4210: Status: Open (was: In Progress) > Create custom hbase index to solve data skew issue on hbase regions >

[jira] [Created] (HUDI-4464) fix CI warning

2022-07-25 Thread Jian Feng (Jira)
Jian Feng created HUDI-4464: --- Summary: fix CI warning Key: HUDI-4464 URL: https://issues.apache.org/jira/browse/HUDI-4464 Project: Apache Hudi Issue Type: Improvement Reporter: Jian

[jira] [Assigned] (HUDI-4437) resolve conflicts between TestHiveSyncGlobalCommitTool and TestHiveSyncTool

2022-07-21 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4437: --- Assignee: Jian Feng > resolve conflicts between TestHiveSyncGlobalCommitTool and TestHiveSyncTool >

[jira] [Created] (HUDI-4437) resolve conflicts between TestHiveSyncGlobalCommitTool and TestHiveSyncTool

2022-07-21 Thread Jian Feng (Jira)
Jian Feng created HUDI-4437: --- Summary: resolve conflicts between TestHiveSyncGlobalCommitTool and TestHiveSyncTool Key: HUDI-4437 URL: https://issues.apache.org/jira/browse/HUDI-4437 Project: Apache Hudi

[jira] [Resolved] (HUDI-4065) Implement a self-releasing FileLock for multiple writers scenario

2022-07-19 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4065. - > Implement a self-releasing FileLock for multiple writers scenario >

[jira] [Closed] (HUDI-2414) enable Hot and cold data separate when ingest data

2022-07-19 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng closed HUDI-2414. --- Resolution: Won't Fix > enable Hot and cold data separate when ingest data >

[jira] [Created] (HUDI-4424) add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST

2022-07-19 Thread Jian Feng (Jira)
Jian Feng created HUDI-4424: --- Summary: add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQUEST Key: HUDI-4424 URL: https://issues.apache.org/jira/browse/HUDI-4424 Project: Apache Hudi Issue

[jira] [Assigned] (HUDI-4413) Make the return value of LockProvider's tryLock method more reasonable

2022-07-18 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4413: --- Assignee: Jian Feng > Make the return value of LockProvider's tryLock method more reasonable >

[jira] [Updated] (HUDI-4413) Make the return value of LockProvider's tryLock method more reasonable

2022-07-18 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng updated HUDI-4413: Description: For now, zk lock provider and hive lock provider never return false when tryLock, hence it

[jira] [Created] (HUDI-4413) Make the return value of LockProvider's tryLock method more reasonable

2022-07-18 Thread Jian Feng (Jira)
Jian Feng created HUDI-4413: --- Summary: Make the return value of LockProvider's tryLock method more reasonable Key: HUDI-4413 URL: https://issues.apache.org/jira/browse/HUDI-4413 Project: Apache Hudi

  1   2   >