[jira] [Resolved] (HUDI-5800) Fix test failure in TestHoodieMergeOnReadTable

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-5800. - > Fix test failure in TestHoodieMergeOnReadTable > -- > >

[jira] [Updated] (HUDI-5800) Fix test failure in TestHoodieMergeOnReadTable

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5800: Status: In Progress (was: Open) > Fix test failure in TestHoodieMergeOnReadTable >

[jira] [Updated] (HUDI-5772) Align Flink clustering configuration with HoodieClusteringConfig

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5772: Fix Version/s: 0.13.1 > Align Flink clustering configuration with HoodieClusteringConfig >

[jira] [Updated] (HUDI-5764) Allow lazy rollback for async indexer commit

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5764: Fix Version/s: 0.13.1 > Allow lazy rollback for async indexer commit >

[jira] [Updated] (HUDI-5768) Fail to read metadata table in Spark Datasource

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5768: Fix Version/s: 0.13.1 > Fail to read metadata table in Spark Datasource >

[jira] [Updated] (HUDI-4968) Fix ambiguous stream read config

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4968: Fix Version/s: 0.13.1 > Fix ambiguous stream read config > > >

[jira] [Updated] (HUDI-5270) Duplicate key error when insert_overwrite same partition in multi writer

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5270: Fix Version/s: 0.13.1 (was: 0.14.0) > Duplicate key error when insert_overwrite same

[jira] [Updated] (HUDI-5721) Add Github actions on more validations

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5721: Status: In Progress (was: Open) > Add Github actions on more validations >

[jira] [Resolved] (HUDI-5721) Add Github actions on more validations

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-5721. - > Add Github actions on more validations > -- > > Key:

[jira] [Updated] (HUDI-5720) Improve validation script of staged source release

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5720: Fix Version/s: 0.13.1 > Improve validation script of staged source release >

[jira] [Resolved] (HUDI-5720) Improve validation script of staged source release

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-5720. - > Improve validation script of staged source release > -- > >

[jira] [Updated] (HUDI-5718) Unsupported Operation Exception for compaction

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5718: Fix Version/s: 0.13.1 (was: 0.13.0) > Unsupported Operation Exception for compaction

[jira] [Updated] (HUDI-5720) Improve validation script of staged source release

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5720: Status: In Progress (was: Open) > Improve validation script of staged source release >

[jira] [Updated] (HUDI-5671) BucketIndexPartitioner partition algorithm skew

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5671: Fix Version/s: 0.13.1 (was: 0.13.0) > BucketIndexPartitioner partition algorithm

[jira] [Created] (HUDI-6212) Support spark3.0.x

2023-05-15 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-6212: --- Summary: Support spark3.0.x Key: HUDI-6212 URL: https://issues.apache.org/jira/browse/HUDI-6212 Project: Apache Hudi Issue Type: Improvement Reporter: Yue

[jira] [Updated] (HUDI-5710) Load all partitions in advance when using KEEP_LATEST_FILE_VERSIONS clean policy and MDT enable

2023-02-06 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5710: Summary: Load all partitions in advance when using KEEP_LATEST_FILE_VERSIONS clean policy and MDT enable

[jira] [Created] (HUDI-5710) Load All partitions in advance when using KEEP_LATEST_FILE_VERSIONS clean policy and MDT enable

2023-02-06 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5710: --- Summary: Load All partitions in advance when using KEEP_LATEST_FILE_VERSIONS clean policy and MDT enable Key: HUDI-5710 URL: https://issues.apache.org/jira/browse/HUDI-5710

[jira] [Assigned] (HUDI-5422) Control KEPP_LATEST_VERSIONS clean replaced files immediately or not

2022-12-19 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-5422: --- Assignee: Yue Zhang > Control KEPP_LATEST_VERSIONS clean replaced files immediately or not >

[jira] [Created] (HUDI-5422) Control KEPP_LATEST_VERSIONS clean replaced files immediately or not

2022-12-19 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5422: --- Summary: Control KEPP_LATEST_VERSIONS clean replaced files immediately or not Key: HUDI-5422 URL: https://issues.apache.org/jira/browse/HUDI-5422 Project: Apache Hudi

[jira] [Commented] (HUDI-5369) TestDisruptorExecutionInSpark.testExecutor is flaky

2022-12-11 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645922#comment-17645922 ] Yue Zhang commented on HUDI-5369: - Hi alexey. Sorry for this issue. Is there any details info for this

[jira] [Assigned] (HUDI-5190) Consuming records from Iterator directly instead of using inner message queue

2022-11-10 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-5190: --- Assignee: Yue Zhang > Consuming records from Iterator directly instead of using inner message queue

[jira] [Created] (HUDI-5190) Consuming records from Iterator directly instead of using inner message queue

2022-11-10 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5190: --- Summary: Consuming records from Iterator directly instead of using inner message queue Key: HUDI-5190 URL: https://issues.apache.org/jira/browse/HUDI-5190 Project: Apache Hudi

[jira] [Updated] (HUDI-5186) Parallelism does not take effect when hoodie.combine.before.upsert/insert false

2022-11-09 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5186: Summary: Parallelism does not take effect when hoodie.combine.before.upsert/insert false (was: Parallelism

[jira] [Assigned] (HUDI-5186) Parallelism does not take effect when hoodie.combine.before.upsert false

2022-11-09 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-5186: --- Assignee: Yue Zhang > Parallelism does not take effect when hoodie.combine.before.upsert false >

[jira] [Created] (HUDI-5186) Parallelism does not take effect when hoodie.combine.before.upsert false

2022-11-09 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5186: --- Summary: Parallelism does not take effect when hoodie.combine.before.upsert false Key: HUDI-5186 URL: https://issues.apache.org/jira/browse/HUDI-5186 Project: Apache Hudi

[jira] [Assigned] (HUDI-5175) Improving FileIndex load performance in PARALLELISM mode

2022-11-07 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-5175: --- Assignee: Yue Zhang > Improving FileIndex load performance in PARALLELISM mode >

[jira] [Created] (HUDI-5175) Improving FileIndex load performance in PARALLELISM mode

2022-11-07 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5175: --- Summary: Improving FileIndex load performance in PARALLELISM mode Key: HUDI-5175 URL: https://issues.apache.org/jira/browse/HUDI-5175 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-5106) Unify the test code for both bounded in memory queue and disruptor

2022-10-31 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-5106: --- Summary: Unify the test code for both bounded in memory queue and disruptor Key: HUDI-5106 URL: https://issues.apache.org/jira/browse/HUDI-5106 Project: Apache Hudi

[jira] [Comment Edited] (HUDI-1575) Early detection by periodically checking last written commit & active markers

2022-05-20 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540154#comment-17540154 ] Yue Zhang edited comment on HUDI-1575 at 5/20/22 2:58 PM: -- Eager conflict

[jira] [Commented] (HUDI-1575) Early detection by periodically checking last written commit & active markers

2022-05-20 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540158#comment-17540158 ] Yue Zhang commented on HUDI-1575: - Also we may need more design here for example 1. How to adapt both

[jira] [Commented] (HUDI-1575) Early detection by periodically checking last written commit & active markers

2022-05-20 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540154#comment-17540154 ] Yue Zhang commented on HUDI-1575: - Eager conflict detection based on marker file For now we have three

[jira] [Updated] (HUDI-3963) Use Lock-Free Message Queue Improving Hoodie Writing Efficiency

2022-05-12 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3963: Summary: Use Lock-Free Message Queue Improving Hoodie Writing Efficiency (was: Using Lock-Free Message

[jira] [Updated] (HUDI-3963) Using Lock-Free Message Queue Improving Hoodie Writing Efficiency

2022-05-12 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3963: Summary: Using Lock-Free Message Queue Improving Hoodie Writing Efficiency (was: Improve Hoodie Writing

[jira] [Created] (HUDI-3963) Improve Hoodie Writing Data Efficiency Using Lock-Free Message Queue

2022-04-24 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3963: --- Summary: Improve Hoodie Writing Data Efficiency Using Lock-Free Message Queue Key: HUDI-3963 URL: https://issues.apache.org/jira/browse/HUDI-3963 Project: Apache Hudi

[jira] [Created] (HUDI-3916) New Key Generator Option: NanoidKeyGenerator

2022-04-19 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3916: --- Summary: New Key Generator Option: NanoidKeyGenerator Key: HUDI-3916 URL: https://issues.apache.org/jira/browse/HUDI-3916 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-3858) Shade javax.servlet for Spark bundle jar

2022-04-11 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3858: --- Summary: Shade javax.servlet for Spark bundle jar Key: HUDI-3858 URL: https://issues.apache.org/jira/browse/HUDI-3858 Project: Apache Hudi Issue Type: Bug

[jira] [Commented] (HUDI-3636) Clustering fails due to marker creation failure

2022-04-04 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516839#comment-17516839 ] Yue Zhang commented on HUDI-3636: - Have a try but couldn't reproduce this error based on master branch.

[jira] [Assigned] (HUDI-3116) HoodieDropPartitionsTool

2022-04-03 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-3116: --- Assignee: Yue Zhang > HoodieDropPartitionsTool > > > Key:

[jira] [Commented] (HUDI-3650) Revisit all usages of filterPendingCompactionTimeline()

2022-04-03 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516464#comment-17516464 ] Yue Zhang commented on HUDI-3650: - Based on master branch, There are several places calling this

[jira] [Created] (HUDI-3766) Check MDT flag in hoodie.properties for readers if necessary

2022-03-31 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3766: --- Summary: Check MDT flag in hoodie.properties for readers if necessary Key: HUDI-3766 URL: https://issues.apache.org/jira/browse/HUDI-3766 Project: Apache Hudi Issue

[jira] [Commented] (HUDI-3688) Double check MT init behavior for MT rollout

2022-03-29 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514389#comment-17514389 ] Yue Zhang commented on HUDI-3688: - Step1: do several normal ingestion using 0.10.0. Step2: do another

[jira] [Assigned] (HUDI-3647) Ignore errors if metadata table has not been initialized fully

2022-03-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-3647: --- Assignee: Yue Zhang > Ignore errors if metadata table has not been initialized fully >

[jira] [Assigned] (HUDI-3635) Fix HoodieMetadataTableValidator around comparison of partition path listing

2022-03-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-3635: --- Assignee: Yue Zhang > Fix HoodieMetadataTableValidator around comparison of partition path listing

[jira] [Commented] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

2022-03-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17510409#comment-17510409 ] Yue Zhang commented on HUDI-3495: - CC [~guoyihua] and [~pwason] > Reading keys in parallel from

[jira] [Comment Edited] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

2022-03-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17510408#comment-17510408 ] Yue Zhang edited comment on HUDI-3495 at 3/22/22, 11:01 AM: Hi [~xushiyan]

[jira] [Commented] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

2022-03-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17510408#comment-17510408 ] Yue Zhang commented on HUDI-3495: - Hi [~xushiyan] Sorry for late response. Just missing this Ticket :(

[jira] [Commented] (HUDI-3453) Metadata table throws NPE when scheduling compaction plan

2022-03-20 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509564#comment-17509564 ] Yue Zhang commented on HUDI-3453: - Hi [~danny0405] Looks like there is a concurrency issue here. Could you

[jira] [Commented] (HUDI-3377) Metadata table and FS mismatch during writing new partition

2022-02-20 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495281#comment-17495281 ] Yue Zhang commented on HUDI-3377: - Hi [~vinoth], It's not files mismatched. It' s partitions info. When

[jira] [Comment Edited] (HUDI-3301) MergedLogRecordReader inline reading should be stateless and thread safe

2022-02-16 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493623#comment-17493623 ] Yue Zhang edited comment on HUDI-3301 at 2/17/22, 3:38 AM: --- Hi [~manojg] and

[jira] [Comment Edited] (HUDI-3301) MergedLogRecordReader inline reading should be stateless and thread safe

2022-02-16 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493623#comment-17493623 ] Yue Zhang edited comment on HUDI-3301 at 2/17/22, 3:26 AM: --- Hi [~manojg] and

[jira] [Commented] (HUDI-3301) MergedLogRecordReader inline reading should be stateless and thread safe

2022-02-16 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493623#comment-17493623 ] Yue Zhang commented on HUDI-3301: - Hi [~manojg] and [~guoyihua] Just a quick think of this problem, for

[jira] [Assigned] (HUDI-3429) Support clustering scheduleAndExecute for hudi-cli and add clustering-cli Tests

2022-02-14 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-3429: --- Assignee: Yue Zhang > Support clustering scheduleAndExecute for hudi-cli and add clustering-cli >

[jira] [Created] (HUDI-3429) Support clustering scheduleAndExecute for hudi-cli and add clustering-cli Tests

2022-02-14 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3429: --- Summary: Support clustering scheduleAndExecute for hudi-cli and add clustering-cli Tests Key: HUDI-3429 URL: https://issues.apache.org/jira/browse/HUDI-3429 Project: Apache

[jira] [Created] (HUDI-3428) Improve BaseFileUtils for HFile and improve TableSchemaResolver UT

2022-02-14 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3428: --- Summary: Improve BaseFileUtils for HFile and improve TableSchemaResolver UT Key: HUDI-3428 URL: https://issues.apache.org/jira/browse/HUDI-3428 Project: Apache Hudi

[jira] [Assigned] (HUDI-3421) Pending clustering may break AbstractTableFileSystemView

2022-02-14 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang reassigned HUDI-3421: --- Assignee: Yue Zhang > Pending clustering may break AbstractTableFileSystemView >

[jira] [Updated] (HUDI-3421) Pending clustering may break AbstractTableFileSystemView

2022-02-14 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3421: Summary: Pending clustering may break AbstractTableFileSystemView (was: [HUDI-]Pending clustering may

[jira] [Created] (HUDI-3421) [HUDI-]Pending clustering may break AbstractTableFileSystemView

2022-02-14 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3421: --- Summary: [HUDI-]Pending clustering may break AbstractTableFileSystemView Key: HUDI-3421 URL: https://issues.apache.org/jira/browse/HUDI-3421 Project: Apache Hudi

[jira] [Commented] (HUDI-3398) Schema validation fails for metadata table base file

2022-02-09 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489494#comment-17489494 ] Yue Zhang commented on HUDI-3398: - The root cause is that https://github.com/apache/hudi/pull/4649 brings

[jira] [Created] (HUDI-3370) The files recorded in the commit do not match the actual ones for MOR Compaction

2022-02-06 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3370: --- Summary: The files recorded in the commit do not match the actual ones for MOR Compaction Key: HUDI-3370 URL: https://issues.apache.org/jira/browse/HUDI-3370 Project: Apache

[jira] [Updated] (HUDI-3369) New ScheduleAndExecute mode for HoodieCompactor and hudi-cli

2022-02-05 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3369: Description: Users can use --mode schedule to build a compaction plan and execute this plan immediately

[jira] [Updated] (HUDI-3369) New ScheduleAndExecute mode for HoodieCompactor and hudi-cli

2022-02-05 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3369: Description: Users can use --mode schedule to build a compaction plan and execute this plan immediately

[jira] [Created] (HUDI-3369) New ScheduleAndExecute mode for HoodieCompactor and hudi-cli

2022-02-05 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3369: --- Summary: New ScheduleAndExecute mode for HoodieCompactor and hudi-cli Key: HUDI-3369 URL: https://issues.apache.org/jira/browse/HUDI-3369 Project: Apache Hudi Issue

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2022-01-19 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479084#comment-17479084 ] Yue Zhang commented on HUDI-3122: - Since https://github.com/apache/hudi/pull/4551 is merged maybe we can

[jira] [Created] (HUDI-3281) Tuning performance of getAllPartitionPaths in FileSystemBackedTableMetadata

2022-01-19 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3281: --- Summary: Tuning performance of getAllPartitionPaths in FileSystemBackedTableMetadata Key: HUDI-3281 URL: https://issues.apache.org/jira/browse/HUDI-3281 Project: Apache Hudi

[jira] [Created] (HUDI-3212) Tuning merge small archive files

2022-01-11 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3212: --- Summary: Tuning merge small archive files Key: HUDI-3212 URL: https://issues.apache.org/jira/browse/HUDI-3212 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-3183) Fix wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter

2022-01-05 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3183: --- Summary: Fix wrong result of HoodieArchivedTimeline loadInstants with TimeRangeFilter Key: HUDI-3183 URL: https://issues.apache.org/jira/browse/HUDI-3183 Project: Apache Hudi

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2022-01-03 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468314#comment-17468314 ] Yue Zhang commented on HUDI-3122: - Hi [~wenningd] is there any updates ? :) > presto query failed for

[jira] [Commented] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466689#comment-17466689 ] Yue Zhang commented on HUDI-3107: - Hi Raymond, this is the wrong GitHub id. -- Yue (Daniel) Zhang,

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466280#comment-17466280 ] Yue Zhang commented on HUDI-3122: - I see it in hudi-presto-bundle.jar but i am not sure if it solve your

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466278#comment-17466278 ] Yue Zhang commented on HUDI-3122: - Okay I got it dep.hudi.version in presto is 0.5.3 which means there is

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466271#comment-17466271 ] Yue Zhang commented on HUDI-3122: - Also I actually add hbase-shaded-server in hudi presto bundle, But as

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466270#comment-17466270 ] Yue Zhang commented on HUDI-3122: - emmm what i means it that if you used Presto 0.261 , there will be a

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466268#comment-17466268 ] Yue Zhang commented on HUDI-3122: - emmm it's little strange here, as the

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466265#comment-17466265 ] Yue Zhang commented on HUDI-3122: - Hi what presto version did you used? > presto query failed for

[jira] [Created] (HUDI-3116) HoodieDropPartitionsTool

2021-12-28 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3116: --- Summary: HoodieDropPartitionsTool Key: HUDI-3116 URL: https://issues.apache.org/jira/browse/HUDI-3116 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-27 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3107: --- Summary: Fix HiveSyncTool drop partitions using JDBC Key: HUDI-3107 URL: https://issues.apache.org/jira/browse/HUDI-3107 Project: Apache Hudi Issue Type: Bug

[jira] [Created] (HUDI-3070) Improve Test

2021-12-20 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3070: --- Summary: Improve Test Key: HUDI-3070 URL: https://issues.apache.org/jira/browse/HUDI-3070 Project: Apache Hudi Issue Type: Improvement Reporter: Yue Zhang

[jira] [Created] (HUDI-3045) new ClusteringPlanStrategy to use regex choose partitions when building clustering plan.

2021-12-16 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3045: --- Summary: new ClusteringPlanStrategy to use regex choose partitions when building clustering plan. Key: HUDI-3045 URL: https://issues.apache.org/jira/browse/HUDI-3045 Project:

[jira] [Created] (HUDI-3038) Comprehensive mechanism around cleaning the archived timeline

2021-12-16 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-3038: --- Summary: Comprehensive mechanism around cleaning the archived timeline Key: HUDI-3038 URL: https://issues.apache.org/jira/browse/HUDI-3038 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-2892) Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

2021-11-30 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2892: Description: details could find in https://github.com/apache/hudi/issues/4163 (was:   Step 1  Do a normal

[jira] [Updated] (HUDI-2892) Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

2021-11-30 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2892: Description:   Step 1  Do a normal hudi insert  drwxr-xr-x   3 yuezhang  FREEWHEELMEDIA\Domain Users    

[jira] [Updated] (HUDI-2892) Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

2021-11-30 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2892: Description: **Describe the problem you faced** If there's a pending clustering instant still existed in

[jira] [Created] (HUDI-2892) Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results

2021-11-30 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2892: --- Summary: Pending Clustering may stain the ActiveTimeLine and lead to incomplete query results Key: HUDI-2892 URL: https://issues.apache.org/jira/browse/HUDI-2892 Project:

[jira] [Created] (HUDI-2833) Clean up unused archive files instead of expanding indefinitely

2021-11-22 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2833: --- Summary: Clean up unused archive files instead of expanding indefinitely Key: HUDI-2833 URL: https://issues.apache.org/jira/browse/HUDI-2833 Project: Apache Hudi

[jira] [Created] (HUDI-2683) Parallelize deleting archived hoodie commits

2021-11-03 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2683: --- Summary: Parallelize deleting archived hoodie commits Key: HUDI-2683 URL: https://issues.apache.org/jira/browse/HUDI-2683 Project: Apache Hudi Issue Type: Task

[jira] [Created] (HUDI-2658) When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED

2021-10-31 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2658: --- Summary: When disable auto clean, do not check if MIN_COMMITS_TO_KEEP was larger CLEANER_COMMITS_RETAINED Key: HUDI-2658 URL: https://issues.apache.org/jira/browse/HUDI-2658

[jira] [Updated] (HUDI-2648) Retry FileSystem action when caught runtime exception

2021-10-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2648: Description: Hoodie will do lots of list/get/put/delete etc actions on filesystem. Sometimes will meet the

[jira] [Created] (HUDI-2648) Retry FileSystem action when caught runtime exception

2021-10-28 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2648: --- Summary: Retry FileSystem action when caught runtime exception Key: HUDI-2648 URL: https://issues.apache.org/jira/browse/HUDI-2648 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-2533) new option for hoodieClusteringJob to check, rollback and re-execute the last failed clustering job

2021-10-08 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2533: --- Summary: new option for hoodieClusteringJob to check, rollback and re-execute the last failed clustering job Key: HUDI-2533 URL: https://issues.apache.org/jira/browse/HUDI-2533

[jira] [Created] (HUDI-2489) Tuning HoodieROTablePathFilter by caching, aiming to reduce unnecessary list/get requests

2021-09-26 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2489: --- Summary: Tuning HoodieROTablePathFilter by caching, aiming to reduce unnecessary list/get requests Key: HUDI-2489 URL: https://issues.apache.org/jira/browse/HUDI-2489 Project:

[jira] [Created] (HUDI-2435) Tuning clustering job handle errors

2021-09-15 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2435: --- Summary: Tuning clustering job handle errors Key: HUDI-2435 URL: https://issues.apache.org/jira/browse/HUDI-2435 Project: Apache Hudi Issue Type: Task

[jira] [Created] (HUDI-2409) Using HBase shaded jars in Hudi presto bundle

2021-09-09 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2409: --- Summary: Using HBase shaded jars in Hudi presto bundle Key: HUDI-2409 URL: https://issues.apache.org/jira/browse/HUDI-2409 Project: Apache Hudi Issue Type: Task

[jira] [Comment Edited] (HUDI-2355) after clustering with archive meet data incorrect

2021-08-26 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404991#comment-17404991 ] Yue Zhang edited comment on HUDI-2355 at 8/26/21, 8:20 AM: --- Actually, this

[jira] [Comment Edited] (HUDI-2355) after clustering with archive meet data incorrect

2021-08-26 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404991#comment-17404991 ] Yue Zhang edited comment on HUDI-2355 at 8/26/21, 8:19 AM: --- Actually, this

[jira] [Comment Edited] (HUDI-2355) after clustering with archive meet data incorrect

2021-08-26 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404991#comment-17404991 ] Yue Zhang edited comment on HUDI-2355 at 8/26/21, 6:41 AM: --- Actually, this

[jira] [Commented] (HUDI-2355) after clustering with archive meet data incorrect

2021-08-26 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404991#comment-17404991 ] Yue Zhang commented on HUDI-2355: - Actually, this problems does exist  based to current master branch that

[jira] [Comment Edited] (HUDI-2355) after clustering with archive meet data incorrect

2021-08-26 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404991#comment-17404991 ] Yue Zhang edited comment on HUDI-2355 at 8/26/21, 6:40 AM: --- Actually, this

[jira] [Created] (HUDI-2345) Supply build-in user UserDefinedBulkInsertPartitioner named RDDCustomColumnsSortPartitioner for bulk insert action

2021-08-23 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2345: --- Summary: Supply build-in user UserDefinedBulkInsertPartitioner named RDDCustomColumnsSortPartitioner for bulk insert action Key: HUDI-2345 URL:

[jira] [Created] (HUDI-2338) Hoodie data update reject clustering using SparkRejectClusteringStrategy

2021-08-19 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2338: --- Summary: Hoodie data update reject clustering using SparkRejectClusteringStrategy Key: HUDI-2338 URL: https://issues.apache.org/jira/browse/HUDI-2338 Project: Apache Hudi

[jira] [Created] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2277: --- Summary: Let HoodieDeltaStreamer reading ORC files using ORCDFSSource Key: HUDI-2277 URL: https://issues.apache.org/jira/browse/HUDI-2277 Project: Apache Hudi Issue

<    1   2   3   4   5   >