[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use *sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition +sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition +sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition _sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition _ sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition _sameCols.size() !=

[jira] [Updated] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-7436: Description: In HoodieMergeHelper#composeSchemaEvolutionTransformer we use this condition _sameCols.size() !=

[jira] [Created] (HUDI-7436) Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE

2024-02-21 Thread lei w (Jira)
lei w created HUDI-7436: --- Summary: Not need to reWriteRecord when method SchemaCompatibility.checkReaderWriterCompatibility return SchemaCompatibilityType.COMPATIBLE Key: HUDI-7436 URL:

[jira] [Created] (HUDI-6143) Use the startoffset of each logfile recorded in the deltacommit metadata to read the logfile

2023-04-26 Thread lei w (Jira)
lei w created HUDI-6143: --- Summary: Use the startoffset of each logfile recorded in the deltacommit metadata to read the logfile Key: HUDI-6143 URL: https://issues.apache.org/jira/browse/HUDI-6143 Project:

[jira] [Created] (HUDI-6081) Fix load partition's files duplicately in READ_AS_STREAMING mode

2023-04-14 Thread lei w (Jira)
lei w created HUDI-6081: --- Summary: Fix load partition's files duplicately in READ_AS_STREAMING mode Key: HUDI-6081 URL: https://issues.apache.org/jira/browse/HUDI-6081 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-4613) Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2023-04-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-4613: Attachment: withoutChange.png > Avoid the use of regex expressions when call hoodieFileGroup#addLogFile > function

[jira] [Updated] (HUDI-4613) Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2023-04-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-4613: Attachment: withChange.png > Avoid the use of regex expressions when call hoodieFileGroup#addLogFile > function >

[jira] [Created] (HUDI-6029) Rollback may omit invalid files when commitMetadata is not completed for MOR

2023-04-03 Thread lei w (Jira)
lei w created HUDI-6029: --- Summary: Rollback may omit invalid files when commitMetadata is not completed for MOR Key: HUDI-6029 URL: https://issues.apache.org/jira/browse/HUDI-6029 Project: Apache Hudi

[jira] [Closed] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

2023-01-31 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w closed HUDI-5565. --- Resolution: Fixed > Application restart may cause data lose when task parallelism is changed >

[jira] [Commented] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

2023-01-31 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682514#comment-17682514 ] lei w commented on HUDI-5565: - [HUDI-5506|https://github.com/apache/hudi/pull/7611] has resolved. >

[jira] [Updated] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

2023-01-17 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-5565: Description: [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start

[jira] [Commented] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

2023-01-16 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677597#comment-17677597 ] lei w commented on HUDI-5565: - [~danny0405] Could you give me some advice? Thanks > Application restart may

[jira] [Created] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

2023-01-16 Thread lei w (Jira)
lei w created HUDI-5565: --- Summary: Application restart may cause data lose when task parallelism is changed Key: HUDI-5565 URL: https://issues.apache.org/jira/browse/HUDI-5565 Project: Apache Hudi

[jira] [Updated] (HUDI-5515) Fix concurrency conflict in ClusteringOperator

2023-01-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-5515: Summary: Fix concurrency conflict in ClusteringOperator (was: Fix concurrency conflict in for flink async

[jira] [Created] (HUDI-5515) Fix concurrency conflict in ClusteringOperator

2023-01-08 Thread lei w (Jira)
lei w created HUDI-5515: --- Summary: Fix concurrency conflict in ClusteringOperator Key: HUDI-5515 URL: https://issues.apache.org/jira/browse/HUDI-5515 Project: Apache Hudi Issue Type: Bug

[jira] [Commented] (HUDI-5504) Fix concurrency conflict when asyncCompaction is enabled

2023-01-04 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654737#comment-17654737 ] lei w commented on HUDI-5504: - [~danny0405] Do you have any advice? > Fix concurrency conflict when

[jira] [Updated] (HUDI-5504) Fix concurrency conflict when asyncCompaction is enabled

2023-01-04 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-5504: https://github.com/apache/hudi/pull/7609 > Fix concurrency conflict when asyncCompaction is enabled >

[jira] [Created] (HUDI-5504) Fix concurrency conflict when asyncCompaction is enabled

2023-01-04 Thread lei w (Jira)
lei w created HUDI-5504: --- Summary: Fix concurrency conflict when asyncCompaction is enabled Key: HUDI-5504 URL: https://issues.apache.org/jira/browse/HUDI-5504 Project: Apache Hudi Issue Type: Bug

[jira] [Commented] (HUDI-3661) Flink async compaction is not thread safe when use watermark

2022-12-01 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641823#comment-17641823 ] lei w commented on HUDI-3661: - hi,[~hd zhou] , [~danny0405]. In our cluster, we removed the async executor to

[jira] [Comment Edited] (HUDI-2833) Clean up unused archive files instead of expanding indefinitely

2022-11-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640509#comment-17640509 ] lei w edited comment on HUDI-2833 at 11/29/22 9:41 AM: --- hi ,[~zhangyue19921010],

[jira] [Commented] (HUDI-2833) Clean up unused archive files instead of expanding indefinitely

2022-11-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640509#comment-17640509 ] lei w commented on HUDI-2833: - hi ,[~zhangyue19921010], this feature is to merage all candidateFiles(small

[jira] [Updated] (HUDI-4613) Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2022-08-15 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-4613: Summary: Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function (was: Avoid the use of

[jira] [Updated] (HUDI-4613) Avoid the use of regular expressions when call hoodieFileGroup#addLogFile function

2022-08-12 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-4613: Description: When the number of logFile files exceeds a certain amount of data, the construction of fsview will

[jira] [Updated] (HUDI-4613) Avoid the use of regular expressions when call hoodieFileGroup#addLogFile function

2022-08-12 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HUDI-4613: Component/s: core (was: compaction) > Avoid the use of regular expressions when call

[jira] [Created] (HUDI-4613) Avoid the use of regular expressions when call hoodieFileGroup#addLogFile function

2022-08-12 Thread lei w (Jira)
lei w created HUDI-4613: --- Summary: Avoid the use of regular expressions when call hoodieFileGroup#addLogFile function Key: HUDI-4613 URL: https://issues.apache.org/jira/browse/HUDI-4613 Project: Apache Hudi

[jira] [Created] (HUDI-4328) New partitions will not be consumed in snapshot read

2022-06-27 Thread lei w (Jira)
lei w created HUDI-4328: --- Summary: New partitions will not be consumed in snapshot read Key: HUDI-4328 URL: https://issues.apache.org/jira/browse/HUDI-4328 Project: Apache Hudi Issue Type: Bug