Re: [PR] lazy init secondary view in FS view [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-2144306711 ## CI report: * 5a90c7ca72ea60d0dfabaa4dd2cc3b7c072a0bdf Azure:

Re: [PR] lazy init secondary view in FS view [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-2144299256 ## CI report: * 5a90c7ca72ea60d0dfabaa4dd2cc3b7c072a0bdf Azure:

Re: [PR] [MINOR] Avoid logging full commit metadata at info level [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11382: URL: https://github.com/apache/hudi/pull/11382#issuecomment-2144292602 ## CI report: * 29045b7b2a08af0ec4fc23174a9980d0f6d17f5b Azure:

Re: [PR] lazy init secondary view in FS view [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-2144250418 ## CI report: * 5a90c7ca72ea60d0dfabaa4dd2cc3b7c072a0bdf Azure:

Re: [PR] [HUDI-7825]Support Report pending clustering and compaction plan metric [hudi]

2024-06-02 Thread via GitHub
danny0405 commented on code in PR #11377: URL: https://github.com/apache/hudi/pull/11377#discussion_r1623764919 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java: ## @@ -272,6 +275,28 @@ public void

Re: [PR] lazy init secondary view in FS view [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-2144243418 ## CI report: * 51ff1e0f5fe7c211c13961fb2a7709a456519fb3 Azure:

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on code in PR #11381: URL: https://github.com/apache/hudi/pull/11381#discussion_r1623759939 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/utils/AvroSchemaEvolutionUtils.java: ## @@ -113,6 +120,21 @@ public static InternalSchema

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
danny0405 commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2144233051 There are still test failures: ```java [ERROR] Failures: [ERROR] TestWriteCopyOnWrite.testCheckpoint:69 Expected: is "20240602151939789" but: was

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on code in PR #11381: URL: https://github.com/apache/hudi/pull/11381#discussion_r1623757086 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/utils/AvroSchemaEvolutionUtils.java: ## @@ -113,6 +120,21 @@ public static InternalSchema

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on code in PR #11381: URL: https://github.com/apache/hudi/pull/11381#discussion_r1623756369 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/utils/AvroSchemaEvolutionUtils.java: ## @@ -113,6 +120,21 @@ public static InternalSchema

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on code in PR #11381: URL: https://github.com/apache/hudi/pull/11381#discussion_r1623755994 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/utils/AvroSchemaEvolutionUtils.java: ## @@ -113,6 +120,21 @@ public static InternalSchema

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
nsivabalan commented on code in PR #11381: URL: https://github.com/apache/hudi/pull/11381#discussion_r1623751675 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/utils/AvroSchemaEvolutionUtils.java: ## @@ -113,6 +120,21 @@ public static InternalSchema

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11381: URL: https://github.com/apache/hudi/pull/11381#issuecomment-2144206868 ## CI report: * abd7d6583d8d4469ded1cf054c395e86e14eeef0 Azure:

Re: [PR] [MINOR] Avoid logging full commit metadata at info level [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11382: URL: https://github.com/apache/hudi/pull/11382#issuecomment-2144206887 ## CI report: * 29045b7b2a08af0ec4fc23174a9980d0f6d17f5b Azure:

Re: [PR] [MINOR] Avoid logging full commit metadata at info level [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11382: URL: https://github.com/apache/hudi/pull/11382#issuecomment-2144201342 ## CI report: * 29045b7b2a08af0ec4fc23174a9980d0f6d17f5b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-5956] fix spark DAG ui when write [hudi]

2024-06-02 Thread via GitHub
KnightChess closed pull request #11376: [HUDI-5956] fix spark DAG ui when write URL: https://github.com/apache/hudi/pull/11376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [MINOR] Avoid logging full commit metadata at info level [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on PR #11372: URL: https://github.com/apache/hudi/pull/11372#issuecomment-2144174244 > Could you also raise a PR against https://github.com/apache/hudi/tree/branch-0.x? @yihua https://github.com/apache/hudi/pull/11382 -- This is an automated message

[PR] [MINOR] Avoid logging full commit metadata at info level [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown opened a new pull request, #11382: URL: https://github.com/apache/hudi/pull/11382 ### Change Logs - Updates log messages to avoid logging full commit metadata after each table service to reduce volume of logs when working with large tables - Copy of

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
usberkeley commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2144173860 Danny, the test is shown as passed. If you have any suggestions for changes to the PR, please feel free to suggest them, I'd love to hear them and fix them. -- This is an automated

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11381: URL: https://github.com/apache/hudi/pull/11381#issuecomment-2144163955 ## CI report: * abd7d6583d8d4469ded1cf054c395e86e14eeef0 Azure:

Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11381: URL: https://github.com/apache/hudi/pull/11381#issuecomment-2144158079 ## CI report: * abd7d6583d8d4469ded1cf054c395e86e14eeef0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

[PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown opened a new pull request, #11381: URL: https://github.com/apache/hudi/pull/11381 ### Change Logs When deducing the writer schema, we should mark any missing columns as nullable when hoodie.write.set.null.for.missing.columns is set to true ### Impact

[jira] [Updated] (HUDI-7826) hoodie.write.set.null.for.missing.columns results in invalid objects

2024-06-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7826: - Labels: pull-request-available (was: ) > hoodie.write.set.null.for.missing.columns results in

[jira] [Created] (HUDI-7826) hoodie.write.set.null.for.missing.columns results in invalid objects

2024-06-02 Thread Timothy Brown (Jira)
Timothy Brown created HUDI-7826: --- Summary: hoodie.write.set.null.for.missing.columns results in invalid objects Key: HUDI-7826 URL: https://issues.apache.org/jira/browse/HUDI-7826 Project: Apache Hudi

Re: [PR] Bump io.airlift:aircompressor from 0.25 to 0.27 [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11380: URL: https://github.com/apache/hudi/pull/11380#issuecomment-2144105434 ## CI report: * 5042e73eb652ed8f770be16cf36fa59cf49637d0 Azure:

Re: [PR] Bump io.airlift:aircompressor from 0.25 to 0.27 [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11380: URL: https://github.com/apache/hudi/pull/11380#issuecomment-2144055525 ## CI report: * 5042e73eb652ed8f770be16cf36fa59cf49637d0 Azure:

Re: [PR] Bump io.airlift:aircompressor from 0.25 to 0.27 [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11380: URL: https://github.com/apache/hudi/pull/11380#issuecomment-2144053219 ## CI report: * 5042e73eb652ed8f770be16cf36fa59cf49637d0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

(hudi) branch dependabot/maven/io.airlift-aircompressor-0.27 created (now 5042e73eb65)

2024-06-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/io.airlift-aircompressor-0.27 in repository https://gitbox.apache.org/repos/asf/hudi.git at 5042e73eb65 Bump io.airlift:aircompressor from 0.25 to 0.27 No new

[PR] Bump io.airlift:aircompressor from 0.25 to 0.27 [hudi]

2024-06-02 Thread via GitHub
dependabot[bot] opened a new pull request, #11380: URL: https://github.com/apache/hudi/pull/11380 Bumps [io.airlift:aircompressor](https://github.com/airlift/aircompressor) from 0.25 to 0.27. Commits

Re: [PR] [HUDI-7713] Enforce ordering of fields during schema reconciliation [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11154: URL: https://github.com/apache/hudi/pull/11154#issuecomment-2143989224 ## CI report: * 12038dbde068e26f733a7b1c9cc7217019c31f25 UNKNOWN * 66bec1a1a6c32c79250e4bba3d94699b7d66b74d Azure:

Re: [PR] [HUDI-7713] Enforce ordering of fields during schema reconciliation [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11154: URL: https://github.com/apache/hudi/pull/11154#issuecomment-2143977418 ## CI report: * 12038dbde068e26f733a7b1c9cc7217019c31f25 UNKNOWN * 0d8349e7a74982fd69267c7db481b9648bcd80a5 Azure:

Re: [PR] [HUDI-7713] Enforce ordering of fields during schema reconciliation [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11154: URL: https://github.com/apache/hudi/pull/11154#issuecomment-2143975417 ## CI report: * 12038dbde068e26f733a7b1c9cc7217019c31f25 UNKNOWN * 0d8349e7a74982fd69267c7db481b9648bcd80a5 Azure:

Re: [PR] [HUDI-7713] Enforce ordering of fields during schema reconciliation [hudi]

2024-06-02 Thread via GitHub
the-other-tim-brown commented on code in PR #11154: URL: https://github.com/apache/hudi/pull/11154#discussion_r1623573818 ## hudi-common/src/main/java/org/apache/hudi/internal/schema/convert/AvroInternalSchemaConverter.java: ## @@ -151,11 +163,11 @@ public static Schema

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2143915382 ## CI report: * a7f8320f47046e457868ced0c56e98f2f35001e4 Azure:

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2143895337 ## CI report: * dcf9a4a7947b75943814493f528b90b68ee2b9aa Azure:

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2143892188 ## CI report: * dcf9a4a7947b75943814493f528b90b68ee2b9aa Azure:

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
usberkeley commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2143809929 Thanks Danny, you helped me locate the problem. I am fixing and adding some unit tests related to ALLOW_EMPTY_COMMIT and will submit the code soon. -- This is an automated message

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-06-02 Thread via GitHub
usberkeley commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2143809569 Thank you Danny for helping me locate the problem. I am currently working on the unit tests for ALLOW_EMPTY_COMMIT and will submit the code soon. -- This is an automated message from

Re: [PR] [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline [hudi]

2024-06-02 Thread via GitHub
xuzifu666 closed pull request #11365: [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline URL: https://github.com/apache/hudi/pull/11365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [HUDI-7805] FileSystemBasedLockProvider need delete lock file auto when occur lock conflict to avoid next write failed [hudi]

2024-06-02 Thread via GitHub
xuzifu666 closed pull request #11347: [HUDI-7805] FileSystemBasedLockProvider need delete lock file auto when occur lock conflict to avoid next write failed URL: https://github.com/apache/hudi/pull/11347 -- This is an automated message from the Apache Git Service. To respond to the message,