[jira] [Closed] (HUDI-7811) Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

2024-05-30 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7811. - Resolution: Fixed Fixed in the original PR itself - https://github.com/apache/hudi/pull/11043#discussion_

[jira] [Assigned] (HUDI-7811) Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

2024-05-30 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7811: - Assignee: Sagar Sumit > Enhance SparkBaseIndexSupport.getPrunedFileNames to return partition path

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621825753 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BloomFiltersIndexSupport.scala: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on PR #11043: URL: https://github.com/apache/hudi/pull/11043#issuecomment-2141335520 @KnightChess Addressed your feedback. Please take a look again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Created] (HUDI-7820) For bloom index reader path, prune based on min/max if colstats is enabled

2024-05-30 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7820: - Summary: For bloom index reader path, prune based on min/max if colstats is enabled Key: HUDI-7820 URL: https://issues.apache.org/jira/browse/HUDI-7820 Project: Apache Hudi

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621821717 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621821717 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621813395 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621813044 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
usberkeley commented on code in PR #11370: URL: https://github.com/apache/hudi/pull/11370#discussion_r1621802709 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -388,7 +388,7 @@ public static ConflictResolutionStrategy

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2141282627 ## CI report: * dcf9a4a7947b75943814493f528b90b68ee2b9aa Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2141237293 ## CI report: * dcf9a4a7947b75943814493f528b90b68ee2b9aa Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
danny0405 commented on code in PR #11370: URL: https://github.com/apache/hudi/pull/11370#discussion_r1621717283 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -388,7 +388,7 @@ public static ConflictResolutionStrategy

Re: [PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11370: URL: https://github.com/apache/hudi/pull/11370#issuecomment-2141231803 ## CI report: * dcf9a4a7947b75943814493f528b90b68ee2b9aa UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding [hudi]

2024-05-30 Thread via GitHub
yihua merged PR #11369: URL: https://github.com/apache/hudi/pull/11369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

(hudi) branch master updated: [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding (#11369)

2024-05-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0e55f0900d8 [HUDI-7817] Use Jackson Core instead of

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
usberkeley commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2141196482 There are many conflicts between my local code and Remote. This is my mistake. To make the PR record beautiful, I opened a new PR: https://github.com/apache/hudi/pull/11370 -- This i

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
usberkeley closed pull request #11359: [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… URL: https://github.com/apache/hudi/pull/11359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (HUDI-7819) Fix OptionsResolver#allowCommitOnEmptyBatch default value bug

2024-05-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7819: - Labels: pull-request-available (was: ) > Fix OptionsResolver#allowCommitOnEmptyBatch default valu

[PR] [HUDI-7819] Fix OptionsResolver#allowCommitOnEmptyBatch default value bug [hudi]

2024-05-30 Thread via GitHub
usberkeley opened a new pull request, #11370: URL: https://github.com/apache/hudi/pull/11370 ### Change Logs OptionsResolver#allowCommitOnEmptyBatch has a hardcoded default value of "false", while ALLOW_EMPTY_COMMIT (hoodie.allow.empty.commit) defaults to "true", this function return

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
KnightChess commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1621639791 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2141151906 ## CI report: * 4b149d9085498be66c6426b0c3fde90ddf382cec UNKNOWN * c8b14bd35eb233306750d8b31780d3da8ba2547d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2141146409 ## CI report: * 4b149d9085498be66c6426b0c3fde90ddf382cec UNKNOWN * 9ce101ca9d0c194af5b31b533c83fb21549ca8d3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[jira] [Closed] (HUDI-7810) Fix OptionsResolver#allowCommitOnEmptyBatch default value bug

2024-05-30 Thread bradley (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bradley closed HUDI-7810. - Resolution: Later > Fix OptionsResolver#allowCommitOnEmptyBatch default value bug > --

[jira] [Created] (HUDI-7819) Fix OptionsResolver#allowCommitOnEmptyBatch default value bug

2024-05-30 Thread bradley (Jira)
bradley created HUDI-7819: - Summary: Fix OptionsResolver#allowCommitOnEmptyBatch default value bug Key: HUDI-7819 URL: https://issues.apache.org/jira/browse/HUDI-7819 Project: Apache Hudi Issue Type

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2141115466 ## CI report: * 4b149d9085498be66c6426b0c3fde90ddf382cec UNKNOWN * 9ce101ca9d0c194af5b31b533c83fb21549ca8d3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2141109747 ## CI report: * 4b149d9085498be66c6426b0c3fde90ddf382cec UNKNOWN * 9ce101ca9d0c194af5b31b533c83fb21549ca8d3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [MINOR][TESTING][DNM] Validating 0.15.0 RC2 bundles [hudi]

2024-05-30 Thread via GitHub
yihua closed pull request #11340: [MINOR][TESTING][DNM] Validating 0.15.0 RC2 bundles URL: https://github.com/apache/hudi/pull/11340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [MINOR][Test][DNM] Test Azure CI on branch-0.x [hudi]

2024-05-30 Thread via GitHub
yihua closed pull request #10766: [MINOR][Test][DNM] Test Azure CI on branch-0.x URL: https://github.com/apache/hudi/pull/10766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Updated] (HUDI-7818) Flink Table planner not loading problem

2024-05-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7818: - Sprint: Sprint 2023-04-26 > Flink Table planner not loading problem >

[jira] [Created] (HUDI-7818) Flink Table planner not loading problem

2024-05-30 Thread Danny Chen (Jira)
Danny Chen created HUDI-7818: Summary: Flink Table planner not loading problem Key: HUDI-7818 URL: https://issues.apache.org/jira/browse/HUDI-7818 Project: Apache Hudi Issue Type: Improvement

Re: [PR] [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11369: URL: https://github.com/apache/hudi/pull/11369#issuecomment-2140985222 ## CI report: * 1718840e241dd32dc4c11885ba2bf1311bf822ec Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT]How to improve the speed of Flink writing to hudi ? [hudi]

2024-05-30 Thread via GitHub
HuangZhenQiu commented on issue #8071: URL: https://github.com/apache/hudi/issues/8071#issuecomment-2140979488 @danny0405 Do we have some best practices of (COW and MOR ) for Flink ingestion to Hudi? -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11369: URL: https://github.com/apache/hudi/pull/11369#issuecomment-2140935788 ## CI report: * 1718840e241dd32dc4c11885ba2bf1311bf822ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7816]: Provide SourceProfileSupplier option into the SnapshotLoadQuerySplitter [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11368: URL: https://github.com/apache/hudi/pull/11368#issuecomment-2140935768 ## CI report: * 1dde761d4147e9c1a94914759ca0bfd0f7d23ec7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11369: URL: https://github.com/apache/hudi/pull/11369#issuecomment-2140928147 ## CI report: * 1718840e241dd32dc4c11885ba2bf1311bf822ec UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7814] Exclude unused transitive dependencies that introduce vulnerabilities [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11364: URL: https://github.com/apache/hudi/pull/11364#issuecomment-2140920017 ## CI report: * ff1e3d8a934fe1a2c92e341be610516476bf5d7a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Updated] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7817: - Labels: pull-request-available (was: ) > Use Jackson Core instead of org.codehaus.jackson for JSO

[jira] [Updated] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7817: Description: org.codehaus.jackson is a older version of Jackson Core (com.fasterxml.jackson.core:jackson-cor

[PR] [HUDI-7817] Use Jackson Core instead of org.codehaus.jackson for JSON encoding [hudi]

2024-05-30 Thread via GitHub
yihua opened a new pull request, #11369: URL: https://github.com/apache/hudi/pull/11369 ### Change Logs `org.codehaus.jackson` is a older version of Jackson Core (`com.fasterxml.jackson.core:jackson-core`). `org.codehaus.jackson:jackson-mapper-asl` has critical vulnerabilities which

[jira] [Updated] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7817: Description: org.codehaus.jackson is a older version of Jackson Core (com.fasterxml.jackson.core:jackson-cor

[jira] [Updated] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7817: Description: org.codehaus.jackson is a older version of  > Use Jackson Core instead of org.codehaus.jackson

[jira] [Assigned] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-7817: --- Assignee: Ethan Guo > Use Jackson Core instead of org.codehaus.jackson for JSON encoding > --

[jira] [Updated] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7817: Fix Version/s: 1.0.0 > Use Jackson Core instead of org.codehaus.jackson for JSON encoding >

[jira] [Created] (HUDI-7817) Use Jackson Core instead of org.codehaus.jackson for JSON encoding

2024-05-30 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-7817: --- Summary: Use Jackson Core instead of org.codehaus.jackson for JSON encoding Key: HUDI-7817 URL: https://issues.apache.org/jira/browse/HUDI-7817 Project: Apache Hudi I

Re: [PR] [HUDI-7816]: Provide SourceProfileSupplier option into the SnapshotLoadQuerySplitter [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11368: URL: https://github.com/apache/hudi/pull/11368#issuecomment-2140868406 ## CI report: * 1dde761d4147e9c1a94914759ca0bfd0f7d23ec7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7814] Exclude unused transitive dependencies that introduce vulnerabilities [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11364: URL: https://github.com/apache/hudi/pull/11364#issuecomment-2140858167 ## CI report: * 3337f90b44d58d07c8a4055c9544f0e957d93226 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7816]: Add SourceProfileSupplier option to SnapshotLoadQuerySplitter [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11368: URL: https://github.com/apache/hudi/pull/11368#issuecomment-2140858256 ## CI report: * 1dde761d4147e9c1a94914759ca0bfd0f7d23ec7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7814] Exclude unused transitive dependencies that introduce vulnerabilities [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11364: URL: https://github.com/apache/hudi/pull/11364#issuecomment-2140848183 ## CI report: * 3337f90b44d58d07c8a4055c9544f0e957d93226 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Updated] (HUDI-7816) Pass the source profile to the snapshot query splitter

2024-05-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7816: - Labels: pull-request-available (was: ) > Pass the source profile to the snapshot query splitter >

[PR] [HUDI-7816]: Add SourceProfileSupplier option to SnapshotLoadQuerySplitter [hudi]

2024-05-30 Thread via GitHub
mattwong949 opened a new pull request, #11368: URL: https://github.com/apache/hudi/pull/11368 ### Change Logs Expanding the interface of the SnapshotLoadQuerySplitter to accept SourceProfileSupplier option. ### Impact Some SnapshotLoadQuerySplitter implementations may wa

[jira] [Created] (HUDI-7816) Pass the source profile to the snapshot query splitter

2024-05-30 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-7816: - Summary: Pass the source profile to the snapshot query splitter Key: HUDI-7816 URL: https://issues.apache.org/jira/browse/HUDI-7816 Project: Apache Hudi Is

(hudi) branch master updated (c758508b62f -> db7480820e3)

2024-05-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c758508b62f [HUDI-7769] Fix Hudi CDC read on Spark 3.3.4 and 3.4.3 (#11242) add db7480820e3 [MINOR] Fix GitHub CI c

Re: [PR] [MINOR] Fix GitHub CI concurrency [hudi]

2024-05-30 Thread via GitHub
yihua merged PR #11361: URL: https://github.com/apache/hudi/pull/11361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-7146] Integrate secondary index on reader path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11162: URL: https://github.com/apache/hudi/pull/11162#issuecomment-2140765294 ## CI report: * 3c52961bdbcb210e4c7140f5939143cfda7adb50 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-5863] Fix HoodieMetadataFileSystemView serving stale view at the timeline server [hudi]

2024-05-30 Thread via GitHub
Gatsby-Lee commented on PR #8079: URL: https://github.com/apache/hudi/pull/8079#issuecomment-2140708258 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

(hudi) branch branch-0.x updated: [MINOR] Fix GitHub CI concurrency (#11362)

2024-05-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch branch-0.x in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/branch-0.x by this push: new 70094deb391 [MINOR] Fix GitHub CI concurren

Re: [PR] [MINOR] Fix GitHub CI concurrency [hudi]

2024-05-30 Thread via GitHub
yihua merged PR #11362: URL: https://github.com/apache/hudi/pull/11362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

[jira] [Updated] (HUDI-7779) Guarding archival to not archive unintended commits

2024-05-30 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-7779: -- Description: Archiving commits from active timeline could lead to data consistency issue

Re: [PR] [HUDI-7146] Integrate secondary index on reader path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11162: URL: https://github.com/apache/hudi/pull/11162#issuecomment-2140514821 ## CI report: * a602c9c4234062e66877fc4bf2c50f94f43767bc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7146] Integrate secondary index on reader path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11162: URL: https://github.com/apache/hudi/pull/11162#issuecomment-2140488986 ## CI report: * a602c9c4234062e66877fc4bf2c50f94f43767bc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2140461564 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 475a1bc220eaee04fa78ba46a922b434b8306047 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [I] [SUPPORT] Spark-Hudi: Unable to perform Hard delete using Pyspark on HUDI table from AWS Glue [hudi]

2024-05-30 Thread via GitHub
soumilshah1995 commented on issue #11349: URL: https://github.com/apache/hudi/issues/11349#issuecomment-2140440102 good to hear that your issue is resolved cheers ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] [SUPPORT] Spark-Hudi: Unable to perform Hard delete using Pyspark on HUDI table from AWS Glue [hudi]

2024-05-30 Thread via GitHub
Ssv-21 commented on issue #11349: URL: https://github.com/apache/hudi/issues/11349#issuecomment-2140322503 Actually, I was using the native glue-based Hudi. But after going through your blogspot post, I tried using Hudi 0.14.0-Spark 3.3 bundle jar, and it worked. I believe something is w

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2140301377 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 63737caa30a0ba2ccc66b05bbeb3005d185eb4b7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2140271817 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 540d122ed1f6c9ee56730ec85fde9f0355b5d67a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

(hudi) branch master updated: [HUDI-7769] Fix Hudi CDC read on Spark 3.3.4 and 3.4.3 (#11242)

2024-05-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c758508b62f [HUDI-7769] Fix Hudi CDC read on Spark

Re: [PR] [HUDI-7769] Fix Hudi CDC read with legacy parquet file format on Spark [hudi]

2024-05-30 Thread via GitHub
yihua merged PR #11242: URL: https://github.com/apache/hudi/pull/11242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2140088833 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 540d122ed1f6c9ee56730ec85fde9f0355b5d67a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2140060183 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 540d122ed1f6c9ee56730ec85fde9f0355b5d67a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2140031009 ## CI report: * 4b149d9085498be66c6426b0c3fde90ddf382cec UNKNOWN * 9ce101ca9d0c194af5b31b533c83fb21549ca8d3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[jira] [Closed] (HUDI-7407) Add optional clean support to standalone compaction and clustering jobs

2024-05-30 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7407. - Resolution: Fixed > Add optional clean support to standalone compaction and clustering jobs >

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2139791741 ## CI report: * c8bf966468abfcab8121f7ba7a63f8098bbf965a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7407] Making clean optional in standalone compaction and clustering jobs [hudi]

2024-05-30 Thread via GitHub
codope merged PR #10668: URL: https://github.com/apache/hudi/pull/10668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

(hudi) branch master updated: [HUDI-7407] Making clean optional in standalone compaction and clustering jobs (#10668)

2024-05-30 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f0c1a88f8d0 [HUDI-7407] Making clean optional in s

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2139638319 ## CI report: * c8bf966468abfcab8121f7ba7a63f8098bbf965a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
usberkeley commented on code in PR #11359: URL: https://github.com/apache/hudi/pull/11359#discussion_r1620794903 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -370,7 +370,7 @@ public static ConflictResolutionStrategy

Re: [PR] [HUDI-7407] Making clean optional in standalone compaction and clustering jobs [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10668: URL: https://github.com/apache/hudi/pull/10668#issuecomment-2139636243 ## CI report: * 5a6c7723f716d5719a8011150f73077ab1ba3a1f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
usberkeley commented on code in PR #11359: URL: https://github.com/apache/hudi/pull/11359#discussion_r1620734871 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -370,7 +370,7 @@ public static ConflictResolutionStrategy

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11359: URL: https://github.com/apache/hudi/pull/11359#issuecomment-2139622234 ## CI report: * c8bf966468abfcab8121f7ba7a63f8098bbf965a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

(hudi) annotated tag release-0.15.0-rc3 updated (d0df1d4a94d -> 987b4dd1741)

2024-05-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to annotated tag release-0.15.0-rc3 in repository https://gitbox.apache.org/repos/asf/hudi.git *** WARNING: tag release-0.15.0-rc3 was modified! *** from d0df1d4a94d (commit) to 987b4dd1741 (tag)

svn commit: r69471 - in /dev/hudi/hudi-0.15.0-rc3: ./ hudi-0.15.0-rc3.src.tgz hudi-0.15.0-rc3.src.tgz.asc hudi-0.15.0-rc3.src.tgz.sha512

2024-05-30 Thread yihua
Author: yihua Date: Thu May 30 13:52:32 2024 New Revision: 69471 Log: Add Apache Hudi 0.15.0 RC3 source release Added: dev/hudi/hudi-0.15.0-rc3/ dev/hudi/hudi-0.15.0-rc3/hudi-0.15.0-rc3.src.tgz (with props) dev/hudi/hudi-0.15.0-rc3/hudi-0.15.0-rc3.src.tgz.asc dev/hudi/hudi-0.15.

Re: [PR] [HUDI-7810] Fix OptionsResolver#allowCommitOnEmptyBatch default value… [hudi]

2024-05-30 Thread via GitHub
usberkeley commented on code in PR #11359: URL: https://github.com/apache/hudi/pull/11359#discussion_r1620734871 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -370,7 +370,7 @@ public static ConflictResolutionStrategy

Re: [PR] [HUDI-7407] Making clean optional in standalone compaction and clustering jobs [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10668: URL: https://github.com/apache/hudi/pull/10668#issuecomment-2139516375 ## CI report: * b24eafcc00d5cf4a27ae7f9d7e70b1bfc5a12b1a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7407] Making clean optional in standalone compaction and clustering jobs [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #10668: URL: https://github.com/apache/hudi/pull/10668#issuecomment-2139501982 ## CI report: * b24eafcc00d5cf4a27ae7f9d7e70b1bfc5a12b1a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7407] Making clean optional in standalone compaction and clustering jobs [hudi]

2024-05-30 Thread via GitHub
codope commented on code in PR #10668: URL: https://github.com/apache/hudi/pull/10668#discussion_r1620652100 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java: ## @@ -92,6 +92,8 @@ public static class Config implements Serializable { public

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
KnightChess commented on code in PR #11043: URL: https://github.com/apache/hudi/pull/11043#discussion_r1620531072 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestBloomFiltersIndexSupport.scala: ## @@ -0,0 +1,261 @@ +/* + * Licensed to the Apache

[I] [SUPPORT] using spark's observe feature on dataframes saved by hudi is stuck [hudi]

2024-05-30 Thread via GitHub
szingerpeter opened a new issue, #11367: URL: https://github.com/apache/hudi/issues/11367 **Describe the problem you faced** When trying to use the [observe](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.observe.html) function on data

Re: [PR] [HUDI-7146] Implement secondary index write path [hudi]

2024-05-30 Thread via GitHub
codope merged PR #11146: URL: https://github.com/apache/hudi/pull/11146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

(hudi) branch master updated: [HUDI-7146] Implement secondary index write path (#11146)

2024-05-30 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new cd62c31f368 [HUDI-7146] Implement secondary index

Re: [PR] [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11365: URL: https://github.com/apache/hudi/pull/11365#issuecomment-2139068484 ## CI report: * 8147454d905761bd2256aac273ef69aa1e56fba8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7146] Integrate secondary index on reader path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11162: URL: https://github.com/apache/hudi/pull/11162#issuecomment-2139067855 ## CI report: * a602c9c4234062e66877fc4bf2c50f94f43767bc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT] Hudi Sink Connector shows broker disconnected [hudi]

2024-05-30 Thread via GitHub
prabodh1194 commented on issue #9070: URL: https://github.com/apache/hudi/issues/9070#issuecomment-2139020981 but still facing a bunch of issues in the java classpath. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline [hudi]

2024-05-30 Thread via GitHub
xuzifu666 commented on code in PR #11365: URL: https://github.com/apache/hudi/pull/11365#discussion_r1620230806 ## hudi-common/src/main/java/org/apache/hudi/common/util/ClusteringUtils.java: ## @@ -69,7 +69,7 @@ public class ClusteringUtils { public static Stream> getAllPend

Re: [PR] [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline [hudi]

2024-05-30 Thread via GitHub
danny0405 commented on code in PR #11365: URL: https://github.com/apache/hudi/pull/11365#discussion_r1620205175 ## hudi-common/src/main/java/org/apache/hudi/common/util/ClusteringUtils.java: ## @@ -69,7 +69,7 @@ public class ClusteringUtils { public static Stream> getAllPend

Re: [PR] [HUDI-7146] Implement secondary index write path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11146: URL: https://github.com/apache/hudi/pull/11146#issuecomment-2138926167 ## CI report: * 470bc5f44e7a6658a8717ef1b77e92afcdd90087 UNKNOWN * 43f73661f79eb87ac52d29fa153b996a15f29b99 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7007] Add bloom_filters index support on read side [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11043: URL: https://github.com/apache/hudi/pull/11043#issuecomment-2138904248 ## CI report: * 541b544049e68b3d22cdf0f5159fbd9b0005d345 UNKNOWN * 6ece7645a69b367901c71ab78dea15f39d69fca5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[I] [SUPPORT] CVE problems in latest 0.14.1 [hudi]

2024-05-30 Thread via GitHub
Smith-Cruise opened a new issue, #11366: URL: https://github.com/apache/hudi/issues/11366 CVE jars were introduced by `hudi-common`(in `hbase-server` and `hbase-client` transitive dependency) Could you let me know if the community plans to resolve these CVE dependencies? ```bash

Re: [PR] [HUDI-7815] Multiple writer with bulkinsert getAllPendingClusteringPlans should refresh timeline [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11365: URL: https://github.com/apache/hudi/pull/11365#issuecomment-2138822739 ## CI report: * 8147454d905761bd2256aac273ef69aa1e56fba8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7146] Integrate secondary index on reader path [hudi]

2024-05-30 Thread via GitHub
hudi-bot commented on PR #11162: URL: https://github.com/apache/hudi/pull/11162#issuecomment-2138822123 ## CI report: * 9d0e80222f6cc69b2dba6f4cdbfc642f31a95e52 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

  1   2   >