Re: [PR] [HUDI-8002] Added back Flink 1.15 and 1.14 bundle validation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11653: URL: https://github.com/apache/hudi/pull/11653#issuecomment-2244373389 ## CI report: * 4c71492db69e0b80a9a57ce3cd962a16d383fd0f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7948] RFC-80: Support column families for wide tables [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11559: URL: https://github.com/apache/hudi/pull/11559#issuecomment-2244373037 ## CI report: * cdd674f3c99d3b885b65a8983d6ad6b89603b464 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8002] Added back Flink 1.15 and 1.14 bundle validation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11653: URL: https://github.com/apache/hudi/pull/11653#issuecomment-2244360937 ## CI report: * 4c71492db69e0b80a9a57ce3cd962a16d383fd0f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7948] RFC-80: Support column families for wide tables [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11559: URL: https://github.com/apache/hudi/pull/11559#issuecomment-2244360526 ## CI report: * cdd674f3c99d3b885b65a8983d6ad6b89603b464 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-2955] Support Hadoop3 [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11572: URL: https://github.com/apache/hudi/pull/11572#issuecomment-2244349550 ## CI report: * 2639f581f20ab0b8fddf22d0fcfeb54f164ec346 UNKNOWN * f941907c2a63cd74800aed296468563215dc381b UNKNOWN * 2a4a27ff9ae126da6051ee7b2ad2594ae5a098a5 Azure: [FAIL

[jira] [Updated] (HUDI-7996) Store partition type with partition fields in table configs

2024-07-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7996: -- Description: This jira aims to store partition type as well in `hoodie.table.partition.fields`. Current

[jira] [Updated] (HUDI-8014) Fix Exception "FileID of partition path xxx=xx does not exist"

2024-07-22 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka updated HUDI-8014: Component/s: writer-core > Fix Exception "FileID of partition path xxx=xx does not exist" >

Re: [I] [SUPPORT]FileID of partition path xxx=xx does not exist. [hudi]

2024-07-22 Thread via GitHub
ad1happy2go commented on issue #11202: URL: https://github.com/apache/hudi/issues/11202#issuecomment-2244323842 Created tracking JIRA - https://issues.apache.org/jira/browse/HUDI-8014 cc @nishantpanwar18 -- This is an automated message from the Apache Git Service. To respond to the mes

[jira] [Created] (HUDI-8014) Fix Exception "FileID of partition path xxx=xx does not exist"

2024-07-22 Thread Aditya Goenka (Jira)
Aditya Goenka created HUDI-8014: --- Summary: Fix Exception "FileID of partition path xxx=xx does not exist" Key: HUDI-8014 URL: https://issues.apache.org/jira/browse/HUDI-8014 Project: Apache Hudi

Re: [PR] [HUDI-2955] Support Hadoop3 [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11572: URL: https://github.com/apache/hudi/pull/11572#issuecomment-2244296644 ## CI report: * 2639f581f20ab0b8fddf22d0fcfeb54f164ec346 UNKNOWN * f941907c2a63cd74800aed296468563215dc381b UNKNOWN * 0c412665c9dcf1c6642160f2d3ab8a5dd3aaff0b Azure: [FAIL

Re: [PR] [HUDI-8002] Added back Flink 1.15 and 1.14 bundle validation [hudi]

2024-07-22 Thread via GitHub
yihua commented on code in PR #11653: URL: https://github.com/apache/hudi/pull/11653#discussion_r1687406391 ## .github/workflows/bot.yml: ## @@ -511,6 +511,14 @@ jobs: flinkProfile: 'flink1.16' sparkProfile: 'spark3.3' sparkRuntime: 'spark3

Re: [PR] [HUDI-4625] Clean up KafkaOffsetGen: introduce retrying KafkaConsumer [hudi]

2024-07-22 Thread via GitHub
wombatu-kun commented on code in PR #11664: URL: https://github.com/apache/hudi/pull/11664#discussion_r1687364371 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/config/KafkaSourceConfig.java: ## @@ -66,14 +66,6 @@ public class KafkaSourceConfig extends HoodieConfig {

[PR] [DOCS] Fix typo in metadata.md [hudi]

2024-07-22 Thread via GitHub
paul8263 opened a new pull request, #11673: URL: https://github.com/apache/hudi/pull/11673 ### Change Logs Fix typo in metadata.md ### Impact No impact. ### Risk level (write none, low medium or high below) No risks. ### Documentation Update Fi

Re: [PR] [HUDI-8009] Optimize the code of HoodieTable#getPendingCommitTimeline [hudi]

2024-07-22 Thread via GitHub
usberkeley commented on code in PR #11665: URL: https://github.com/apache/hudi/pull/11665#discussion_r1687348238 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java: ## @@ -367,8 +367,8 @@ public HoodieTimeline getCompletedCommitTimeline() {

Re: [I] [SUPPORT] NoClassDefFoundError for org.apache.hudi.org.apache.hadoop.hbase.io.hfile.HFile [hudi]

2024-07-22 Thread via GitHub
wardlican commented on issue #8507: URL: https://github.com/apache/hudi/issues/8507#issuecomment-2244122922 I also noticed this issue with Hudi: 0.14.1 Hadoop: 3.2.2 Spark: 3.4.2 Hbase: 2.4.5 Does anyone have a solution? -- This is an automated message from the Apache Git Se

Re: [I] Enforce Python code style [hudi-rs]

2024-07-22 Thread via GitHub
muyihao commented on issue #5: URL: https://github.com/apache/hudi-rs/issues/5#issuecomment-2244098545 Hi, I'd like to work on this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[jira] [Created] (HUDI-8013) Test Plan for multi-modal index

2024-07-22 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-8013: - Summary: Test Plan for multi-modal index Key: HUDI-8013 URL: https://issues.apache.org/jira/browse/HUDI-8013 Project: Apache Hudi Issue Type: Sub-task

Re: [PR] [HUDI-4625] Clean up KafkaOffsetGen: introduce retrying KafkaConsumer [hudi]

2024-07-22 Thread via GitHub
danny0405 commented on code in PR #11664: URL: https://github.com/apache/hudi/pull/11664#discussion_r1687304605 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/config/KafkaSourceConfig.java: ## @@ -66,14 +66,6 @@ public class KafkaSourceConfig extends HoodieConfig {

Re: [I] [SUPPORT] multi writer insert overwrite different partition conflict resolve stragegy throw StringIndexOutOfBoundsException [hudi]

2024-07-22 Thread via GitHub
danny0405 commented on issue #11516: URL: https://github.com/apache/hudi/issues/11516#issuecomment-2244063087 Based on the variable name `compactionAndClusteringPendingTimeline`, it looks like your fix is reasonable. -- This is an automated message from the Apache Git Service. To respond

[jira] [Closed] (HUDI-8006) glue sync may not update columns

2024-07-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8006. Resolution: Fixed Fixed via master branch: a0bae464592f44cfa7463d9d09aa7235fd63b5fe > glue sync may not upd

[jira] [Updated] (HUDI-8006) glue sync may not update columns

2024-07-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8006: - Fix Version/s: 1.1.0 > glue sync may not update columns > - > >

(hudi) branch master updated: [HUDI-8006] Fix glue sync async call to update table (#11659)

2024-07-22 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new a0bae464592 [HUDI-8006] Fix glue sync async cal

Re: [PR] [HUDI-8006] Fix glue sync async call to update table [hudi]

2024-07-22 Thread via GitHub
danny0405 merged PR #11659: URL: https://github.com/apache/hudi/pull/11659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

(hudi) branch master updated: [HUDI-8008] Handle proto schema references in returned schema (#11660)

2024-07-22 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new ccb8f1d6945 [HUDI-8008] Handle proto schema ref

Re: [I] Exception while using HoodieStreamer protobuf data from Kafka [hudi]

2024-07-22 Thread via GitHub
danny0405 closed issue #11598: Exception while using HoodieStreamer protobuf data from Kafka URL: https://github.com/apache/hudi/issues/11598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] [HUDI-8008] Handle proto schema references in returned schema [hudi]

2024-07-22 Thread via GitHub
danny0405 merged PR #11660: URL: https://github.com/apache/hudi/pull/11660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

Re: [PR] [HUDI-8009] Optimize the code of HoodieTable#getPendingCommitTimeline [hudi]

2024-07-22 Thread via GitHub
danny0405 commented on code in PR #11665: URL: https://github.com/apache/hudi/pull/11665#discussion_r1687287842 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java: ## @@ -367,8 +367,8 @@ public HoodieTimeline getCompletedCommitTimeline() {

Re: [PR] [MINOR] Fix SyntaxWarning in pr_compliance.py [hudi]

2024-07-22 Thread via GitHub
yihua merged PR #11669: URL: https://github.com/apache/hudi/pull/11669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

(hudi) branch master updated (ad10bd4e040 -> edb1c758d8e)

2024-07-22 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from ad10bd4e040 [HUDI-712] Revert back to overwrite=true in HoodieSnapshotExporter (#11668) add edb1c758d8e [MINOR] Fix

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2244035061 ## CI report: * 34906c1f0f07e3ee01a9daaca5150409cf378b2b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2243976667 ## CI report: * 89d0beac2b61fb5502a93d62239c9c978686e534 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8011] allow schema.on.read with positional merging [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11671: URL: https://github.com/apache/hudi/pull/11671#issuecomment-2243934242 ## CI report: * 01ae79a7921216b27154e145a19975ec5f2583cf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2243934362 ## CI report: * 89d0beac2b61fb5502a93d62239c9c978686e534 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2243923034 ## CI report: * 89d0beac2b61fb5502a93d62239c9c978686e534 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2243914542 ## CI report: * 89d0beac2b61fb5502a93d62239c9c978686e534 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243914473 ## CI report: * 8b9c108b939ae4d983581d327a41f7729d6a8896 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8006] Fix glue sync async call to update table [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11659: URL: https://github.com/apache/hudi/pull/11659#issuecomment-2243914393 ## CI report: * 797bff381ffdd1420929e38896fff6a56ae9e566 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8011] allow schema.on.read with positional merging [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11671: URL: https://github.com/apache/hudi/pull/11671#issuecomment-2243863295 ## CI report: * 01ae79a7921216b27154e145a19975ec5f2583cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11672: URL: https://github.com/apache/hudi/pull/11672#issuecomment-2243863343 ## CI report: * 89d0beac2b61fb5502a93d62239c9c978686e534 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-8012: Status: Patch Available (was: In Progress) > Update checkstyle.xml based on the new release > -

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-8012: Status: In Progress (was: Open) > Update checkstyle.xml based on the new release >

Re: [PR] [HUDI-8011] allow schema.on.read with positional merging [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11671: URL: https://github.com/apache/hudi/pull/11671#issuecomment-2243852369 ## CI report: * 01ae79a7921216b27154e145a19975ec5f2583cf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-8012: Description: checkstyle.xml works with older checkstyle release, 8.20, only.  We need to make it work with

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-8012: Description: checkstyle.xml works with older checkstyle release, 8.20, only.  We need to make it work with r

[jira] [Assigned] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-8012: --- Assignee: Ethan Guo > Update checkstyle.xml based on the new release > --

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-8012: Description: I > Update checkstyle.xml based on the new release > --

[jira] [Updated] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8012: - Labels: pull-request-available (was: ) > Update checkstyle.xml based on the new release > ---

[PR] [HUDI-8012] Update checkstyle.xml based on the new release [hudi]

2024-07-22 Thread via GitHub
yihua opened a new pull request, #11672: URL: https://github.com/apache/hudi/pull/11672 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[jira] [Created] (HUDI-8012) Update checkstyle.xml based on the new release

2024-07-22 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-8012: --- Summary: Update checkstyle.xml based on the new release Key: HUDI-8012 URL: https://issues.apache.org/jira/browse/HUDI-8012 Project: Apache Hudi Issue Type: New Featur

Re: [PR] Fix SyntaxWarning in pr_compliance.py [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11669: URL: https://github.com/apache/hudi/pull/11669#issuecomment-2243840602 ## CI report: * 417132345ecc34fa322b9b314a0335bb1c8891b9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8006] Fix glue sync async call to update table [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11659: URL: https://github.com/apache/hudi/pull/11659#issuecomment-2243840486 ## CI report: * b14f4b40f87e4b85fd8f6d64b3652926dc23da50 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Updated] (HUDI-8011) Allow schema.on.read with positional merging

2024-07-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-8011: -- Status: Patch Available (was: In Progress) > Allow schema.on.read with positional merging > ---

[jira] [Updated] (HUDI-8011) Allow schema.on.read with positional merging

2024-07-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-8011: -- Status: In Progress (was: Open) > Allow schema.on.read with positional merging > --

[jira] [Updated] (HUDI-8011) Allow schema.on.read with positional merging

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8011: - Labels: pull-request-available (was: ) > Allow schema.on.read with positional merging > -

[PR] [HUDI-8011] allow schema.on.read with positional merging [hudi]

2024-07-22 Thread via GitHub
jonvex opened a new pull request, #11671: URL: https://github.com/apache/hudi/pull/11671 ### Change Logs Add internalschemaopt as param to the hoodie spark readers. add the positional merge column to the internal schema when positional merging could be used. Need to stor

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687133956 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

[jira] [Created] (HUDI-8011) Allow schema.on.read with positional merging

2024-07-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-8011: - Summary: Allow schema.on.read with positional merging Key: HUDI-8011 URL: https://issues.apache.org/jira/browse/HUDI-8011 Project: Apache Hudi Issue Type:

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687119981 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243778161 ## CI report: * 5b907b18a1718ecb33d2e00cd747d77b20efd17b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-6453] Cascade Glue schema changes to partitions [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11670: URL: https://github.com/apache/hudi/pull/11670#issuecomment-2243778217 ## CI report: * 87b0a55f5fc6f1616b0c450992045c050d81b59f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8006] Fix glue sync async call to update table [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11659: URL: https://github.com/apache/hudi/pull/11659#issuecomment-2243778053 ## CI report: * b14f4b40f87e4b85fd8f6d64b3652926dc23da50 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687116969 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243765016 ## CI report: * 5b907b18a1718ecb33d2e00cd747d77b20efd17b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687110917 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687110393 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-6453] Cascade Glue schema changes to partitions [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11670: URL: https://github.com/apache/hudi/pull/11670#issuecomment-2243765104 ## CI report: * 87b0a55f5fc6f1616b0c450992045c050d81b59f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687107980 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [I] [SUPPORT] HoodieSnapshotExporter is broken with PR HUDI-712 [hudi]

2024-07-22 Thread via GitHub
xushiyan closed issue #11640: [SUPPORT] HoodieSnapshotExporter is broken with PR HUDI-712 URL: https://github.com/apache/hudi/issues/11640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [HUDI-7948] RFC-80: Support column families for wide tables [hudi]

2024-07-22 Thread via GitHub
vinothchandar commented on code in PR #11559: URL: https://github.com/apache/hudi/pull/11559#discussion_r1687076827 ## rfc/rfc-80/rfc-80.md: ## @@ -0,0 +1,173 @@ + +# RFC-80: Support column families for wide tbles + +## Proposers + +- @xiarixiaoyao +- @wombatu-kun + +## Approver

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687094555 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

(hudi) branch master updated: [HUDI-712] Revert back to overwrite=true in HoodieSnapshotExporter (#11668)

2024-07-22 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new ad10bd4e040 [HUDI-712] Revert back to overwrite=

Re: [PR] [HUDI-712] Revert back to overwrite=true in HoodieSnapshotExporter [hudi]

2024-07-22 Thread via GitHub
xushiyan merged PR #11668: URL: https://github.com/apache/hudi/pull/11668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687091052 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687089711 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [I] [SUPPORT] Use Case Validation for CDC Ingestion from Kafka/Debezium (SQL Server) into Apache Hudi [hudi]

2024-07-22 Thread via GitHub
YousifS7 commented on issue #11662: URL: https://github.com/apache/hudi/issues/11662#issuecomment-2243730494 Hi @ad1happy2go, Thank you for getting back to me. I appreciate the confirmation. We will perform few tests and will post back if there are any issues. -- This is an automat

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687081729 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687080897 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

Re: [PR] [HUDI-7882][WIP] Adding RFC 78 for bridge release to assist users to migrate to 1.x from 0.x [hudi]

2024-07-22 Thread via GitHub
nsivabalan commented on code in PR #11514: URL: https://github.com/apache/hudi/pull/11514#discussion_r1687080177 ## rfc/rfc-78/rfc-78.md: ## @@ -0,0 +1,339 @@ + +# RFC-76: [Bridge release for 1.x] + +## Proposers + +- @nsivabalan +- @vbalaji + +## Approvers + - @yihua + - @codop

[PR] [HUDI-6453] Cascade Glue schema changes to partitions [hudi]

2024-07-22 Thread via GitHub
parisni opened a new pull request, #11670: URL: https://github.com/apache/hudi/pull/11670 ### Change Logs This PR addresses ongoing issue with glue and athena/redshift-spectrum query engines: currently in some schema evolution scenario (see below), the modified columns can't be r

Re: [PR] Fix SyntaxWarning in pr_compliance.py [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11669: URL: https://github.com/apache/hudi/pull/11669#issuecomment-2243684083 ## CI report: * 417132345ecc34fa322b9b314a0335bb1c8891b9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243684008 ## CI report: * 5b907b18a1718ecb33d2e00cd747d77b20efd17b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243670987 ## CI report: * 020641e0c7f323b69b4a23da7f457573c2d2160b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] Fix SyntaxWarning in pr_compliance.py [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11669: URL: https://github.com/apache/hudi/pull/11669#issuecomment-2243671080 ## CI report: * 417132345ecc34fa322b9b314a0335bb1c8891b9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-6558] support SQL update for no-precombine field tables [hudi]

2024-07-22 Thread via GitHub
kazdy closed pull request #9277: [HUDI-6558] support SQL update for no-precombine field tables URL: https://github.com/apache/hudi/pull/9277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[PR] Fix SyntaxWarning in pr_compliance.py [hudi]

2024-07-22 Thread via GitHub
Gatsby-Lee opened a new pull request, #11669: URL: https://github.com/apache/hudi/pull/11669 ### Change Logs The regex for the title validation doesn't follow the syntax. So, this PR updates the regex string to use raw string ### Impact no more SyntaxWarning in pr_comp

Re: [PR] Revert back to overwrite=true in HoodieSnapshotExporter [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11668: URL: https://github.com/apache/hudi/pull/11668#issuecomment-2243585727 ## CI report: * 5b7c982dfa17db6d7a3bdc5550871e037897557c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243585660 ## CI report: * 020641e0c7f323b69b4a23da7f457573c2d2160b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] Revert back to overwrite=true in HoodieSnapshotExporter [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11668: URL: https://github.com/apache/hudi/pull/11668#issuecomment-2243569827 ## CI report: * 5b7c982dfa17db6d7a3bdc5550871e037897557c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11667: URL: https://github.com/apache/hudi/pull/11667#issuecomment-2243569746 ## CI report: * 020641e0c7f323b69b4a23da7f457573c2d2160b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[PR] Revert back to overwrite=true in HoodieSnapshotExporter [hudi]

2024-07-22 Thread via GitHub
Gatsby-Lee opened a new pull request, #11668: URL: https://github.com/apache/hudi/pull/11668 ### Change Logs The discussion about this change has been done with @xushiyan. ( https://github.com/apache/hudi/issues/11640 ) The HoodieSnapshotExporter was updated with overwrite=fals

Re: [I] [SUPPORT] HoodieSnapshotExporter is broken with PR HUDI-712 [hudi]

2024-07-22 Thread via GitHub
Gatsby-Lee commented on issue #11640: URL: https://github.com/apache/hudi/issues/11640#issuecomment-2243514273 @xushiyan @ad1happy2go Thank you. I made a PR. ( https://github.com/apache/hudi/pull/11668/files ) It would be great if this PR can be applied to 0.15.1 -- This is an au

[jira] [Assigned] (HUDI-8010) Avoid creating zero row group parquet file

2024-07-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain reassigned HUDI-8010: - Assignee: (was: Lokesh Jain) > Avoid creating zero row group parquet file > -

[jira] [Created] (HUDI-8010) Avoid creating zero row group parquet file

2024-07-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8010: - Summary: Avoid creating zero row group parquet file Key: HUDI-8010 URL: https://issues.apache.org/jira/browse/HUDI-8010 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-8005) New lock provider implementation

2024-07-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-8005: - Labels: pull-request-available (was: ) > New lock provider implementation > -

[PR] [HUDI-8005] Add new lock provider implementation [hudi]

2024-07-22 Thread via GitHub
Davis-Zhang-Onehouse opened a new pull request, #11667: URL: https://github.com/apache/hudi/pull/11667 OSS jira: https://issues.apache.org/jira/browse/HUDI-8005 OSS PR: https://github.com/apache/hudi/compare/master...Davis-Zhang-Onehouse:hudi-oss:hudi-8005 The change adds a new loc

[jira] [Assigned] (HUDI-7952) Incorrect partition pruning when TimestampBasedKeyGenerator is used in partition column

2024-07-22 Thread Geser Dugarov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geser Dugarov reassigned HUDI-7952: --- Assignee: (was: Geser Dugarov) > Incorrect partition pruning when TimestampBasedKeyGenera

Re: [I] Configure pre commit hook to format code style [hudi-rs]

2024-07-22 Thread via GitHub
xushiyan commented on issue #73: URL: https://github.com/apache/hudi-rs/issues/73#issuecomment-2243380742 @muyihao Pls comment on the issue so i can assign you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] [HUDI-4625] Clean up KafkaOffsetGen: introduce retrying KafkaConsumer [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11664: URL: https://github.com/apache/hudi/pull/11664#issuecomment-2243236039 ## CI report: * 3a7a1fcdf3e135165512c2f8da582949a68a12f3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT] HoodieSnapshotExporter is broken with PR HUDI-712 [hudi]

2024-07-22 Thread via GitHub
xushiyan commented on issue #11640: URL: https://github.com/apache/hudi/issues/11640#issuecomment-2243195056 @Gatsby-Lee yup sg -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [HUDI-8009] Optimize the code of HoodieTable#getPendingCommitTimeline [hudi]

2024-07-22 Thread via GitHub
hudi-bot commented on PR #11665: URL: https://github.com/apache/hudi/pull/11665#issuecomment-2243138412 ## CI report: * 3e1f7c4ca8955aa93401fbdc1a9bf4a8eb3dcb12 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

  1   2   >