[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
hudi-bot commented on PR #8542: URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518514660 ## CI report: * ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
hudi-bot commented on PR #8542: URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518513191 ## CI report: * ccd692493bc1e1669f521bc1368324fa3fb7032f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8537: [HUDI-6122] Unify options in call procedure
hudi-bot commented on PR #8537: URL: https://github.com/apache/hudi/pull/8537#issuecomment-1518511826 ## CI report: * 6720828d44070be1152eb035d7a59412e5c4925f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16531) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
hudi-bot commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518511662 ## CI report: * d99a80f1654e96f474fc2a047e38bf232a0bed81 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16527) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16546) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16548) * 86db1f4f7be8f8b8dcd56e13bd945977fc804591 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16549) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 opened a new pull request, #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
danny0405 opened a new pull request, #8542: URL: https://github.com/apache/hudi/pull/8542 ### Change Logs The LockManager#lock is invoked from the TransactionManager which already takes the check for the requirement of a explicit lock. ### Impact none ### Risk level (write none, low medium or high below) none ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
hudi-bot commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518503545 ## CI report: * d99a80f1654e96f474fc2a047e38bf232a0bed81 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16527) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16546) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16548) * 86db1f4f7be8f8b8dcd56e13bd945977fc804591 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518503512 ## CI report: * 6d86ef170bc1b9cb1f120cb665a8999008e7cf03 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16539) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8484: [HUDI-6092] Reuse schema objects while deserializing log blocks.
danny0405 commented on PR #8484: URL: https://github.com/apache/hudi/pull/8484#issuecomment-1518503019 The test is flaky: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=16505=logs=3b6e910d-b98f-5de6-b9cb-1e5ff571f5de=30b5aae4-0ea0-5566-42d0-febf71a7061a=717331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
hudi-bot commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518501973 ## CI report: * d99a80f1654e96f474fc2a047e38bf232a0bed81 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16527) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16546) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16548) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8506: [HUDI-6104] Clean deleted partition with clean policy
hudi-bot commented on PR #8506: URL: https://github.com/apache/hudi/pull/8506#issuecomment-1518502098 ## CI report: * 8ec5618449900b90ce465037700b129c9d51cde4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16478) * 1753392a8f1cbc9370ce082c6a803317ab4eef71 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16547) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518501940 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16544) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16545) * 6d86ef170bc1b9cb1f120cb665a8999008e7cf03 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] huangxiaopingRD commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
huangxiaopingRD commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518501695 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8506: [HUDI-6104] Clean deleted partition with clean policy
hudi-bot commented on PR #8506: URL: https://github.com/apache/hudi/pull/8506#issuecomment-1518500230 ## CI report: * 8ec5618449900b90ce465037700b129c9d51cde4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16478) * 1753392a8f1cbc9370ce082c6a803317ab4eef71 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
hudi-bot commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518500145 ## CI report: * d99a80f1654e96f474fc2a047e38bf232a0bed81 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16527) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16546) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] huangxiaopingRD commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
huangxiaopingRD commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518494041 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] xuzifu666 opened a new issue, #8541: [SUPPORT] write to a mor table 12times, but no compaction instance with default configuration
xuzifu666 opened a new issue, #8541: URL: https://github.com/apache/hudi/issues/8541 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subscr...@hudi.apache.org. - If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly. **Describe the problem you faced** A clear and concise description of the problem. **To Reproduce** Steps to reproduce the behavior: 1. 2. 3. 4. **Expected behavior** A clear and concise description of what you expected to happen. **Environment Description** * Hudi version : * Spark version : * Hive version : * Hadoop version : * Storage (HDFS/S3/GCS..) : * Running on Docker? (yes/no) : **Additional context** Add any other context about the problem here. **Stacktrace** ```Add the stacktrace of the error.``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518490750 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16544) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16545) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wkhappy1 commented on issue #8485: [SUPPORT] hudi connector partition filter only support equal
wkhappy1 commented on issue #8485: URL: https://github.com/apache/hudi/issues/8485#issuecomment-1518488898 Can I, if I can, I will do my best -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518486654 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16544) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16545) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 closed issue #8533: [SUPPORT] Can hoodie.compact.inline config been used in Flink SQL WITH clause?
danny0405 closed issue #8533: [SUPPORT] Can hoodie.compact.inline config been used in Flink SQL WITH clause? URL: https://github.com/apache/hudi/issues/8533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #8533: [SUPPORT] Can hoodie.compact.inline config been used in Flink SQL WITH clause?
danny0405 commented on issue #8533: URL: https://github.com/apache/hudi/issues/8533#issuecomment-1518485125 No, Flink only support async compaction online or separate compaction job with batch execution mode. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518484955 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8512: [HUDI-6057] Support Flink 1.17
danny0405 commented on PR #8512: URL: https://github.com/apache/hudi/pull/8512#issuecomment-1518484494 Thanks for the contribution, reviewing now ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #8485: [SUPPORT] hudi connector partition filter only support equal
danny0405 commented on issue #8485: URL: https://github.com/apache/hudi/issues/8485#issuecomment-1518483597 > in trino,HudiUtil.java seems like do the same thing . do we need copy the same logic Nice findings, can you fire a PR to fix that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HUDI-6119) Add hudi timeline timezone testcase
[ https://issues.apache.org/jira/browse/HUDI-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6119: - Fix Version/s: 0.14.0 > Add hudi timeline timezone testcase > --- > > Key: HUDI-6119 > URL: https://issues.apache.org/jira/browse/HUDI-6119 > Project: Apache Hudi > Issue Type: Test >Reporter: inki hwang >Priority: Minor > Labels: pull-request-available > Fix For: 0.14.0 > > > related to https://issues.apache.org/jira/browse/HUDI-5978 > Add timeline timezone testcase for UTC -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (HUDI-6119) Add hudi timeline timezone testcase
[ https://issues.apache.org/jira/browse/HUDI-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6119. Resolution: Fixed > Add hudi timeline timezone testcase > --- > > Key: HUDI-6119 > URL: https://issues.apache.org/jira/browse/HUDI-6119 > Project: Apache Hudi > Issue Type: Test >Reporter: inki hwang >Priority: Minor > Labels: pull-request-available > Fix For: 0.14.0 > > > related to https://issues.apache.org/jira/browse/HUDI-5978 > Add timeline timezone testcase for UTC -- This message was sent by Atlassian Jira (v8.20.10#820010)
[hudi] branch master updated (dc074666718 -> 99eabd562c5)
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from dc074666718 [HUDI-6091] Add Java 11 and 17 to bundle validation (#8481) add 99eabd562c5 [HUDI-6119] Add testcase for timeline timezone UTC (#8528) No new revisions were added by this update. Summary of changes: .../org/apache/hudi/TestHoodieSparkSqlWriter.scala | 48 +- 1 file changed, 46 insertions(+), 2 deletions(-)
[GitHub] [hudi] danny0405 merged pull request #8528: [HUDI-6119] Add testcase for timeline timezone UTC
danny0405 merged PR #8528: URL: https://github.com/apache/hudi/pull/8528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518482969 ## CI report: * aadc218154cc2a80344fee1e18c02dd3f19ed4f0 UNKNOWN * 5ea64f04d9d1e008cef03e3c92e22f02ab961ae8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16538) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518472508 ## CI report: * 6d86ef170bc1b9cb1f120cb665a8999008e7cf03 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16539) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518470550 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16541) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16540) * 6d86ef170bc1b9cb1f120cb665a8999008e7cf03 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wkhappy1 commented on issue #8485: [SUPPORT] hudi connector partition filter only support equal
wkhappy1 commented on issue #8485: URL: https://github.com/apache/hudi/issues/8485#issuecomment-1518469373 in trino,HudiUtil.java seems like do the same thing . do we need copy the same logic。Direct copy, my idea is not good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wkhappy1 commented on issue #8485: [SUPPORT] hudi connector partition filter only support equal
wkhappy1 commented on issue #8485: URL: https://github.com/apache/hudi/issues/8485#issuecomment-1518463105 i see a pull request https://github.com/prestodb/presto/pull/18606/files,it also filter partition after getting partition -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8488: [HUDI-5957] Fix table not exist when using 'db.table' in createHoodieClientFromPath
hudi-bot commented on PR #8488: URL: https://github.com/apache/hudi/pull/8488#issuecomment-1518454988 ## CI report: * e151236bbe169543f582096f2cc724cc057bdbb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16515) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16533) * d242b57634551d7d4dcdd37a810613c178760232 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16542) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8488: [HUDI-5957] Fix table not exist when using 'db.table' in createHoodieClientFromPath
hudi-bot commented on PR #8488: URL: https://github.com/apache/hudi/pull/8488#issuecomment-1518453139 ## CI report: * e151236bbe169543f582096f2cc724cc057bdbb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16515) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16533) * d242b57634551d7d4dcdd37a810613c178760232 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518450892 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16541) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16540) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518437898 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16540) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16541) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518435060 ## CI report: * b33e8f41d6206519e849a178f99e5d1972d90387 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
yihua commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518431864 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8539: [HUDI-6067] Auto capitalize enum configs
hudi-bot commented on PR #8539: URL: https://github.com/apache/hudi/pull/8539#issuecomment-1518431337 ## CI report: * 343ab147919cb79a5b3e2f44028afb065b63f9d2 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16535) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8538: [HUDI-6062] Standardize irregular enum configs
hudi-bot commented on PR #8538: URL: https://github.com/apache/hudi/pull/8538#issuecomment-1518431318 ## CI report: * dd36f97fb00d7825f64a8cc40122ff6309a90035 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16534) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8488: [HUDI-5957] Fix table not exist when using 'db.table' in createHoodieClientFromPath
hudi-bot commented on PR #8488: URL: https://github.com/apache/hudi/pull/8488#issuecomment-1518431185 ## CI report: * e151236bbe169543f582096f2cc724cc057bdbb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16515) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16533) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
hudi-bot commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518430937 ## CI report: * 6d86ef170bc1b9cb1f120cb665a8999008e7cf03 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16480) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16539) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7673: Bump mybatis from 3.4.6 to 3.5.6 in /hudi-platform-service/hudi-metaserver/hudi-metaserver-server
hudi-bot commented on PR #7673: URL: https://github.com/apache/hudi/pull/7673#issuecomment-1518430427 ## CI report: * caf261ae75fd5eb3c900593b43493fef8488f0d7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16522) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8319: [HUDI-5934] Remove archival configs for metadata table
yihua commented on PR #8319: URL: https://github.com/apache/hudi/pull/8319#issuecomment-1518430572 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] easonwood commented on issue #8040: [SUPPORT] Getting error when writing into MOR HUDI table if schema changed (datatype changed / column dropped)
easonwood commented on issue #8040: URL: https://github.com/apache/hudi/issues/8040#issuecomment-1518427810 https://github.com/apache/hudi/issues/8540 Hi, Can we check this similar ticket ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] easonwood opened a new issue, #8540: [SUPPORT] Getting error when writing into COW HUDI table if schema changed (datatype changed / column dropped)
easonwood opened a new issue, #8540:
URL: https://github.com/apache/hudi/issues/8540
**Describe the problem you faced**
We have a hudi table created in 20221130, and we did column change by adding
some columns.
Now when we do DELETE on whole partitions, it will get error
**To Reproduce**
Steps to reproduce the behavior:
1. We have old data ::
>>> old_df =
spark.read.parquet("s3://our-bucket/data-lake-hudi/ods_meeting_mmr_log/ods_mmr_conf_join/cluster=aw1/year=2022/month=11/day=30")
>>> old_df.printSchema()
root
|-- _hoodie_commit_time: string (nullable = true)
|-- _hoodie_commit_seqno: string (nullable = true)
|-- _hoodie_record_key: string (nullable = true)
|-- _hoodie_partition_path: string (nullable = true)
|-- _hoodie_file_name: string (nullable = true)
|-- _hoodie_is_deleted: boolean (nullable = true)
|-- account_id: string (nullable = true)
|-- cluster: string (nullable = true)
|-- day: string (nullable = true)
|-- error_fields: string (nullable = true)
|-- host_name: string (nullable = true)
|-- instance_id: string (nullable = true)
|-- log_time: string (nullable = true)
|-- meeting_uuid: string (nullable = true)
|-- month: string (nullable = true)
|-- region: string (nullable = true)
|-- rowkey: string (nullable = true)
|-- year: string (nullable = true)
|-- mmr_addr: string (nullable = true)
|-- top_mmr_addr: string (nullable = true)
|-- zone_name: string (nullable = true)
|-- client_addr: string (nullable = true)
|-- user_id: long (nullable = true)
|-- user_type: long (nullable = true)
|-- result: long (nullable = true)
|-- failover: long (nullable = true)
|-- name: string (nullable = true)
|-- viewonly_user: long (nullable = true)
|-- db_user_id: string (nullable = true)
|-- host_account: string (nullable = true)
|-- time_cost: long (nullable = true)
|-- device_id: string (nullable = true)
|-- index: long (nullable = true)
|-- persist_index: long (nullable = true)
|-- email: string (nullable = true)
|-- cluster_info: string (nullable = true)
|-- kms_enable: long (nullable = true)
|-- need_holding: long (nullable = true)
|-- not_guest: long (nullable = true)
|-- need_hide_pii: long (nullable = true)
|-- acct_id: string (nullable = true)
**|-- rest_fields: string (nullable = true)**
2. >>> new_df =
spark.read.parquet("s3://our-bucket/data-lake-hudi/ods_meeting_mmr_log/ods_mmr_conf_join/cluster=aw1/year=2023/month=04/day=16")
>>> new_df.printSchema()
root
|-- _hoodie_commit_time: string (nullable = true)
|-- _hoodie_commit_seqno: string (nullable = true)
|-- _hoodie_record_key: string (nullable = true)
|-- _hoodie_partition_path: string (nullable = true)
|-- _hoodie_file_name: string (nullable = true)
|-- _hoodie_is_deleted: boolean (nullable = true)
|-- account_id: string (nullable = true)
|-- cluster: string (nullable = true)
|-- day: string (nullable = true)
|-- error_fields: string (nullable = true)
|-- host_name: string (nullable = true)
|-- instance_id: string (nullable = true)
|-- log_time: string (nullable = true)
|-- meeting_uuid: string (nullable = true)
|-- month: string (nullable = true)
|-- region: string (nullable = true)
|-- rowkey: string (nullable = true)
|-- year: string (nullable = true)
|-- mmr_addr: string (nullable = true)
|-- top_mmr_addr: string (nullable = true)
|-- zone_name: string (nullable = true)
|-- client_addr: string (nullable = true)
|-- user_id: long (nullable = true)
|-- user_type: long (nullable = true)
|-- result: long (nullable = true)
|-- failover: long (nullable = true)
|-- name: string (nullable = true)
|-- viewonly_user: long (nullable = true)
|-- db_user_id: string (nullable = true)
|-- host_account: string (nullable = true)
|-- time_cost: long (nullable = true)
|-- device_id: string (nullable = true)
|-- index: long (nullable = true)
|-- persist_index: long (nullable = true)
|-- email: string (nullable = true)
|-- cluster_info: string (nullable = true)
|-- kms_enable: long (nullable = true)
|-- need_holding: long (nullable = true)
|-- not_guest: long (nullable = true)
|-- need_hide_pii: long (nullable = true)
|-- acct_id: string (nullable = true)
**|-- customer_key: string (nullable = true)
|-- phone_id: long (nullable = true)
|-- bind_number: long (nullable = true)
|-- tracking_id: string (nullable = true)
|-- reg_user_id: string (nullable = true)
|-- user_sn: string (nullable = true)
|-- gateways: string (nullable = true)
|-- jid: string (nullable = true)
|-- options2: string (nullable = true)
|-- attendee_option: long (nullable = true)
|-- is_user_failover:
[GitHub] [hudi] hudi-bot commented on pull request #7627: [HUDI-5517] HoodieTimeline support filter instants by state transition time
hudi-bot commented on PR #7627: URL: https://github.com/apache/hudi/pull/7627#issuecomment-1518412070 ## CI report: * 85b25f5cda4ccd8189a1607259e1732a910c3262 UNKNOWN * ef5fad3612bc910902d52de0296ef4c8852e5319 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16523) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #8319: [HUDI-5934] Remove archival configs for metadata table
yihua commented on code in PR #8319:
URL: https://github.com/apache/hudi/pull/8319#discussion_r1174213439
##
hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/TestGcsEventsHoodieIncrSource.java:
##
@@ -215,7 +215,7 @@ private HoodieRecord getGcsMetadataRecord(String
commitTime, String filename, St
private HoodieWriteConfig getWriteConfig() {
return getConfigBuilder(basePath(), metaClient)
-
.withArchivalConfig(HoodieArchivalConfig.newBuilder().archiveCommitsWith(2,
3).build())
+
.withArchivalConfig(HoodieArchivalConfig.newBuilder().archiveCommitsWith(4,
5).build())
.withCleanConfig(HoodieCleanConfig.newBuilder().retainCommits(1).build())
Review Comment:
There is no conflict in the production code, where 20 and 30 are set for min
and max commits for archival. And there's a reason to keep a few versions for
MDT from cleaning. It's only in some tests that we take shortcuts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters
hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1518384718 ## CI report: * 52af3eccbadac900b0cce37e8a281ad332d524f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16524) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8534: [MINOR] Fix misleading permitted write operation value
hudi-bot commented on PR #8534: URL: https://github.com/apache/hudi/pull/8534#issuecomment-1518371722 ## CI report: * 0dfb2388ac4261298870c6753dc0873938a797f9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16526) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #8535: [SUPPORT] manually deleting file under .hoodie/archived
nsivabalan commented on issue #8535: URL: https://github.com/apache/hudi/issues/8535#issuecomment-1518366401 files in archived are generally for book-keeping purposes. just incase you wish to go back in time and investigate what happened. but if you really dont' need them, you can delete them. you can sort based on last mod time and delete all files except the latest 50 to 100 files may be in the archived folder. I assume you have accrued 1000 or more files and wish to delete them. we do want to add automatic support to delete them if not required. but did not get a chance to add the support -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518343154 ## CI report: * c74e2758bbf78ada461819d45690debe67b31548 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16537) * aadc218154cc2a80344fee1e18c02dd3f19ed4f0 UNKNOWN * 5ea64f04d9d1e008cef03e3c92e22f02ab961ae8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16538) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518330745 ## CI report: * c74e2758bbf78ada461819d45690debe67b31548 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16537) * aadc218154cc2a80344fee1e18c02dd3f19ed4f0 UNKNOWN * 5ea64f04d9d1e008cef03e3c92e22f02ab961ae8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8355: [HUDI-6016] HoodieCLIUtils supports creating HoodieClient with non-default database
hudi-bot commented on PR #8355: URL: https://github.com/apache/hudi/pull/8355#issuecomment-1518330336 ## CI report: * d99a80f1654e96f474fc2a047e38bf232a0bed81 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16527) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518293432 ## CI report: * 7c8f2c661ca7a9f786b780ddcbbfa2aae67c0c95 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16462) * c74e2758bbf78ada461819d45690debe67b31548 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16537) * aadc218154cc2a80344fee1e18c02dd3f19ed4f0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518286839 ## CI report: * 7c8f2c661ca7a9f786b780ddcbbfa2aae67c0c95 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16462) * c74e2758bbf78ada461819d45690debe67b31548 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16537) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8529: [HUDI-6120]filter base file when there is only one file slice fetched
hudi-bot commented on PR #8529: URL: https://github.com/apache/hudi/pull/8529#issuecomment-1518279239 ## CI report: * ef1923c2ed2ef78ff2bae95ad3ac16e8ccf4ae97 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16528) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
hudi-bot commented on PR #8480: URL: https://github.com/apache/hudi/pull/8480#issuecomment-1518279027 ## CI report: * 7c8f2c661ca7a9f786b780ddcbbfa2aae67c0c95 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16462) * c74e2758bbf78ada461819d45690debe67b31548 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] neeruks commented on issue #6341: [SUPPORT] Hudi delete not working via spark apis
neeruks commented on issue #6341: URL: https://github.com/apache/hudi/issues/6341#issuecomment-1518260595 I have the same issue. I am trying to delete records from the Hudi table, but it is not happening. I am using the following EMR configuration EMR -5.33.0 , Spark 2.4.7 and Hudi 0.7.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8493: [HUDI-6098] Use bulk insert prepped for the initial write into MDT.
hudi-bot commented on PR #8493: URL: https://github.com/apache/hudi/pull/8493#issuecomment-1518236949 ## CI report: * 6d9d24f2e0ab70b97fb912505f2d0da60dfea86f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16507) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16529) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] gtwuser commented on issue #8532: [SUPPORT]org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 11 partition 1
gtwuser commented on issue #8532: URL: https://github.com/apache/hudi/issues/8532#issuecomment-1518197562 @ad1happy2go Yes its consistent. Sorry since im bit new to spark, if possible can you share any link or steps to setup spark ui and how to view it. :slightly_smiling_face: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua merged pull request #8481: [HUDI-6091] Add Java 11 and 17 to bundle validation
yihua merged PR #8481: URL: https://github.com/apache/hudi/pull/8481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[hudi] branch master updated (52efac958f2 -> dc074666718)
This is an automated email from the ASF dual-hosted git repository.
yihua pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 52efac958f2 [HUDI-6039] Fixing FS based listing for full cleaning in
clean Planner (#8384)
add dc074666718 [HUDI-6091] Add Java 11 and 17 to bundle validation (#8481)
No new revisions were added by this update.
Summary of changes:
.github/workflows/bot.yml | 21 -
packaging/bundle-validation/README.md | 53 ++
.../{Dockerfile-base => base-dev/Dockerfile} | 24 ++
.../{Dockerfile-base => base/Dockerfile} | 2 +-
.../base/build_flink1136hive313spark313.sh | 12 +++--
.../base/build_flink1146hive313spark323.sh | 12 +++--
.../base/build_flink1153hive313spark331.sh | 12 +++--
packaging/bundle-validation/ci_run.sh | 3 +-
packaging/bundle-validation/validate.sh| 44 --
9 files changed, 151 insertions(+), 32 deletions(-)
create mode 100644 packaging/bundle-validation/README.md
copy packaging/bundle-validation/{Dockerfile-base => base-dev/Dockerfile} (70%)
rename packaging/bundle-validation/{Dockerfile-base => base/Dockerfile} (96%)
copy scripts/release/validate_source_rat.sh =>
packaging/bundle-validation/base/build_flink1136hive313spark313.sh (68%)
copy scripts/release/validate_source_rat.sh =>
packaging/bundle-validation/base/build_flink1146hive313spark323.sh (68%)
copy scripts/release/validate_source_rat.sh =>
packaging/bundle-validation/base/build_flink1153hive313spark331.sh (68%)
[GitHub] [hudi] yihua commented on pull request #8481: [HUDI-6091] Add Java 11 and 17 to bundle validation
yihua commented on PR #8481: URL: https://github.com/apache/hudi/pull/8481#issuecomment-1518181911 Flink IT failed due to flakiness, which is irrelevant to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #8480: [HUDI-6090] Optimise payload size for list of FileGroupDTO
nsivabalan commented on code in PR #8480:
URL: https://github.com/apache/hudi/pull/8480#discussion_r1174041178
##
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/dto/FileGroupDTO.java:
##
@@ -46,17 +49,50 @@ public class FileGroupDTO {
TimelineDTO timeline;
public static FileGroupDTO fromFileGroup(HoodieFileGroup fileGroup) {
+return fromFileGroup(fileGroup, true);
+ }
+
+ public static List fromFileGroup(List
fileGroups) {
Review Comment:
this is a POJO class. lets not add static utils methods here. can we move it
to elsewhere. something like DTOUtils may be.
##
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/RequestHandler.java:
##
@@ -160,6 +160,8 @@ private boolean isLocalViewBehind(Context ctx) {
if (LOG.isDebugEnabled()) {
LOG.debug("Client [ LastTs=" + lastKnownInstantFromClient + ",
TimelineHash=" + timelineHashFromClient
+ "], localTimeline=" + localTimeline.getInstants());
+} else {
+ LOG.info("Client [ LastTs=" + lastKnownInstantFromClient + ",
TimelineHash=" + timelineHashFromClient + "]");
Review Comment:
this gets logged for every request reaching timeline server. so, better to
avoid this
##
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/dto/FileGroupDTO.java:
##
@@ -46,17 +49,50 @@ public class FileGroupDTO {
TimelineDTO timeline;
public static FileGroupDTO fromFileGroup(HoodieFileGroup fileGroup) {
+return fromFileGroup(fileGroup, true);
+ }
+
+ public static List fromFileGroup(List
fileGroups) {
+if (fileGroups.isEmpty()) {
+ return Collections.emptyList();
+}
+
+List fileGroupDTOs = fileGroups.stream()
+.map(fg -> FileGroupDTO.fromFileGroup(fg,
false)).collect(Collectors.toList());
+// Timeline exists only in the first file group DTO. Optimisation to
reduce payload size.
+fileGroupDTOs.set(0, FileGroupDTO.fromFileGroup(fileGroups.get(0), true));
+return fileGroupDTOs;
+ }
+
+ public static HoodieFileGroup toFileGroup(FileGroupDTO dto,
HoodieTableMetaClient metaClient) {
+return toFileGroup(dto, metaClient, null);
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8539: [HUDI-6067] Auto capitalize enum configs
hudi-bot commented on PR #8539: URL: https://github.com/apache/hudi/pull/8539#issuecomment-1518108578 ## CI report: * 343ab147919cb79a5b3e2f44028afb065b63f9d2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16535) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8536: [MINOR](hudi-metaserver) fix typos in README.md
hudi-bot commented on PR #8536: URL: https://github.com/apache/hudi/pull/8536#issuecomment-1518108504 ## CI report: * 397a7a1a090c741c9cd9e9c3dd01e81cc427cc42 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16530) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8539: [HUDI-6067] Auto capitalize enum configs
hudi-bot commented on PR #8539: URL: https://github.com/apache/hudi/pull/8539#issuecomment-1518099339 ## CI report: * 343ab147919cb79a5b3e2f44028afb065b63f9d2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8481: [HUDI-6091] Add Java 11 and 17 to bundle validation
hudi-bot commented on PR #8481: URL: https://github.com/apache/hudi/pull/8481#issuecomment-1518098955 ## CI report: * a2cd52729bec940ced866ab16174f9365dbcc8d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16470) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] ad1happy2go commented on issue #8532: [SUPPORT]org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 11 partition 1
ad1happy2go commented on issue #8532: URL: https://github.com/apache/hudi/issues/8532#issuecomment-1518097045 @gtwuser Is this error consistent? Can you provide the screenshots of spark UI with the executor/driver logs to look into it more. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HUDI-6062) Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum notation
[ https://issues.apache.org/jira/browse/HUDI-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-6062: -- Status: In Progress (was: Open) > Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum > notation > -- > > Key: HUDI-6062 > URL: https://issues.apache.org/jira/browse/HUDI-6062 > Project: Apache Hudi > Issue Type: Improvement > Components: clustering, code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > Labels: pull-request-available > > ClusteringOperator and LayoutOptimizationStrategy have enums with values that > are not capitalized snake case like every other config. We need to maintain > backwards compatibility so we can't just change this. To make this change, we > need to have the old values be translated to the updated values so that if a > user uses the old values, it will still work. For example if the > hoodie.layout.optimize.strategy config is set to "z-order" we need to > translate it to "ZORDER" and then use "ZORDER" internally. But the user could > also set the config to "ZORDER" of course. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6067) Auto capitalize all config values
[ https://issues.apache.org/jira/browse/HUDI-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-6067: -- Status: In Progress (was: Open) > Auto capitalize all config values > - > > Key: HUDI-6067 > URL: https://issues.apache.org/jira/browse/HUDI-6067 > Project: Apache Hudi > Issue Type: Improvement > Components: code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > Labels: pull-request-available > > Auto capitalize all config values as soon as they are input into Hudi so that > we minimize having to deal with capitalization internally. This is blocked > until all config values are capitalized -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HUDI-5761) Create "Type" configs for current configs that take in classes
[ https://issues.apache.org/jira/browse/HUDI-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-5761: - Assignee: Jonathan Vexler > Create "Type" configs for current configs that take in classes > -- > > Key: HUDI-5761 > URL: https://issues.apache.org/jira/browse/HUDI-5761 > Project: Apache Hudi > Issue Type: Improvement > Components: configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Major > Fix For: 0.14.0 > > > Take for example `hoodie.datasource.write.keygenerator.type` and its > counterpart `hoodie.datasource.write.keygenerator.class`. We should make > configs like keygenerator.type for configs currently like keygenerator.class. > Take for example `hoodie.datasource.write.record.merger.impls`. If you want > HoodieAvroRecordMerger you need to set the value to > org.apache.hudi.common.model.HoodieAvroRecordMerger. However, if you want > HoodieSparkRecordMerger, the full class path is > org.apache.hudi.HoodieSparkRecordMerger. Differences like this are an easy > cause for mistakes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6067) Auto capitalize all config values
[ https://issues.apache.org/jira/browse/HUDI-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6067: - Labels: pull-request-available (was: ) > Auto capitalize all config values > - > > Key: HUDI-6067 > URL: https://issues.apache.org/jira/browse/HUDI-6067 > Project: Apache Hudi > Issue Type: Improvement > Components: code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > Labels: pull-request-available > > Auto capitalize all config values as soon as they are input into Hudi so that > we minimize having to deal with capitalization internally. This is blocked > until all config values are capitalized -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] jonvex opened a new pull request, #8539: [HUDI-6067] Auto capitalize enum configs
jonvex opened a new pull request, #8539: URL: https://github.com/apache/hudi/pull/8539 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8538: [HUDI-6062] Standardize irregular enum configs
hudi-bot commented on PR #8538: URL: https://github.com/apache/hudi/pull/8538#issuecomment-1518054656 ## CI report: * dd36f97fb00d7825f64a8cc40122ff6309a90035 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16534) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8538: [HUDI-6062] Standardize irregular enum configs
hudi-bot commented on PR #8538: URL: https://github.com/apache/hudi/pull/8538#issuecomment-1518044628 ## CI report: * dd36f97fb00d7825f64a8cc40122ff6309a90035 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8488: [HUDI-5957] Fix table not exist when using 'db.table' in createHoodieClientFromPath
hudi-bot commented on PR #8488: URL: https://github.com/apache/hudi/pull/8488#issuecomment-1518032316 ## CI report: * e151236bbe169543f582096f2cc724cc057bdbb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16515) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16533) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-6062) Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum notation
[ https://issues.apache.org/jira/browse/HUDI-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715056#comment-17715056 ] Jonathan Vexler commented on HUDI-6062: --- ClusteringOperator seems to be internal only so I will leave as is > Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum > notation > -- > > Key: HUDI-6062 > URL: https://issues.apache.org/jira/browse/HUDI-6062 > Project: Apache Hudi > Issue Type: Improvement > Components: clustering, code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > Labels: pull-request-available > > ClusteringOperator and LayoutOptimizationStrategy have enums with values that > are not capitalized snake case like every other config. We need to maintain > backwards compatibility so we can't just change this. To make this change, we > need to have the old values be translated to the updated values so that if a > user uses the old values, it will still work. For example if the > hoodie.layout.optimize.strategy config is set to "z-order" we need to > translate it to "ZORDER" and then use "ZORDER" internally. But the user could > also set the config to "ZORDER" of course. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6062) Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum notation
[ https://issues.apache.org/jira/browse/HUDI-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6062: - Labels: pull-request-available (was: ) > Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum > notation > -- > > Key: HUDI-6062 > URL: https://issues.apache.org/jira/browse/HUDI-6062 > Project: Apache Hudi > Issue Type: Improvement > Components: clustering, code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > Labels: pull-request-available > > ClusteringOperator and LayoutOptimizationStrategy have enums with values that > are not capitalized snake case like every other config. We need to maintain > backwards compatibility so we can't just change this. To make this change, we > need to have the old values be translated to the updated values so that if a > user uses the old values, it will still work. For example if the > hoodie.layout.optimize.strategy config is set to "z-order" we need to > translate it to "ZORDER" and then use "ZORDER" internally. But the user could > also set the config to "ZORDER" of course. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[hudi] branch master updated (a6f85b77c0f -> 52efac958f2)
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from a6f85b77c0f [HUDI-5832] add relocated prefix for hbase classes in hbase-site.xml (#8029) add 52efac958f2 [HUDI-6039] Fixing FS based listing for full cleaning in clean Planner (#8384) No new revisions were added by this update. Summary of changes: .../hudi/table/action/clean/CleanPlanner.java | 11 +-- .../hudi/metadata/HoodieTableMetadataUtil.java | 22 + .../spark/sql/hudi/HoodieSparkSqlTestBase.scala| 13 +++ .../sql/hudi/TestAlterTableDropPartition.scala | 98 +- 4 files changed, 111 insertions(+), 33 deletions(-)
[GitHub] [hudi] nsivabalan merged pull request #8384: [HUDI-6039] Fixing FS based listing for full cleaning in clean Planner
nsivabalan merged PR #8384: URL: https://github.com/apache/hudi/pull/8384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on pull request #8488: [HUDI-5957] Fix table not exist when using 'db.table' in createHoodieClientFromPath
Zouxxyy commented on PR #8488: URL: https://github.com/apache/hudi/pull/8488#issuecomment-151804 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] jonvex opened a new pull request, #8538: [HUDI-6062] Standardize irregular enum configs
jonvex opened a new pull request, #8538: URL: https://github.com/apache/hudi/pull/8538 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8537: [HUDI-6122] Unify options in call procedure
hudi-bot commented on PR #8537: URL: https://github.com/apache/hudi/pull/8537#issuecomment-1517984772 ## CI report: * 6720828d44070be1152eb035d7a59412e5c4925f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16531) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8537: [HUDI-6122] Unify options in call procedure
hudi-bot commented on PR #8537: URL: https://github.com/apache/hudi/pull/8537#issuecomment-1517972355 ## CI report: * 6720828d44070be1152eb035d7a59412e5c4925f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8481: [HUDI-6091] Add Java 11 and 17 to bundle validation
hudi-bot commented on PR #8481: URL: https://github.com/apache/hudi/pull/8481#issuecomment-1517971915 ## CI report: * a2cd52729bec940ced866ab16174f9365dbcc8d2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16470) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8536: [MINOR](hudi-metaserver) fix typos in README.md
hudi-bot commented on PR #8536: URL: https://github.com/apache/hudi/pull/8536#issuecomment-1517960668 ## CI report: * 397a7a1a090c741c9cd9e9c3dd01e81cc427cc42 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16530) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8530: [HUDI-6121] Log the exception in the hudi commit kafka callback
hudi-bot commented on PR #8530: URL: https://github.com/apache/hudi/pull/8530#issuecomment-1517960618 ## CI report: * 1ac5b9dd216813f017012744533490d91ff3ca3d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16519) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8481: [HUDI-6091] Add Java 11 and 17 to bundle validation
hudi-bot commented on PR #8481: URL: https://github.com/apache/hudi/pull/8481#issuecomment-1517960269 ## CI report: * a2cd52729bec940ced866ab16174f9365dbcc8d2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HUDI-6067) Auto capitalize all config values
[ https://issues.apache.org/jira/browse/HUDI-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-6067: - Assignee: Jonathan Vexler > Auto capitalize all config values > - > > Key: HUDI-6067 > URL: https://issues.apache.org/jira/browse/HUDI-6067 > Project: Apache Hudi > Issue Type: Improvement > Components: code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > > Auto capitalize all config values as soon as they are input into Hudi so that > we minimize having to deal with capitalization internally. This is blocked > until all config values are capitalized -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HUDI-6062) Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum notation
[ https://issues.apache.org/jira/browse/HUDI-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-6062: - Assignee: Jonathan Vexler > Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum > notation > -- > > Key: HUDI-6062 > URL: https://issues.apache.org/jira/browse/HUDI-6062 > Project: Apache Hudi > Issue Type: Improvement > Components: clustering, code-quality, configs >Reporter: Jonathan Vexler >Assignee: Jonathan Vexler >Priority: Minor > > ClusteringOperator and LayoutOptimizationStrategy have enums with values that > are not capitalized snake case like every other config. We need to maintain > backwards compatibility so we can't just change this. To make this change, we > need to have the old values be translated to the updated values so that if a > user uses the old values, it will still work. For example if the > hoodie.layout.optimize.strategy config is set to "z-order" we need to > translate it to "ZORDER" and then use "ZORDER" internally. But the user could > also set the config to "ZORDER" of course. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6122) Call clean/compaction support custom options
[ https://issues.apache.org/jira/browse/HUDI-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6122: - Labels: pull-request-available (was: ) > Call clean/compaction support custom options > > > Key: HUDI-6122 > URL: https://issues.apache.org/jira/browse/HUDI-6122 > Project: Apache Hudi > Issue Type: Improvement >Reporter: zouxxyy >Assignee: zouxxyy >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] Zouxxyy opened a new pull request, #8537: [HUDI-6122] Unify options' building in call procedure
Zouxxyy opened a new pull request, #8537: URL: https://github.com/apache/hudi/pull/8537 ### Change Logs - Don't explicitly set default values for hoodie confs in the call procedure, this is because **hudi should build the value of hoodie confs itself** according to its rules (for example, build from table props, hudi globle conf and so on) when user does not specify. ```scala // before ProcedureParameter.optional(4, "retain_commits", DataTypes.IntegerType, HoodieCleanConfig.CLEANER_COMMITS_RETAINED.defaultValue().toInt) // after ProcedureParameter.optional(4, "retain_commits", DataTypes.IntegerType) ``` - Add options in run_clean/compacion for use to specifie the running configs of the table service ```scala call run_compaction(table => "tbl", op => "run", options => "x=a,y=b") ``` - Format code: `default` in the `ProcedureParameter` needn't to be added if no default value is specified. And when the `ProcedureParameter` is require, `default` should not be exposed. ### Impact Above ### Risk level (write none, low medium or high below) low ### Documentation Update None ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] Hans-Raintree commented on issue #8278: [SUPPORT] Deltastreamer Fails with AWSDmsAvroPayload
Hans-Raintree commented on issue #8278: URL: https://github.com/apache/hudi/issues/8278#issuecomment-1517912977 Yeah, I don't seem to have this issue with 0.12.2-amzn-0, I need 0.13.0 for the schema evolution and cdc log aspect, but running to the above issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] littleeleventhwolf commented on pull request #8536: [MINOR](hudi-metaserver) fix typos in README.md
littleeleventhwolf commented on PR #8536: URL: https://github.com/apache/hudi/pull/8536#issuecomment-1517903290 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8536: [MINOR](hudi-metaserver) fix typos in README.md
hudi-bot commented on PR #8536: URL: https://github.com/apache/hudi/pull/8536#issuecomment-1517901244 ## CI report: * 397a7a1a090c741c9cd9e9c3dd01e81cc427cc42 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8528: [HUDI-6119] Add testcase for timeline timezone UTC
hudi-bot commented on PR #8528: URL: https://github.com/apache/hudi/pull/8528#issuecomment-1517901127 ## CI report: * 05b00cfdc3dd6072a297bf990bc6dd95fd2a21f7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16517) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] littleeleventhwolf opened a new pull request, #8536: [MINOR](hudi-metaserver) fix typos in README.md
littleeleventhwolf opened a new pull request, #8536: URL: https://github.com/apache/hudi/pull/8536 ### Change Logs Fix typos and format text-blocks properly. ### Impact No ### Risk level (write none, low medium or high below) No ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
