[GitHub] [hudi] Zouxxyy commented on issue #8265: [SUPPORT] Flink Table planner not loading problem

2023-06-23 Thread via GitHub
Zouxxyy commented on issue #8265: URL: https://github.com/apache/hudi/issues/8265#issuecomment-1605291612 @ertanden I also found this problem, it may can be solved by: (1) Copy the codegen module from flink into hudi, but it is written by scala; (2) Try to use `PlannerCodeLoader` in fli

[GitHub] [hudi] codope merged pull request #9043: [HUDI-6385] Fixing archival tests after RLI base patch

2023-06-23 Thread via GitHub
codope merged PR #9043: URL: https://github.com/apache/hudi/pull/9043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[hudi] branch master updated: [HUDI-6385] Fixing archival tests after RLI base patch (#9043)

2023-06-23 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8a5288202e5 [HUDI-6385] Fixing archival tests afte

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1605279901 ## CI report: * 02abed756a576433d5c14bf9b62e5cd7cba46309 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] pratyakshsharma commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
pratyakshsharma commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1605275364 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] pratyakshsharma commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
pratyakshsharma commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1605275269 @bvaradar @yihua This is good to review now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot commented on pull request #8978: [HUDI-6315] Optimize DELETE codepath to use meta fields instead of key generation and index lookup

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8978: URL: https://github.com/apache/hudi/pull/8978#issuecomment-1605230689 ## CI report: * 29f11d44f38fa9f9f282ecc29688b7411270b390 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9035: [HUDI-6416] Completion markers for handling execution engine (spark) …

2023-06-23 Thread via GitHub
nsivabalan commented on code in PR #9035: URL: https://github.com/apache/hudi/pull/9035#discussion_r1240377379 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java: ## @@ -138,9 +139,35 @@ protected Path makeNewFilePath(String partitionPath,

[GitHub] [hudi] hudi-bot commented on pull request #9043: [HUDI-6385] Fixing archival tests after RLI base patch

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9043: URL: https://github.com/apache/hudi/pull/9043#issuecomment-1605143517 ## CI report: * a5a2e46728626d500238b8fed5446c51e9526b07 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] parisni commented on a diff in pull request #8997: [HUDI-6399] Warn when datadog api key is wrong

2023-06-23 Thread via GitHub
parisni commented on code in PR #8997: URL: https://github.com/apache/hudi/pull/8997#discussion_r1240486727 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/datadog/DatadogHttpClient.java: ## @@ -84,8 +84,8 @@ private void validateApiKey() { try (Clos

[GitHub] [hudi] hudi-bot commented on pull request #8978: [HUDI-6315] Optimize DELETE codepath to use meta fields instead of key generation and index lookup

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8978: URL: https://github.com/apache/hudi/pull/8978#issuecomment-1605084500 ## CI report: * f36579155cff886bc038dbd6955879386e20b23f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #9037: [HUDI-6420] Fixing Hfile on-demand and prefix based reads to use optimized apis

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9037: URL: https://github.com/apache/hudi/pull/9037#issuecomment-1605080021 ## CI report: * 07f58aad9d3d0ad866f5ee44e05d7ee5022c43c1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] hudi-bot commented on pull request #8978: [HUDI-6315] Optimize DELETE codepath to use meta fields instead of key generation and index lookup

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8978: URL: https://github.com/apache/hudi/pull/8978#issuecomment-1605079944 ## CI report: * f36579155cff886bc038dbd6955879386e20b23f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] amrishlal commented on a diff in pull request #8978: [HUDI-6315] Optimize DELETE codepath to use meta fields instead of key generation and index lookup

2023-06-23 Thread via GitHub
amrishlal commented on code in PR #8978: URL: https://github.com/apache/hudi/pull/8978#discussion_r1240446113 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -254,11 +254,26 @@ object HoodieSparkSqlWriter { val

[hudi] branch master updated: [HUDI-6406] Pass in Spark Engine Context Wrapper for DeltaSync instead of spark engine context (#9008)

2023-06-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 407389ae9a2 [HUDI-6406] Pass in Spark Engine Contex

[GitHub] [hudi] yihua merged pull request #9008: [HUDI-6406] Pass in Spark Engine Context Wrapper for DeltaSync instead of spark engine context

2023-06-23 Thread via GitHub
yihua merged PR #9008: URL: https://github.com/apache/hudi/pull/9008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] yihua commented on a diff in pull request #8518: [HUDI-6112] [DOCS] Fix HoodieConfigDocGenerator to emit table layout …

2023-06-23 Thread via GitHub
yihua commented on code in PR #8518: URL: https://github.com/apache/hudi/pull/8518#discussion_r1240373903 ## hudi-utils/pom.xml: ## @@ -22,7 +22,7 @@ org.apache.hudi hudi-utils jar -1.0-SNAPSHOT +0.14.0-SNAPSHOT Review Comment: Do we still want to keep

[GitHub] [hudi] yihua merged pull request #7087: [MINOR] Fix creation of timeline instants in cleaner test cases

2023-06-23 Thread via GitHub
yihua merged PR #7087: URL: https://github.com/apache/hudi/pull/7087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[hudi] branch master updated (f8cafbbba1e -> 3ccc5ce499e)

2023-06-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from f8cafbbba1e [HUDI-6412] Handle record index payload update (#9036) add 3ccc5ce499e [MINOR] Fix creation of timeline

[GitHub] [hudi] hudi-bot commented on pull request #9043: [HUDI-6385] Fixing archival tests after RLI base patch

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9043: URL: https://github.com/apache/hudi/pull/9043#issuecomment-1604964700 ## CI report: * a5a2e46728626d500238b8fed5446c51e9526b07 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] hudi-bot commented on pull request #9037: [HUDI-6420] Fixing Hfile on-demand and prefix based reads to use optimized apis

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9037: URL: https://github.com/apache/hudi/pull/9037#issuecomment-1604964552 ## CI report: * 950b7830763ecbae01958b01fd73d219d65ccd05 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #9043: [HUDI-6385] Fixing archival tests after RLI base patch

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9043: URL: https://github.com/apache/hudi/pull/9043#issuecomment-1604953748 ## CI report: * a5a2e46728626d500238b8fed5446c51e9526b07 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9037: [HUDI-6420] Fixing Hfile on-demand and prefix based reads to use optimized apis

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9037: URL: https://github.com/apache/hudi/pull/9037#issuecomment-1604953563 ## CI report: * 950b7830763ecbae01958b01fd73d219d65ccd05 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] yihua commented on a diff in pull request #8997: [HUDI-6399] Warn when datadog api key is wrong

2023-06-23 Thread via GitHub
yihua commented on code in PR #8997: URL: https://github.com/apache/hudi/pull/8997#discussion_r1240365880 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/datadog/DatadogHttpClient.java: ## @@ -84,8 +84,8 @@ private void validateApiKey() { try (Closea

[hudi] branch asf-site updated: [DOCS] Update data quality docs (#8552)

2023-06-23 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 7141e68c64a [DOCS] Update data quality docs (#8

[GitHub] [hudi] yihua merged pull request #8552: [DOCS] Update data quality docs

2023-06-23 Thread via GitHub
yihua merged PR #8552: URL: https://github.com/apache/hudi/pull/8552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[jira] [Updated] (HUDI-6385) Fix testArchivalWithAutoAdjustmentBasedOnCleanConfigs

2023-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6385: - Labels: pull-request-available (was: ) > Fix testArchivalWithAutoAdjustmentBasedOnCleanConfigs >

[GitHub] [hudi] nsivabalan opened a new pull request, #9043: [HUDI-6385] Fixing archival tests after RLI base patch

2023-06-23 Thread via GitHub
nsivabalan opened a new pull request, #9043: URL: https://github.com/apache/hudi/pull/9043 ### Change Logs Fixed an archival related test which was disabled when RLI patch was added. Fixes are only in test code. ### Impact Fixed an archival related test which was disabl

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9037: [HUDI-6420] Fixing Hfile on-demand and prefix based reads to use optimized apis

2023-06-23 Thread via GitHub
nsivabalan commented on code in PR #9037: URL: https://github.com/apache/hudi/pull/9037#discussion_r1240286597 ## hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieAvroHFileReader.java: ## @@ -120,20 +120,14 @@ public HoodieAvroHFileReader(Path path, HFile.Reader reade

[GitHub] [hudi] hudi-bot commented on pull request #7087: [MINOR] Fix creation of timeline instants in cleaner test cases

2023-06-23 Thread via GitHub
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1604825955 ## CI report: * 345eec7ddb287af6b1e51187581356173acc85b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] yihua commented on a diff in pull request #8552: [DOCS] Update data quality docs

2023-06-23 Thread via GitHub
yihua commented on code in PR #8552: URL: https://github.com/apache/hudi/pull/8552#discussion_r1225597838 ## website/docs/precommit_validator.md: ## @@ -17,13 +18,18 @@ spark.write.format("hudi") Today you can use any of these validators and even have the flexibility to extend

[GitHub] [hudi] yihua commented on a diff in pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
yihua commented on code in PR #9039: URL: https://github.com/apache/hudi/pull/9039#discussion_r1240218048 ## hudi-common/src/main/java/org/apache/hudi/common/util/CompactionUtils.java: ## @@ -345,22 +345,34 @@ public static Option> getDeltaCommitsSinceLa * @return the oldes

[GitHub] [hudi] yihua commented on pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
yihua commented on PR #9039: URL: https://github.com/apache/hudi/pull/9039#issuecomment-1604718015 > 2. getOldestInstantToRetainForCompaction also needs to check earliestInstantToRetain like getOldestInstantToRetainForClustering, to ensure that the files related to the compaction instant ha

[GitHub] [hudi] hudi-bot commented on pull request #7087: [MINOR] Fix creation of timeline instants in cleaner test cases

2023-06-23 Thread via GitHub
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1604628996 ## CI report: * c3eb4a4a4581adb693ba8d2baa54e7c490fafd2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] yihua commented on a diff in pull request #8840: [HUDI-5352] Fix `LocalDate` serialization in colstats

2023-06-23 Thread via GitHub
yihua commented on code in PR #8840: URL: https://github.com/apache/hudi/pull/8840#discussion_r1240098101 ## hudi-common/src/test/java/org/apache/hudi/common/util/TestJsonUtils.java: ## @@ -1,60 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or mo

[GitHub] [hudi] hudi-bot commented on pull request #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2023-06-23 Thread via GitHub
hudi-bot commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1604569309 ## CI report: * c3eb4a4a4581adb693ba8d2baa54e7c490fafd2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1265

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1604560207 ## CI report: * 02abed756a576433d5c14bf9b62e5cd7cba46309 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1604548936 ## CI report: * 1ec25e96a87e00a2a2a0ba966cdb012720902008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] dht7 opened a new issue, #9042: [SUPPORT] Cannot write nullable values to non-null column

2023-06-23 Thread via GitHub
dht7 opened a new issue, #9042: URL: https://github.com/apache/hudi/issues/9042 **Describe the problem you faced** Apache Hudi tables created using CTAS command in Spark-SQL with array column types fail to insert overwrite data. **To Reproduce** Steps to reproduce the be

[GitHub] [hudi] hudi-bot commented on pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9030: URL: https://github.com/apache/hudi/pull/9030#issuecomment-1604398698 ## CI report: * 9d724202d3fde4914bb1351baf9c355977c08eb6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1807

[GitHub] [hudi] ankitchandnani commented on issue #8672: [SUPPORT] INSERT_OVERWRITE_TABLE operation not working on Hudi 0.12.2 using EMR Deltastreamer

2023-06-23 Thread via GitHub
ankitchandnani commented on issue #8672: URL: https://github.com/apache/hudi/issues/8672#issuecomment-1604361672 Hi @ad1happy2go , looks like the patch is not applied in Hudi 0.13.0. Unfortunately, no AWS EMR version supports a hudi version with a fix. I will be waiting for the next EMR ver

[jira] [Closed] (HUDI-5272) Align with Flink to support no_precombine in spark

2023-06-23 Thread kazdy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kazdy closed HUDI-5272. --- Resolution: Won't Fix > Align with Flink to support no_precombine in spark > -

[jira] [Closed] (HUDI-5707) Support offset reset strategy w/ spark streaming read from hudi table

2023-06-23 Thread kazdy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kazdy closed HUDI-5707. --- Fix Version/s: (was: 1.0.0) Resolution: Won't Fix > Support offset reset strategy w/ spark streaming read f

[GitHub] [hudi] hudi-bot commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1604318527 ## CI report: * 1ec25e96a87e00a2a2a0ba966cdb012720902008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1604318283 ## CI report: * 02abed756a576433d5c14bf9b62e5cd7cba46309 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] pratyakshsharma commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
pratyakshsharma commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1604315657 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] hudi-bot commented on pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9030: URL: https://github.com/apache/hudi/pull/9030#issuecomment-1604184594 ## CI report: * 891553af5efed14209f4922b0f6dedb44d224ce6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804

[GitHub] [hudi] hudi-bot commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1604182762 ## CI report: * 1ec25e96a87e00a2a2a0ba966cdb012720902008 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9030: URL: https://github.com/apache/hudi/pull/9030#issuecomment-1604175499 ## CI report: * 891553af5efed14209f4922b0f6dedb44d224ce6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1804

[GitHub] [hudi] beyond1920 commented on a diff in pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
beyond1920 commented on code in PR #9030: URL: https://github.com/apache/hudi/pull/9030#discussion_r1239726999 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java: ## @@ -328,6 +330,20 @@ public class HoodieClusteringConfig extends

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1604173799 ## CI report: * 02abed756a576433d5c14bf9b62e5cd7cba46309 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] pratyakshsharma commented on pull request #7087: [minor]: Fix creation of timeline instants in cleaner test cases

2023-06-23 Thread via GitHub
pratyakshsharma commented on PR #7087: URL: https://github.com/apache/hudi/pull/7087#issuecomment-1604172739 @nsivabalan @yihua Can one of you review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] beyond1920 commented on a diff in pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
beyond1920 commented on code in PR #9030: URL: https://github.com/apache/hudi/pull/9030#discussion_r1239722712 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/cluster/ITTestFlinkConsistentHashingClustering.java: ## @@ -0,0 +1,194 @@ +/* + * Licensed to the

[GitHub] [hudi] beyond1920 commented on a diff in pull request #9030: [HUDI-6328] Flink support generate resize plan for consistent bucket index

2023-06-23 Thread via GitHub
beyond1920 commented on code in PR #9030: URL: https://github.com/apache/hudi/pull/9030#discussion_r1239722712 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/cluster/ITTestFlinkConsistentHashingClustering.java: ## @@ -0,0 +1,194 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9039: URL: https://github.com/apache/hudi/pull/9039#issuecomment-1604167529 ## CI report: * b367bb301e65778c657305ef01e5e6fbeab0c24e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9039: URL: https://github.com/apache/hudi/pull/9039#issuecomment-1604041150 ## CI report: * 6a8f5400d5063a2b9e1df7b7fcb3a886d66d8e6c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1805

[GitHub] [hudi] hudi-bot commented on pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9039: URL: https://github.com/apache/hudi/pull/9039#issuecomment-1604031504 ## CI report: * 6a8f5400d5063a2b9e1df7b7fcb3a886d66d8e6c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1805

[GitHub] [hudi] zhuanshenbsj1 commented on pull request #9039: [HUDI-6424] getOldestInstantToRetainForCompaction needs to add clean validation

2023-06-23 Thread via GitHub
zhuanshenbsj1 commented on PR #9039: URL: https://github.com/apache/hudi/pull/9039#issuecomment-1604005287 > [6424.patch.zip](https://github.com/apache/hudi/files/11844208/6424.patch.zip) > Thanks for the contribution, I have reviewed and attached a patch here, there are still some test

[GitHub] [hudi] hudi-bot commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1603963248 ## CI report: * ec9f70d65a65dbc1dd810626089791a6e7dc7b31 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1575

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1603962859 ## CI report: * 8a0a65e665e2676c175d9eba8788dd8f29c51f2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1561

[GitHub] [hudi] hudi-bot commented on pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8208: URL: https://github.com/apache/hudi/pull/8208#issuecomment-1603952298 ## CI report: * ec9f70d65a65dbc1dd810626089791a6e7dc7b31 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1575

[GitHub] [hudi] hudi-bot commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1603952106 ## CI report: * 8a0a65e665e2676c175d9eba8788dd8f29c51f2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1561

[GitHub] [hudi] pratyakshsharma commented on pull request #8124: [HUDI-5902]: Parallelise glue sync calls

2023-06-23 Thread via GitHub
pratyakshsharma commented on PR #8124: URL: https://github.com/apache/hudi/pull/8124#issuecomment-1603931319 @xushiyan @codope Please take a pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] ad1happy2go commented on issue #9029: [SUPPORT]hudi upsert data Caused by: org.apache.hadoop.fs.PathIsNotEmptyDirectoryException

2023-06-23 Thread via GitHub
ad1happy2go commented on issue #9029: URL: https://github.com/apache/hudi/issues/9029#issuecomment-1603929336 @zyclove Do you know what created this directory? Did cleaning this dir and resuming the job works? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] pratyakshsharma commented on a diff in pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
pratyakshsharma commented on code in PR #8208: URL: https://github.com/apache/hudi/pull/8208#discussion_r1239514140 ## hudi-aws/src/main/java/org/apache/hudi/config/HoodieAWSConfig.java: ## @@ -63,6 +63,11 @@ public class HoodieAWSConfig extends HoodieConfig { .sinceVer

[GitHub] [hudi] pratyakshsharma commented on a diff in pull request #8208: [HUDI-5903]: Introduce Glue sync configs

2023-06-23 Thread via GitHub
pratyakshsharma commented on code in PR #8208: URL: https://github.com/apache/hudi/pull/8208#discussion_r1239497892 ## hudi-aws/src/main/java/org/apache/hudi/config/HoodieAWSConfig.java: ## @@ -63,6 +63,11 @@ public class HoodieAWSConfig extends HoodieConfig { .sinceVer

[GitHub] [hudi] hudi-bot commented on pull request #9041: [HUDI-6431] Support update partition path in record-level index

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9041: URL: https://github.com/apache/hudi/pull/9041#issuecomment-1603870144 ## CI report: * ff2332533dce2f88e991bf5709d117d1a141cb9b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[GitHub] [hudi] hudi-bot commented on pull request #8978: [HUDI-6315] Optimize DELETE codepath to use meta fields instead of key generation and index lookup

2023-06-23 Thread via GitHub
hudi-bot commented on PR #8978: URL: https://github.com/apache/hudi/pull/8978#issuecomment-1603869891 ## CI report: * f36579155cff886bc038dbd6955879386e20b23f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806

[hudi] branch master updated: [HUDI-6412] Handle record index payload update (#9036)

2023-06-23 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f8cafbbba1e [HUDI-6412] Handle record index payloa

[GitHub] [hudi] codope merged pull request #9036: [HUDI-6412] Handle record index payload update

2023-06-23 Thread via GitHub
codope merged PR #9036: URL: https://github.com/apache/hudi/pull/9036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[GitHub] [hudi] hudi-bot commented on pull request #9036: [HUDI-6412] Handle record index payload update

2023-06-23 Thread via GitHub
hudi-bot commented on PR #9036: URL: https://github.com/apache/hudi/pull/9036#issuecomment-1603771440 ## CI report: * d25d2c593a51dbb165de1fd76d63ae16169b8315 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1806