[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #8638: added new exception types

2023-05-22 Thread via GitHub
the-other-tim-brown commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1201011129 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala: ## @@ -138,18 +139,26 @@ object AvroConversionUtils { def

[GitHub] [hudi] yihua commented on a diff in pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
yihua commented on code in PR #8779: URL: https://github.com/apache/hudi/pull/8779#discussion_r1201007409 ## packaging/bundle-validation/ci_run.sh: ## @@ -96,14 +99,73 @@ fi # Copy bundle jars to temp dir for mounting TMP_JARS_DIR=/tmp/jars/$(date +%s) mkdir -p $TMP_JARS_DIR

[GitHub] [hudi] yihua commented on a diff in pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
yihua commented on code in PR #8779: URL: https://github.com/apache/hudi/pull/8779#discussion_r1201005064 ## .github/workflows/bot.yml: ## @@ -210,3 +210,62 @@ jobs: run: | HUDI_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)

[GitHub] [hudi] yihua commented on a diff in pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
yihua commented on code in PR #8779: URL: https://github.com/apache/hudi/pull/8779#discussion_r1201004113 ## packaging/bundle-validation/README.md: ## @@ -50,4 +50,20 @@ Note that for each library like Hive and Spark, the download and extraction happ only one layer is

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1557893116 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 6cbafd0d08e12fc4e77a9f0058fe24b23e352a69 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557892286 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
nsivabalan commented on code in PR #8779: URL: https://github.com/apache/hudi/pull/8779#discussion_r1200947437 ## packaging/bundle-validation/README.md: ## @@ -50,4 +50,20 @@ Note that for each library like Hive and Spark, the download and extraction happ only one layer is

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557807149 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #8778: [MINOR] Avoid synchronized block in HoodieLockMetrics if key is present in cache

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8778: URL: https://github.com/apache/hudi/pull/8778#issuecomment-1557768583 ## CI report: * e27ac9168cce2c695173ca0d9110cdecbc3164b5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8781: [MINOR] disable schema validation in master

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8781: URL: https://github.com/apache/hudi/pull/8781#issuecomment-1557691605 ## CI report: * d26759cbd7fdef9819979804b94db7e019ac7490 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557690341 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #8781: [MINOR] disable schema validation in master

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8781: URL: https://github.com/apache/hudi/pull/8781#issuecomment-1557681131 ## CI report: * d26759cbd7fdef9819979804b94db7e019ac7490 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8779: URL: https://github.com/apache/hudi/pull/8779#issuecomment-1557681057 ## CI report: * aff465a8e6b11d76be2f9025013ca4b8eaa9c04a UNKNOWN * 86c4d5baf362e91f796108e16b4a38b7c94e5439 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1557680784 ## CI report: * 7cf387b3253cfa70490b616e54899b3656584152 Azure:

[jira] [Updated] (HUDI-5724) Test MOR table w/ global index w/ update partition path to true

2023-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5724: - Labels: pull-request-available (was: ) > Test MOR table w/ global index w/ update partition path

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8736: [HUDI-5724] Fix merger api usage with more UTs

2023-05-22 Thread via GitHub
nsivabalan commented on code in PR #8736: URL: https://github.com/apache/hudi/pull/8736#discussion_r1200866989 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieAdaptablePayloadDataGenerator.java: ## @@ -136,6 +148,18 @@ public static List getDeletes(List

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557679492 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8779: URL: https://github.com/apache/hudi/pull/8779#issuecomment-1557669792 ## CI report: * aff465a8e6b11d76be2f9025013ca4b8eaa9c04a UNKNOWN * 4fefd25e9d3251058dda43b1c0034ead6fc35498 Azure:

[GitHub] [hudi] jonvex opened a new pull request, #8781: [MINOR] disable schema validation in master

2023-05-22 Thread via GitHub
jonvex opened a new pull request, #8781: URL: https://github.com/apache/hudi/pull/8781 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] hudi-bot commented on pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8779: URL: https://github.com/apache/hudi/pull/8779#issuecomment-1557612358 ## CI report: * aff465a8e6b11d76be2f9025013ca4b8eaa9c04a UNKNOWN * 4fefd25e9d3251058dda43b1c0034ead6fc35498 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557610771 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] codope commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
codope commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1200805206 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieMetaSyncOperations.java: ## @@ -186,16 +188,20 @@ default void

[GitHub] [hudi] codope commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
codope commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1200770892 ## hudi-aws/src/main/java/org/apache/hudi/aws/sync/AWSGlueCatalogSyncClient.java: ## @@ -477,13 +472,19 @@ private static Table getTable(AWSGlue awsGlue, String

[GitHub] [hudi] hudi-bot commented on pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8779: URL: https://github.com/apache/hudi/pull/8779#issuecomment-1557600463 ## CI report: * aff465a8e6b11d76be2f9025013ca4b8eaa9c04a UNKNOWN * 4fefd25e9d3251058dda43b1c0034ead6fc35498 UNKNOWN Bot commands @hudi-bot supports the

[GitHub] [hudi] hudi-bot commented on pull request #8618: [HUDI-4944] Don't decode URI twice in HoodieBootstrapRDD

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8618: URL: https://github.com/apache/hudi/pull/8618#issuecomment-1557599442 ## CI report: * b1d38ff6b1cc82f5cb90c57658a68ae7463c20fe UNKNOWN * 4f1c263e30d80249bcf6d6984afdaa3e11d1d9eb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557598769 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * e64fd632c8dee69355f34ee9f221f8ddef0dfee8 UNKNOWN *

[GitHub] [hudi] ChiehFu opened a new issue, #8780: [SUPPORT] Change of Hudi precombine field

2023-05-22 Thread via GitHub
ChiehFu opened a new issue, #8780: URL: https://github.com/apache/hudi/issues/8780 **Describe the problem you faced** Hi, I have a use case where I want to change the hudi precombine field of an insert_overwrite (with combine-before-insert enabled) table due the existing

[GitHub] [hudi] hudi-bot commented on pull request #8774: [HUDI-6246] Fixing restore for compaction commit

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8774: URL: https://github.com/apache/hudi/pull/8774#issuecomment-1557587611 ## CI report: * afe28587616d76a4f1183944dd2bd4ac8ee5bad7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8779: URL: https://github.com/apache/hudi/pull/8779#issuecomment-1557587781 ## CI report: * aff465a8e6b11d76be2f9025013ca4b8eaa9c04a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] nsivabalan commented on issue #8670: [SUPPORT] Hudi cannot multi-write referring to case #7653

2023-05-22 Thread via GitHub
nsivabalan commented on issue #8670: URL: https://github.com/apache/hudi/issues/8670#issuecomment-1557578573 We do have some plans wrt MVCC for regular writers with 1.x. but as of now, hudi supports OCC which is common in the industry. -- This is an automated message from the Apache Git

[GitHub] [hudi] nsivabalan commented on issue #8670: [SUPPORT] Hudi cannot multi-write referring to case #7653

2023-05-22 Thread via GitHub
nsivabalan commented on issue #8670: URL: https://github.com/apache/hudi/issues/8670#issuecomment-1557575588 Nope. issue is, with OCC (optimistic concurrency control), when two writers are trying to modify the same data concurrently, only one can succeed and another one will fail. So, as

[GitHub] [hudi] nsivabalan closed pull request #8469: [HUDI-6149] Adding a tool to fetch table size for hudi tables

2023-05-22 Thread via GitHub
nsivabalan closed pull request #8469: [HUDI-6149] Adding a tool to fetch table size for hudi tables URL: https://github.com/apache/hudi/pull/8469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] nsivabalan commented on pull request #8469: [HUDI-6149] Adding a tool to fetch table size for hudi tables

2023-05-22 Thread via GitHub
nsivabalan commented on PR #8469: URL: https://github.com/apache/hudi/pull/8469#issuecomment-1557565570 Already incorporated in https://github.com/apache/hudi/commit/a5fde6b5c22fed353e5ac4350fa1beed15b802df -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] yihua commented on issue #7589: [Support] Keep only clustered file(all) after cleaning

2023-05-22 Thread via GitHub
yihua commented on issue #7589: URL: https://github.com/apache/hudi/issues/7589#issuecomment-1557557503 @maheshguptags Cool, thanks. Just to clarify, for a Hudi table on storage, you can always create a savepoint using the base path, regardless of whether the table is registered in the

[GitHub] [hudi] codope commented on a diff in pull request #8774: [HUDI-6246] Fixing restore for compaction commit

2023-05-22 Thread via GitHub
codope commented on code in PR #8774: URL: https://github.com/apache/hudi/pull/8774#discussion_r1200730726 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestSavepointRestoreMergeOnRead.java: ## @@ -102,13 +106,104 @@ void

[jira] [Updated] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6247: - Labels: pull-request-available (was: ) > Add bundle validation based on release candidates >

[GitHub] [hudi] yihua opened a new pull request, #8779: [HUDI-6247] Add bundle validation for release candidates

2023-05-22 Thread via GitHub
yihua opened a new pull request, #8779: URL: https://github.com/apache/hudi/pull/8779 ### Change Logs This PR adds the bundle validation for the release candidates. By default, this is disabled. To enable the bundle validation for release candidates, makes the following changes to

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557512309 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * 78d54f595beb56afcafc17cbab200d1ef479eaa3 Azure:

[GitHub] [hudi] xushiyan commented on pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
xushiyan commented on PR #8775: URL: https://github.com/apache/hudi/pull/8775#issuecomment-1557445721 @lokeshj1703 @LinMingQiang pls check this out as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557427614 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * 78d54f595beb56afcafc17cbab200d1ef479eaa3 Azure:

[jira] [Updated] (HUDI-6248) Validate col stats for MOR data table

2023-05-22 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-6248: -- Description: For COW, col stats processing/evaluation is straight forward. but for MOR

[jira] [Created] (HUDI-6248) Validate col stats for MOR data table

2023-05-22 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6248: - Summary: Validate col stats for MOR data table Key: HUDI-6248 URL: https://issues.apache.org/jira/browse/HUDI-6248 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6247: Description: We should check in the code for validation bundles in release candidates to make the release

[jira] [Assigned] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-6247: --- Assignee: Ethan Guo > Add bundle validation based on release candidates >

[jira] [Updated] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6247: Priority: Critical (was: Major) > Add bundle validation based on release candidates >

[jira] [Updated] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6247: Story Points: 0.5 > Add bundle validation based on release candidates >

[jira] [Created] (HUDI-6247) Add bundle validation based on release candidates

2023-05-22 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-6247: --- Summary: Add bundle validation based on release candidates Key: HUDI-6247 URL: https://issues.apache.org/jira/browse/HUDI-6247 Project: Apache Hudi Issue Type:

[GitHub] [hudi] jarrodcodes commented on issue #5330: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-05-22 Thread via GitHub
jarrodcodes commented on issue #5330: URL: https://github.com/apache/hudi/issues/5330#issuecomment-1557410683 > Did you use the COW table or MOR? We are using COW. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] hudi-bot commented on pull request #8776: [HUDI-5994] Bucket index supports bulk insert row writer

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8776: URL: https://github.com/apache/hudi/pull/8776#issuecomment-1557400845 ## CI report: * 51b6c66f0a8eb053f9334c4fda01e2b4d2acac8d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1557399470 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 2da0fa482136c6179fd14df82843f32f4a0877e3 Azure:

[GitHub] [hudi] danny0405 commented on issue #5330: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-05-22 Thread via GitHub
danny0405 commented on issue #5330: URL: https://github.com/apache/hudi/issues/5330#issuecomment-1557391885 Did you use the COW table or MOR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] ankitchandnani commented on issue #8672: [SUPPORT] INSERT_OVERWRITE_TABLE operation not working on Hudi 0.12.2 using EMR Deltastreamer

2023-05-22 Thread via GitHub
ankitchandnani commented on issue #8672: URL: https://github.com/apache/hudi/issues/8672#issuecomment-1557359620 Any update here @ad1happy2go @codope -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] nsivabalan closed pull request #8231: [HUDI-5963] Release 0.13.1 prep

2023-05-22 Thread via GitHub
nsivabalan closed pull request #8231: [HUDI-5963] Release 0.13.1 prep URL: https://github.com/apache/hudi/pull/8231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #8768: [HUDI-1407] Basic python reader for Hudi

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8768: URL: https://github.com/apache/hudi/pull/8768#issuecomment-1557324869 ## CI report: * 0e64c972e4ba1c307e822a4b0c5344be9eb9d139 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1557323864 ## CI report: * c8cf2d86b1be30d3215b3b6e89b8bda33a1fe5dc UNKNOWN * 333d9faa53e71ba535a7cb8c60ce8b350a33452c UNKNOWN * 2da0fa482136c6179fd14df82843f32f4a0877e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8618: [HUDI-4944] Don't decode URI twice in HoodieBootstrapRDD

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8618: URL: https://github.com/apache/hudi/pull/8618#issuecomment-1557323707 ## CI report: * b1d38ff6b1cc82f5cb90c57658a68ae7463c20fe UNKNOWN * 662e3a540b5450d65f3af5a6d5fa80a45bbefff7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8768: [HUDI-1407] Basic python reader for Hudi

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8768: URL: https://github.com/apache/hudi/pull/8768#issuecomment-1557310913 ## CI report: * 0e64c972e4ba1c307e822a4b0c5344be9eb9d139 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8778: [MINOR] Avoid synchronized block in HoodieLockMetrics if key is present in cache

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8778: URL: https://github.com/apache/hudi/pull/8778#issuecomment-1557311155 ## CI report: * e27ac9168cce2c695173ca0d9110cdecbc3164b5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1557310704 ## CI report: * 8322a8bbc53406a7c997898b3ce376c773d183ca Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8618: [HUDI-4944] Don't decode URI twice in HoodieBootstrapRDD

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8618: URL: https://github.com/apache/hudi/pull/8618#issuecomment-1557309539 ## CI report: * b1d38ff6b1cc82f5cb90c57658a68ae7463c20fe UNKNOWN * 662e3a540b5450d65f3af5a6d5fa80a45bbefff7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1557308690 ## CI report: * fe98254bcb3001cdb13409ff53a6f1958bd365f9 UNKNOWN * 1f9f158675ba301312206710df2fad27982bc0b3 UNKNOWN * 78d54f595beb56afcafc17cbab200d1ef479eaa3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8778: [MINOR] Avoid synchronized block in HoodieLockMetrics if key is present in cache

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8778: URL: https://github.com/apache/hudi/pull/8778#issuecomment-1557294331 ## CI report: * e27ac9168cce2c695173ca0d9110cdecbc3164b5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1557293967 ## CI report: * 8322a8bbc53406a7c997898b3ce376c773d183ca Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1557291995 ## CI report: * 1b207dfb87f2e63eca74f81b85e4effa41794e2b UNKNOWN * 708a1ba81f47ec600e819d74bd6547e4a9d2258d UNKNOWN * 4d4a079d4b113aa82b59396ca5302c12299a1701 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1557291839 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 197d58ce002e65cbe5969b2193fb0e8dffe7eac2 Azure:

[GitHub] [hudi] codope commented on a diff in pull request #8618: [HUDI-4944] Don't decode URI twice in HoodieBootstrapRDD

2023-05-22 Thread via GitHub
codope commented on code in PR #8618: URL: https://github.com/apache/hudi/pull/8618#discussion_r1200539059 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/bootstrap/translator/DecodedBootstrapPartitionPathTranslator.java: ## @@ -0,0 +1,37 @@ +/* + *

[jira] [Closed] (HUDI-5520) Fail MDT when list of log files grows unboundedly

2023-05-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5520. - Resolution: Fixed > Fail MDT when list of log files grows unboundedly >

[GitHub] [hudi] parisni commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

2023-05-22 Thread via GitHub
parisni commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1200536081 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala: ## @@ -59,32 +59,32 @@ private[sql] object SchemaConverters {

[GitHub] [hudi] devanshguptatrepp commented on issue #8777: [SUPPORT] Meta sync error when trying to write to s3 bucket

2023-05-22 Thread via GitHub
devanshguptatrepp commented on issue #8777: URL: https://github.com/apache/hudi/issues/8777#issuecomment-1557221338 `SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in

[GitHub] [hudi] LiJie20190102 opened a new pull request, #8778: avoid synchronized block in HoodieLockMetrics if key is present in cache

2023-05-22 Thread via GitHub
LiJie20190102 opened a new pull request, #8778: URL: https://github.com/apache/hudi/pull/8778 ### Change Logs Avoids acquiring a lock to check whether a value is present in a cache to allow better performance when the value is already in the cache. ### Impact 1、This

[jira] [Updated] (HUDI-6068) Improve logic of getOldestInstantToRetainForClustering when archive timeline

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6068: Fix Version/s: 0.13.1 > Improve logic of getOldestInstantToRetainForClustering when archive timeline >

[jira] [Updated] (HUDI-6068) Improve logic of getOldestInstantToRetainForClustering when archive timeline

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6068: Status: In Progress (was: Open) > Improve logic of getOldestInstantToRetainForClustering when archive

[jira] [Resolved] (HUDI-6068) Improve logic of getOldestInstantToRetainForClustering when archive timeline

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6068. - > Improve logic of getOldestInstantToRetainForClustering when archive timeline >

[jira] [Updated] (HUDI-6132) Fix multiple streaming writers w/ streaming sink

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6132: Fix Version/s: 0.13.1 (was: 0.14.0) > Fix multiple streaming writers w/ streaming

[jira] [Updated] (HUDI-6222) ParquetSchemaConverter shoud always convert the Map key type as not nullable

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6222: Fix Version/s: 0.13.1 (was: 0.14.0) > ParquetSchemaConverter shoud always convert

[jira] [Updated] (HUDI-6134) prevent multi clean run concurrently in flink

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6134: Fix Version/s: 0.13.1 (was: 0.14.0) > prevent multi clean run concurrently in flink

[jira] [Resolved] (HUDI-6199) CDC payload with op field for deletes do not work

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6199. - > CDC payload with op field for deletes do not work > - > >

[jira] [Updated] (HUDI-6199) CDC payload with op field for deletes do not work

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6199: Status: In Progress (was: Open) > CDC payload with op field for deletes do not work >

[jira] [Updated] (HUDI-6204) Add Spark 3.3.2 in bundle validation

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6204: Status: In Progress (was: Open) > Add Spark 3.3.2 in bundle validation >

[jira] [Resolved] (HUDI-6204) Add Spark 3.3.2 in bundle validation

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6204. - > Add Spark 3.3.2 in bundle validation > > > Key:

[jira] [Updated] (HUDI-6196) Keep compatibility for old version archival instants without ACTION_STATE field

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6196: Fix Version/s: 0.13.1 (was: 0.14.0) > Keep compatibility for old version archival

[jira] [Updated] (HUDI-6047) Clustering operation on consistent hashing resulting in duplicate data

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6047: Fix Version/s: 0.13.1 (was: 0.14.0) > Clustering operation on consistent hashing

[jira] [Resolved] (HUDI-5816) Avoid loading archived timeline during meta sync

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-5816. - > Avoid loading archived timeline during meta sync > > >

[jira] [Updated] (HUDI-5816) Avoid loading archived timeline during meta sync

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5816: Status: In Progress (was: Open) > Avoid loading archived timeline during meta sync >

[jira] [Resolved] (HUDI-6174) Fix flaky test testCleanerDeleteReplacedDataWithArchive

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6174. - > Fix flaky test testCleanerDeleteReplacedDataWithArchive >

[jira] [Updated] (HUDI-6027) Unnecessary scala-maven-plugin causes build issue with JDK17

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6027: Fix Version/s: 0.13.1 > Unnecessary scala-maven-plugin causes build issue with JDK17 >

[jira] [Updated] (HUDI-6027) Unnecessary scala-maven-plugin causes build issue with JDK17

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6027: Status: Open (was: In Progress) > Unnecessary scala-maven-plugin causes build issue with JDK17 >

[jira] [Updated] (HUDI-6027) Unnecessary scala-maven-plugin causes build issue with JDK17

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6027: Status: In Progress (was: Open) > Unnecessary scala-maven-plugin causes build issue with JDK17 >

[jira] [Resolved] (HUDI-6027) Unnecessary scala-maven-plugin causes build issue with JDK17

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6027. - > Unnecessary scala-maven-plugin causes build issue with JDK17 >

[jira] [Updated] (HUDI-6027) Unnecessary scala-maven-plugin causes build issue with JDK17

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6027: Status: In Progress (was: Open) > Unnecessary scala-maven-plugin causes build issue with JDK17 >

[jira] [Updated] (HUDI-6184) Improve the test on incremental queries

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6184: Status: In Progress (was: Open) > Improve the test on incremental queries >

[jira] [Resolved] (HUDI-6184) Improve the test on incremental queries

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6184. - > Improve the test on incremental queries > --- > > Key:

[jira] [Updated] (HUDI-6127) Flink Hudi Support Commit on empty batch

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6127: Fix Version/s: 0.13.1 (was: 0.14.0) > Flink Hudi Support Commit on empty batch >

[jira] [Resolved] (HUDI-6090) Optimise payload size for list of FileGroupDTO

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-6090. - > Optimise payload size for list of FileGroupDTO > -- > >

[jira] [Updated] (HUDI-6090) Optimise payload size for list of FileGroupDTO

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6090: Status: In Progress (was: Open) > Optimise payload size for list of FileGroupDTO >

[jira] [Updated] (HUDI-6090) Optimise payload size for list of FileGroupDTO

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6090: Fix Version/s: 0.13.1 > Optimise payload size for list of FileGroupDTO >

[jira] [Updated] (HUDI-6135) FlinkClusteringConfig adds --sort-memory option to support write.sort.memory config

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6135: Fix Version/s: 0.13.1 (was: 0.14.0) > FlinkClusteringConfig adds --sort-memory

[jira] [Updated] (HUDI-4920) fix PartialUpdatePayload cannot return deleted record in preCombine function issue

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4920: Fix Version/s: 0.13.1 > fix PartialUpdatePayload cannot return deleted record in preCombine function >

[jira] [Resolved] (HUDI-4920) fix PartialUpdatePayload cannot return deleted record in preCombine function issue

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang resolved HUDI-4920. - > fix PartialUpdatePayload cannot return deleted record in preCombine function > issue >

[jira] [Updated] (HUDI-4920) fix PartialUpdatePayload cannot return deleted record in preCombine function issue

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4920: Status: In Progress (was: Open) > fix PartialUpdatePayload cannot return deleted record in preCombine

<    1   2   3   4   5   >