[jira] [Updated] (HUDI-6670) Fix timeline check in metadata table validator

[ https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6670: - Labels: pull-request-available (was: ) > Fix timeline check in metadata table validator > ---

[GitHub] [hudi] yihua opened a new pull request, #9405: [HUDI-6670] Fix timeline check in metadata table validator

yihua opened a new pull request, #9405: URL: https://github.com/apache/hudi/pull/9405 ### Change Logs This PR fixes the timeline check in metadata table validator. Metadata table validator (`HoodieMetadataTableValidator`) throws the following exception before this fix when ther

[jira] [Assigned] (HUDI-6670) Fix timeline check in metadata table validator

[ https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-6670: --- Assignee: Ethan Guo > Fix timeline check in metadata table validator > --

[jira] [Updated] (HUDI-6670) Fix timeline check in metadata table validator

[ https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6670: Description: Metadata table validator (`HoodieMetadataTableValidator`) throws the following exception when

[jira] [Updated] (HUDI-6670) Fix timeline check in metadata table validator

[ https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6670: Description: Metadata table validator throws the following exception when there is completed rollback and n

[jira] [Created] (HUDI-6670) Fix timeline check in metadata table validator

Ethan Guo created HUDI-6670: --- Summary: Fix timeline check in metadata table validator Key: HUDI-6670 URL: https://issues.apache.org/jira/browse/HUDI-6670 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #9401: [MINOR] Fix consistent hashing bucket index it failure

hudi-bot commented on PR #9401: URL: https://github.com/apache/hudi/pull/9401#issuecomment-1670714461 ## CI report: * d3edb82483f426a2efef2d5536cbfdccf773c7e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9403: Added kafka key as part of hudi metadata columns for JsonKafkaSource

hudi-bot commented on PR #9403: URL: https://github.com/apache/hudi/pull/9403#issuecomment-1670714500 ## CI report: * 12cdd1c8b5897b9c0db6f4f22aff6a7776d219b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] stream2000 commented on pull request #9401: [MINOR] Fix consistent hashing bucket index it failure

stream2000 commented on PR #9401: URL: https://github.com/apache/hudi/pull/9401#issuecomment-1670705657 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[hudi] branch master updated (121edc5757b -> eb2aa784273)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 121edc5757b [HUDI-6587] Check incomplete commit for time travel query (#9280) add eb2aa784273 [MINOR] Moving to 0.1

[GitHub] [hudi] yihua merged pull request #9404: [MINOR] Moving to 0.15.0-SNAPSHOT on master branch.

yihua merged PR #9404: URL: https://github.com/apache/hudi/pull/9404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #9404: [MINOR] Moving to 0.15.0-SNAPSHOT on master branch.

hudi-bot commented on PR #9404: URL: https://github.com/apache/hudi/pull/9404#issuecomment-1670670129 ## CI report: * 59d4aefca6d5d9ea1f18b1e1047c04aad22dded3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9401: [MINOR] Fix consistent hashing bucket index it failure

hudi-bot commented on PR #9401: URL: https://github.com/apache/hudi/pull/9401#issuecomment-1670670037 ## CI report: * d3edb82483f426a2efef2d5536cbfdccf773c7e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9404: [MINOR] Moving to 0.15.0-SNAPSHOT on master branch.

hudi-bot commented on PR #9404: URL: https://github.com/apache/hudi/pull/9404#issuecomment-1670656397 ## CI report: * 59d4aefca6d5d9ea1f18b1e1047c04aad22dded3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] prashantwason opened a new pull request, #9404: [MINOR] Moving to 0.15.0-SNAPSHOT on master branch.

prashantwason opened a new pull request, #9404: URL: https://github.com/apache/hudi/pull/9404 [MINOR] Moving to 0.15.0-SNAPSHOT on master branch. ### Change Logs Changed pom version to 0.15.0-SNAPSHOT ### Impact None ### Risk level (write none, low medium or

[GitHub] [hudi] prashantwason closed pull request #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch.

prashantwason closed pull request #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch. URL: https://github.com/apache/hudi/pull/9400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] hudi-bot commented on pull request #9403: Added kafka key as part of hudi metadata columns for JsonKafkaSource

hudi-bot commented on PR #9403: URL: https://github.com/apache/hudi/pull/9403#issuecomment-1670610202 ## CI report: * 12cdd1c8b5897b9c0db6f4f22aff6a7776d219b9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287896192 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/LegacyArchivedMetaEntryReader.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software F

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287895034 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/LegacyArchivedMetaEntryReader.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software F

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287894094 ## hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtils.java: ## @@ -278,29 +278,16 @@ public static void createRestoreFile(String basePath, Strin

[GitHub] [hudi] SteNicholas commented on pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

SteNicholas commented on PR #9395: URL: https://github.com/apache/hudi/pull/9395#issuecomment-1670603500 @danny0405, the failure of CI is fixed by #9401. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287892832 ## hudi-common/src/main/java/org/apache/hudi/common/util/ArchivedInstantReadSchemas.java: ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [hudi] SteNicholas commented on a diff in pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

SteNicholas commented on code in PR #9395: URL: https://github.com/apache/hudi/pull/9395#discussion_r1287885445 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/common/HoodieFlinkEngineContext.java: ## @@ -102,12 +102,12 @@ public RuntimeContext getRuntimeCo

[GitHub] [hudi] hudi-bot commented on pull request #9403: Added kafka key as part of hudi metadata columns for JsonKafkaSource

hudi-bot commented on PR #9403: URL: https://github.com/apache/hudi/pull/9403#issuecomment-1670584153 ## CI report: * 12cdd1c8b5897b9c0db6f4f22aff6a7776d219b9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287878544 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieArchivedManifest.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287876922 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/LegacyArchivedMetaEntryReader.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software F

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287874876 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -603,53 +327,4 @@ private boolean deleteArchivedInstants(List

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287874128 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -493,17 +224,18 @@ private Stream getCommitInstantsToArchive()

[GitHub] [hudi] prathit06 commented on issue #9391: [ENHANCEMENT] Kafka Key as part of hudi metadata columns

prathit06 commented on issue #9391: URL: https://github.com/apache/hudi/issues/9391#issuecomment-1670575984 Hi guys, I have created this PR : https://github.com/apache/hudi/pull/9403 for the changes. Request to please review it & let me know if any changes needed. Thanks a lot ! -

[GitHub] [hudi] prathit06 opened a new pull request, #9403: Added kafka key as part of hudi metadata columns for JsonKafkaSource

prathit06 opened a new pull request, #9403: URL: https://github.com/apache/hudi/pull/9403 ### Change Logs This changes add capability to add kafka message key as part of hudi metadata columns for JsonKafkaSource For context : https://github.com/apache/hudi/issues/9391 ### Im

[GitHub] [hudi] hudi-bot commented on pull request #9401: [MINOR] Fix consistent hashing bucket index it failure

hudi-bot commented on PR #9401: URL: https://github.com/apache/hudi/pull/9401#issuecomment-1670574390 ## CI report: * d3edb82483f426a2efef2d5536cbfdccf773c7e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287870522 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -846,6 +846,50 @@ public static List getFileStatusAtLevel( return result; } + /** +

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287865885 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timeline;

[GitHub] [hudi] 1032851561 opened a new issue, #9402: [SUPPORT] HiveSync not support schema evolution

1032851561 opened a new issue, #9402: URL: https://github.com/apache/hudi/issues/9402 **Describe the problem you faced** When I was testing schema evolution, I found that some exception occurred in synchronous hive. **Environment Description** * Hudi version : 0.13.1

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287854438 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/ActiveInstant.java: ## @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [hudi] hudi-bot commented on pull request #9401: [MINOR] Fix consistent hashing bucket index it failure

hudi-bot commented on PR #9401: URL: https://github.com/apache/hudi/pull/9401#issuecomment-1670545842 ## CI report: * d3edb82483f426a2efef2d5536cbfdccf773c7e8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287850688 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -164,230 +100,27 @@ public boolean archiveIfRequired(HoodieEngi

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1287849911 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/execution/benchmark/ArchivedTimelineReadBenchmark.scala: ## @@ -0,0 +1,97 @@ +/* + * Licensed to

[GitHub] [hudi] danny0405 commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

danny0405 commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280276081 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timeline;

[GitHub] [hudi] stream2000 opened a new pull request, #9401: [MINOR] Fix consistent hashing bucket index it failure

stream2000 opened a new pull request, #9401: URL: https://github.com/apache/hudi/pull/9401 ### Change Logs Fix it failure introduced by #9199 ### Impact Fix it failure introduced by #9199 ### Risk level (write none, low medium or high below) NONE

[GitHub] [hudi] danny0405 commented on issue #9384: [SUPPORT] TransactionParticipant not getting created

danny0405 commented on issue #9384: URL: https://github.com/apache/hudi/issues/9384#issuecomment-1670517491 I mean it has non relationship with Hudi, there should be some certification issues with Kafka, did you ever reach out to the AWS fellows for help? -- This is an automated message f

[GitHub] [hudi] danny0405 commented on issue #9391: [ENHANCEMENT] Kafka Key as part of hudi metadata columns

danny0405 commented on issue #9391: URL: https://github.com/apache/hudi/issues/9391#issuecomment-1670514673 > My idea is to extend the Hudi metadata columns itself similar to what is done [here](https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilitie

[GitHub] [hudi] danny0405 commented on a diff in pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

danny0405 commented on code in PR #9395: URL: https://github.com/apache/hudi/pull/9395#discussion_r1287825581 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/common/HoodieFlinkEngineContext.java: ## @@ -102,12 +102,12 @@ public RuntimeContext getRuntimeCont

[GitHub] [hudi] hudi-bot commented on pull request #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch.

hudi-bot commented on PR #9400: URL: https://github.com/apache/hudi/pull/9400#issuecomment-1670501163 ## CI report: * 153f45a43734478fa3abbc8475bdf5e9b8d6c94b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] xiedeyantu closed pull request #8999: Change image hudi-hadoop_2.8.4-history version to linux-arm64-0.10.1 adapt to MacOS M1

xiedeyantu closed pull request #8999: Change image hudi-hadoop_2.8.4-history version to linux-arm64-0.10.1 adapt to MacOS M1 URL: https://github.com/apache/hudi/pull/8999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[jira] [Comment Edited] (HUDI-6596) Propose rollback implementation changes to guard against concurrent jobs

[ https://issues.apache.org/jira/browse/HUDI-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751866#comment-17751866 ] Krishen Bhan edited comment on HUDI-6596 at 8/8/23 11:48 PM: -

[hudi] branch master updated: [HUDI-6587] Check incomplete commit for time travel query (#9280)

This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 121edc5757b [HUDI-6587] Check incomplete commit

[GitHub] [hudi] xushiyan merged pull request #9280: [HUDI-6587] Check incomplete commit for time travel query

xushiyan merged PR #9280: URL: https://github.com/apache/hudi/pull/9280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] hudi-bot commented on pull request #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch.

hudi-bot commented on PR #9400: URL: https://github.com/apache/hudi/pull/9400#issuecomment-1670370579 ## CI report: * 153f45a43734478fa3abbc8475bdf5e9b8d6c94b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch.

hudi-bot commented on PR #9400: URL: https://github.com/apache/hudi/pull/9400#issuecomment-1670363131 ## CI report: * 153f45a43734478fa3abbc8475bdf5e9b8d6c94b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] bhasudha commented on pull request #9346: [DOCS] Update Indexing page with all index types and file layout page

bhasudha commented on PR #9346: URL: https://github.com/apache/hudi/pull/9346#issuecomment-1670354026 After splitting the configs into spark based and flink based ones he page looks locally like this: ![Screenshot 2023-08-08 at 2 48 23 PM](https://github.com/apache/hudi/assets/2179254/44

[GitHub] [hudi] bhasudha commented on a diff in pull request #9346: [DOCS] Update Indexing page with all index types and file layout page

bhasudha commented on code in PR #9346: URL: https://github.com/apache/hudi/pull/9346#discussion_r1287719902 ## website/docs/indexing.md: ## @@ -20,34 +24,90 @@ _Figure: Comparison of merge cost for updates (yellow blocks) against base files ## Index Types in Hudi -Current

[GitHub] [hudi] prashantwason opened a new pull request, #9400: [MINOR] Moving to 0.14.1-SNAPSHOT on master branch.

prashantwason opened a new pull request, #9400: URL: https://github.com/apache/hudi/pull/9400 [MINOR] Moving to 0.14.1-SNAPSHOT on master branch. ### Change Logs Change hudi master pom version to 0.14.1-SNAPSHOT ### Impact New version on hudi master ### Risk

[hudi] branch release-0.14.0 created (now dddfe85f1c1)

This is an automated email from the ASF dual-hosted git repository. pwason pushed a change to branch release-0.14.0 in repository https://gitbox.apache.org/repos/asf/hudi.git at dddfe85f1c1 Create release branch for version 0.14.0. This branch includes the following new commits: new

[hudi] 01/01: Create release branch for version 0.14.0.

This is an automated email from the ASF dual-hosted git repository. pwason pushed a commit to branch release-0.14.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit dddfe85f1c13625a291c2c88786cbeb7b03d1691 Author: Prashant Wason AuthorDate: Tue Aug 8 14:12:17 2023 -0700 Cre

svn commit: r63399 - /release/hudi/KEYS

Author: sivabalan Date: Tue Aug 8 19:58:43 2023 New Revision: 63399 Log: Updating Prashanth's keys Modified: release/hudi/KEYS Modified: release/hudi/KEYS == --- release/hudi/KEYS (original) +++ release/hudi/KEYS Tu

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Check incomplete commit for time travel query

hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1670225535 ## CI report: * db7f890cecc78edba4fb76319ace906f0f63a818 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1920

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Check incomplete commit for time travel query

hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1670215305 ## CI report: * db7f890cecc78edba4fb76319ace906f0f63a818 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

svn commit: r63394 - /dev/hudi/KEYS

Author: pwason Date: Tue Aug 8 19:06:36 2023 New Revision: 63394 Log: Adding dev and release keys HUDI developer pwa...@apache.org Modified: dev/hudi/KEYS Modified: dev/hudi/KEYS == --- dev/hudi/KEYS (original) +++

[GitHub] [hudi] nandubatchu opened a new issue, #9399: [SUPPORT] Unable to read column_stats sub-table of a HUDI table for some tables

nandubatchu opened a new issue, #9399: URL: https://github.com/apache/hudi/issues/9399 **Describe the problem you faced** Not able to read the column_stats index table for some of the HUDI tables from spark **To Reproduce** Steps to reproduce the behavior: Enabled

[hudi] branch master updated (46f41d186c6 -> 92e9f73754a)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 46f41d186c6 [MINOR] Make a copy of partitionPath, since UTF8String could be pointing into a mutable underlying buffer (#

[GitHub] [hudi] nickrvieira opened a new issue, #9398: [SUPPORT] DeltaStreamer non-continuous behavior for S3EventsSource + S3EventsHoodieIncrSource

nickrvieira opened a new issue, #9398: URL: https://github.com/apache/hudi/issues/9398 **Issues** I'm finding a bit of trouble in double-checking if this is the expected behavior or even if I'm missing parametrizations for non-continuous pipelines (run-once) with both S3EventsSource

[GitHub] [hudi] nandubatchu opened a new issue, #9397: [SUPPORT] column_stats index filtering returns empty results

nandubatchu opened a new issue, #9397: URL: https://github.com/apache/hudi/issues/9397 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subs

[hudi] branch asf-site updated: [DOCS] Replace 'Breaking Change' with 'Important' (#9396)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new e2f786d1dbc [DOCS] Replace 'Breaking Change' wi

[GitHub] [hudi] yihua merged pull request #9396: [DOCS] Replace 'Breaking Change' with 'Important'

yihua merged PR #9396: URL: https://github.com/apache/hudi/pull/9396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[hudi] branch asf-site updated: [DOCS]Update slack links due to expiry of old link (#9392)

This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new c12f92abdda [DOCS]Update slack links due

[GitHub] [hudi] bhasudha merged pull request #9392: [DOCS]Update slack links due to upcoming expiry of old link

bhasudha merged PR #9392: URL: https://github.com/apache/hudi/pull/9392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] hudi-bot commented on pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

hudi-bot commented on PR #9395: URL: https://github.com/apache/hudi/pull/9395#issuecomment-1669967315 ## CI report: * a60f7f89b5377119bf8bef6c7ddfd0dc821de1fc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9393: [HUDI-6668] CTAS should not clear existing hoodie table path

hudi-bot commented on PR #9393: URL: https://github.com/apache/hudi/pull/9393#issuecomment-1669967246 ## CI report: * f643044c0cb7182a9544e62b5110c66f5aad7bf7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] prathit06 commented on issue #9391: [ENHANCEMENT] Kafka Key as part of hudi metadata columns

prathit06 commented on issue #9391: URL: https://github.com/apache/hudi/issues/9391#issuecomment-1669955523 Hi @vinothchandar My idea is to `extend the Hudi metadata columns` itself similar to what is done [here](https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/jav

[GitHub] [hudi] amrishlal commented on pull request #9396: Replace 'Breaking Change' with 'Important'

amrishlal commented on PR #9396: URL: https://github.com/apache/hudi/pull/9396#issuecomment-1669942071 https://github.com/apache/hudi/assets/4550395/88702e9c-ef1f-4f9d-a7ab-f10757e4699d";> -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] amrishlal opened a new pull request, #9396: Replace 'Breaking Change' with 'Important'

amrishlal opened a new pull request, #9396: URL: https://github.com/apache/hudi/pull/9396 ### Change Logs Replace 'Breaking Change' section heading with 'Important' heading. ### Impact None ### Risk level (write none, low medium or high below) None ##

[GitHub] [hudi] vinothchandar commented on issue #9391: [ENHANCEMENT] Kafka Key as part of hudi metadata columns

vinothchandar commented on issue #9391: URL: https://github.com/apache/hudi/issues/9391#issuecomment-1669897168 >Similarly to above, i would like to add kafka key as well & same can be implemented for other kafka sources as well. To be clear, the proposal is to generate more columns f

[GitHub] [hudi] PhantomHunt commented on issue #9344: [SUPPORT] Getting error when writing to different HUDI tables in different threads in same job

PhantomHunt commented on issue #9344: URL: https://github.com/apache/hudi/issues/9344#issuecomment-1669805569 Any updates on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

hudi-bot commented on PR #9395: URL: https://github.com/apache/hudi/pull/9395#issuecomment-1669694218 ## CI report: * a60f7f89b5377119bf8bef6c7ddfd0dc821de1fc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

hudi-bot commented on PR #9395: URL: https://github.com/apache/hudi/pull/9395#issuecomment-1669677534 ## CI report: * a60f7f89b5377119bf8bef6c7ddfd0dc821de1fc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-6669) HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

[ https://issues.apache.org/jira/browse/HUDI-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6669: - Labels: pull-request-available (was: ) > HoodieEngineContext should not use parallel stream with

[GitHub] [hudi] SteNicholas opened a new pull request, #9395: [HUDI-6669] HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

SteNicholas opened a new pull request, #9395: URL: https://github.com/apache/hudi/pull/9395 ### Change Logs `HoodieEngineContext` should not use parallel stream with parallelism greater than the number of CPU cores to avoid `OutOfMemoryError` of `ForkJoinTask`, of which stacktrace as

[jira] [Updated] (HUDI-6669) HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

[ https://issues.apache.org/jira/browse/HUDI-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang updated HUDI-6669: - Description: HoodieEngineContext should not use parallel stream with parallelism greater than CPU

[jira] [Updated] (HUDI-6669) HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

[ https://issues.apache.org/jira/browse/HUDI-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Jiang updated HUDI-6669: - Description: HoodieEngineContext should not use parallel stream with parallelism greater than CPU

[jira] [Created] (HUDI-6669) HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores

Nicholas Jiang created HUDI-6669: Summary: HoodieEngineContext should not use parallel stream with parallelism greater than CPU cores Key: HUDI-6669 URL: https://issues.apache.org/jira/browse/HUDI-6669

[hudi] branch master updated (7541cd7e6f8 -> 46f41d186c6)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 7541cd7e6f8 [HUDI-6534]Support consistent hashing row writer (#9199) add 46f41d186c6 [MINOR] Make a copy of part

[GitHub] [hudi] danny0405 merged pull request #9394: [HOTFIX] Make a copy of partitionPath, since UTF8String could be pointing into a mutable underlying buffer

danny0405 merged PR #9394: URL: https://github.com/apache/hudi/pull/9394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] beyond1920 opened a new pull request, #9394: [HOTFIX] Make a copy of partitionPath, since UTF8String could be pointing into a mutable underlying buffer

beyond1920 opened a new pull request, #9394: URL: https://github.com/apache/hudi/pull/9394 ### Change Logs Fix bug of bucket bulk insert ### Impact NA ### Risk level (write none, low medium or high below) NA ### Documentation Update _Describe a

[hudi] branch master updated (7102e0fbe5f -> 7541cd7e6f8)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 7102e0fbe5f [HUDI-4987] Rename Hudi Streamer related configs (#9377) add 7541cd7e6f8 [HUDI-6534]Support consistent h

[GitHub] [hudi] leesf merged pull request #9199: [HUDI-6534]Support consistent hashing row writer

leesf merged PR #9199: URL: https://github.com/apache/hudi/pull/9199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] ad1happy2go commented on issue #9390: Hudi Option "hoodie.combine.before.upsert" does not take effect

ad1happy2go commented on issue #9390: URL: https://github.com/apache/hudi/issues/9390#issuecomment-1669412687 @poocb Yes, This was fixed in 0.13. You can use the OSS hudi 0.13.1 meanwhile. I dont think, there is a plan to fix this in next 0.12 minor version. -- This is an automated mes

[GitHub] [hudi] hudi-bot commented on pull request #9393: [HUDI-6668] CTAS should not clear existing hoodie table path

hudi-bot commented on PR #9393: URL: https://github.com/apache/hudi/pull/9393#issuecomment-1669405184 ## CI report: * f643044c0cb7182a9544e62b5110c66f5aad7bf7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1921

[GitHub] [hudi] hudi-bot commented on pull request #9393: [HUDI-6668] CTAS should not clear existing hoodie table path

hudi-bot commented on PR #9393: URL: https://github.com/apache/hudi/pull/9393#issuecomment-1669393469 ## CI report: * f643044c0cb7182a9544e62b5110c66f5aad7bf7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9389: [HUDI-6667] ClientIds should generate next id automatically with random uuid instead of incremental id

hudi-bot commented on PR #9389: URL: https://github.com/apache/hudi/pull/9389#issuecomment-1669382011 ## CI report: * 60318e83c1dd93c4028e46b8a3dc8bb1c9670f8f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1920

[jira] [Updated] (HUDI-6668) CTAS should not clear existing hoodie table path

[ https://issues.apache.org/jira/browse/HUDI-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6668: - Labels: pull-request-available (was: ) > CTAS should not clear existing hoodie table path > -

[GitHub] [hudi] wecharyu opened a new pull request, #9393: [HUDI-6668] CTAS should not clear existing hoodie table path

wecharyu opened a new pull request, #9393: URL: https://github.com/apache/hudi/pull/9393 ### Change Logs Currently Hudi will clear table path if there is any exception in CTAS command: https://github.com/apache/hudi/blob/7102e0fbe5ff352e5cbb123c3c25b1e5cd238d78/hudi-spark-datasource/h

[GitHub] [hudi] bhasudha commented on pull request #9392: [DOCS]Update slack links due to upcoming expiry of old link

bhasudha commented on PR #9392: URL: https://github.com/apache/hudi/pull/9392#issuecomment-1669367545 Tested locally!  -- This is an automated message from the Apache Git Service. T

[jira] [Created] (HUDI-6668) CTAS should not clear existing hoodie table path

Wechar created HUDI-6668: Summary: CTAS should not clear existing hoodie table path Key: HUDI-6668 URL: https://issues.apache.org/jira/browse/HUDI-6668 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] bhasudha opened a new pull request, #9392: [DOCS]Update slack links due to upcoming expiry of old link

bhasudha opened a new pull request, #9392: URL: https://github.com/apache/hudi/pull/9392 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] danny0405 commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

danny0405 commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1286883771 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java: ## @@ -133,7 +154,7 @@ private static String convertPrimitive

[GitHub] [hudi] danny0405 commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

danny0405 commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1286883188 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java: ## @@ -19,40 +19,61 @@ package org.apache.hudi.sync.common.u

[GitHub] [hudi] danny0405 commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

danny0405 commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1286882926 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/Parquet2SparkSchemaUtils.java: ## @@ -19,40 +19,61 @@ package org.apache.hudi.sync.common.u

[GitHub] [hudi] danny0405 commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

danny0405 commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1286881465 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/ddl/QueryBasedDDLExecutor.java: ## @@ -220,5 +221,16 @@ private List constructChangePartitions(String tab

[GitHub] [hudi] hudi-bot commented on pull request #9280: [HUDI-6587] Check incomplete commit for time travel query

hudi-bot commented on PR #9280: URL: https://github.com/apache/hudi/pull/9280#issuecomment-1669289945 ## CI report: * 02495c9db690fb6523e5cc548a1b118060dd8ff8 UNKNOWN * 55a97bc0f7954b0a5ddb61423957b27c08cfa0bb UNKNOWN * db7f890cecc78edba4fb76319ace906f0f63a818 Azure: [FAILUR

  1   2   >