[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1632712812 ## CI report: * 9e3c05bdf4ef8cb5bb800ccde85fde085b0d07af Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1850

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632712598 ## CI report: * c6e27b201d5fe4ca6eff34f0efdd5a8da5a3a45d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1850

[GitHub] [hudi] amrishlal commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

amrishlal commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632710887 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1632622262 ## CI report: * 74451cd8ee26909fac98e88850ba352c8c415ef5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1849

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1632606284 ## CI report: * 74451cd8ee26909fac98e88850ba352c8c415ef5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1849

[jira] [Created] (HUDI-6527) Fix restore w/ MOR data table with new rollback logic in MDT

sivabalan narayanan created HUDI-6527: - Summary: Fix restore w/ MOR data table with new rollback logic in MDT Key: HUDI-6527 URL: https://issues.apache.org/jira/browse/HUDI-6527 Project: Apache Hu

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1632520943 ## CI report: * 4162ef786e793a3476010858c3175434803c4341 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1852

[GitHub] [hudi] hudi-bot commented on pull request #8610: [HUDI-6156] Prevent leaving tmp file in timeline when multi process t…

hudi-bot commented on PR #8610: URL: https://github.com/apache/hudi/pull/8610#issuecomment-1632503786 ## CI report: * 0bd8531ac13126771f84db19fa02fbe1828b762d UNKNOWN * d28d8dc0e7a243f6ce8806c7b345ed17ecb4b619 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] danny0405 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

danny0405 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1632449022 @Tomccat3 , schema evolution in Hudi is supported, but not automatically, you still need to alter the table schema manually and maybe restart the writing job. -- This is an automat

[GitHub] [hudi] danny0405 commented on issue #9172: [SUPPORT] java.lang.ClassCastException with incremental query

danny0405 commented on issue #9172: URL: https://github.com/apache/hudi/issues/9172#issuecomment-1632445212 cc @yihua , what is the suggested spark version for GCP now ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] bkosuru commented on issue #9172: [SUPPORT] java.lang.ClassCastException with incremental query

bkosuru commented on issue #9172: URL: https://github.com/apache/hudi/issues/9172#issuecomment-1632434591 Hi Danny, We are running in Spark 3.3.2. That is the Spark version supported in GCP Dataproc batches V 1.1. Any other suggestions? -- This is an automated message from the Apache Git

[jira] [Updated] (HUDI-6526) Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

[ https://issues.apache.org/jira/browse/HUDI-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6526: - Fix Version/s: 0.14.0 > Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive > --

[jira] [Closed] (HUDI-6526) Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

[ https://issues.apache.org/jira/browse/HUDI-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6526. Resolution: Fixed Fixed via master branch: acc63fc4193d06fe0c2d22263e3daebb36c5e812 > Hoodie Hive catalog s

[hudi] branch master updated: [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive (#9177)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new acc63fc4193 [HUDI-6526] Hoodie Hive catalog syn

[GitHub] [hudi] danny0405 merged pull request #9177: [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

danny0405 merged PR #9177: URL: https://github.com/apache/hudi/pull/9177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] KnightChess commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

KnightChess commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1632412166 > There are so many test failures and not sure whether it is related with this PR: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=18510&view=logs&j=dcedfe

[GitHub] [hudi] hudi-bot commented on pull request #9106: [HUDI-6118] Some fixes to improve the MDT and record index code base.

hudi-bot commented on PR #9106: URL: https://github.com/apache/hudi/pull/9106#issuecomment-1632403632 ## CI report: * 16ae34ec0e91811bae11a980749f5b77d048adba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] hudi-bot commented on pull request #9177: [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

hudi-bot commented on PR #9177: URL: https://github.com/apache/hudi/pull/9177#issuecomment-1632340631 ## CI report: * ea0257efed993481349e12f72c602ae2635778b4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1852

[GitHub] [hudi] hudi-bot commented on pull request #8610: [HUDI-6156] Prevent leaving tmp file in timeline when multi process t…

hudi-bot commented on PR #8610: URL: https://github.com/apache/hudi/pull/8610#issuecomment-1632335297 ## CI report: * 0bd8531ac13126771f84db19fa02fbe1828b762d UNKNOWN * bfc9450676a0742bad6e25ee1e56aa240e31fe74 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8610: [HUDI-6156] Prevent leaving tmp file in timeline when multi process t…

hudi-bot commented on PR #8610: URL: https://github.com/apache/hudi/pull/8610#issuecomment-1632304488 ## CI report: * 0bd8531ac13126771f84db19fa02fbe1828b762d UNKNOWN * bfc9450676a0742bad6e25ee1e56aa240e31fe74 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] ad1happy2go commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

ad1happy2go commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1632286239 @Tomccat3 Hudi should support it. Are you facing issues? What kind of schema changes you are having? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] Tomccat3 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1632267132 @danny0405 Hello Danny, can you help me with this problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] Tomccat3 opened a new issue, #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 opened a new issue, #9178: URL: https://github.com/apache/hudi/issues/9178 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-subscr.

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1632225017 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 8429f89c5499d33074a6ed0039e5bd9a2da84024 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-6500) Using the RuntimeReplaceable function in the merge into matched condition reports an error

[ https://issues.apache.org/jira/browse/HUDI-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-6500: -- Description: step1: create table and write some data create table test_table ( id string, name string, age int,

[jira] [Updated] (HUDI-6500) Using the RuntimeReplaceable function in the merge into matched condition reports an error

[ https://issues.apache.org/jira/browse/HUDI-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-6500: -- Attachment: image-2023-07-12-17-51-55-790.png > Using the RuntimeReplaceable function in the merge into matched

[jira] [Updated] (HUDI-6500) Using the RuntimeReplaceable function in the merge into matched condition reports an error

[ https://issues.apache.org/jira/browse/HUDI-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-6500: -- Description: step1: create table and write some data create table test_table ( id string, name string, age int,

[jira] [Updated] (HUDI-6500) Using the RuntimeReplaceable function in the merge into matched condition reports an error

[ https://issues.apache.org/jira/browse/HUDI-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-6500: -- Summary: Using the RuntimeReplaceable function in the merge into matched condition reports an error (was: Using

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1632164381 I think currentInstant = null is necessary, but also Send bootstrap write metadata event to coordinator. If you agree, I am willing to submit this pr. -- This is an automated me

[GitHub] [hudi] ankur334 commented on issue #5942: [SUPPORT] Partial Update on Global Index with BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE

ankur334 commented on issue #5942: URL: https://github.com/apache/hudi/issues/5942#issuecomment-1632162090 Partial Update with some partition column/key is not working as expected. Let's suppose I am currently having following event/message. { "id": 1, "langu

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1632160175 > @voonhous -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1632159235 The following steps can reproduce this error 1. Start the Flink task 2. Before the detail commit, some log files are written to the disk. kill TM 3. After waiting for a detail

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1632123918 ## CI report: * 5bfd94b7cc99f9d2bd746758b42c250394b71616 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=185

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1632123687 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * 8ea8e3a853152fcc0d1cf4d0f53a38565ac33990 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] chandu-1101 commented on issue #9141: [SUPPORT] Example from Hudi Quick start doesnt work!

chandu-1101 commented on issue #9141: URL: https://github.com/apache/hudi/issues/9141#issuecomment-1632104122 I think i am able to narrow down the issue to an extent. 1. I tried adding column by column, one by one to the select query and re ran the insert. At one column things broke.

[GitHub] [hudi] eric9204 closed pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer

eric9204 closed pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer URL: https://github.com/apache/hudi/pull/9176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1632057692 ## CI report: * 98654ed70f92bcbe239b999e733b7aa79c296faf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] hudi-bot commented on pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer

hudi-bot commented on PR #9176: URL: https://github.com/apache/hudi/pull/9176#issuecomment-1632043645 ## CI report: * 4e765c59cbf491b4a593e77b0951d9ab74b516bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] ChiaWeiGithub commented on issue #9174: [SUPPORT]flink sql hudi will create duplicate record on same s3 path when the data size is bigger enough to create new filegroup and different f

ChiaWeiGithub commented on issue #9174: URL: https://github.com/apache/hudi/issues/9174#issuecomment-1632043353 Hello Danny, thank you for promptly replied. Since our business scenario is streaming ingestion, would you mind to let me know how to set index state to mitigated the issue? I

[GitHub] [hudi] stream2000 commented on a diff in pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

stream2000 commented on code in PR #9175: URL: https://github.com/apache/hudi/pull/9175#discussion_r1260732347 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java: ## @@ -408,8 +408,13 @@ private void initInstant(String inst

[GitHub] [hudi] hudi-bot commented on pull request #9177: [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

hudi-bot commented on PR #9177: URL: https://github.com/apache/hudi/pull/9177#issuecomment-1631987747 ## CI report: * ea0257efed993481349e12f72c602ae2635778b4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1852

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1631987667 ## CI report: * 98654ed70f92bcbe239b999e733b7aa79c296faf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] hudi-bot commented on pull request #9177: [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

hudi-bot commented on PR #9177: URL: https://github.com/apache/hudi/pull/9177#issuecomment-1631977122 ## CI report: * ea0257efed993481349e12f72c602ae2635778b4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1631977036 ## CI report: * 98654ed70f92bcbe239b999e733b7aa79c296faf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] voonhous commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

voonhous commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1631975169 @big-doudou Any updates? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] danny0405 commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

danny0405 commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1631973470 There are so many test failures and not sure whether it is related with this PR: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=18510&view=logs&j=dcedfe73-9

[GitHub] [hudi] danny0405 commented on a diff in pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

danny0405 commented on code in PR #9175: URL: https://github.com/apache/hudi/pull/9175#discussion_r1260705497 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java: ## @@ -408,8 +408,13 @@ private void initInstant(String insta

[GitHub] [hudi] hudi-bot commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

hudi-bot commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1631965951 ## CI report: * 5a26a9ab4d8ccd2f9d637c3a79517d44dc75cbbc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1631965268 ## CI report: * a0e53d8ef3f506afca5f25fd22adb70581fae426 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1851

[GitHub] [hudi] danny0405 commented on pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

danny0405 commented on PR #9068: URL: https://github.com/apache/hudi/pull/9068#issuecomment-1631965184 @voonhous you can take a look at the PR first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] hudi-bot commented on pull request #8610: [HUDI-6156] Prevent leaving tmp file in timeline when multi process t…

hudi-bot commented on PR #8610: URL: https://github.com/apache/hudi/pull/8610#issuecomment-1631964372 ## CI report: * 0bd8531ac13126771f84db19fa02fbe1828b762d UNKNOWN * bfc9450676a0742bad6e25ee1e56aa240e31fe74 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] danny0405 commented on pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

danny0405 commented on PR #9068: URL: https://github.com/apache/hudi/pull/9068#issuecomment-1631963965 Yeah, this PR needs more careful review before landing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[jira] [Updated] (HUDI-6526) Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive

[ https://issues.apache.org/jira/browse/HUDI-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6526: - Labels: pull-request-available (was: ) > Hoodie Hive catalog sync timestamp(3) as timestamp type

<    1   2