[jira] [Closed] (HUDI-5155) hive reading rt table will get duplicate record

[ https://issues.apache.org/jira/browse/HUDI-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-5155. - Fix Version/s: 0.13.0 Resolution: Fixed Fixed by [https://github.com/apache/hudi/commit/fe43e6f85d

[GitHub] [hudi] codope closed issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

codope closed issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records. URL: https://github.com/apache/hudi/issues/4618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hudi] jtmzheng closed issue #8362: [SUPPORT] Table services with optimistic concurrency control and multiple writers

jtmzheng closed issue #8362: [SUPPORT] Table services with optimistic concurrency control and multiple writers URL: https://github.com/apache/hudi/issues/8362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] jtmzheng commented on issue #8362: [SUPPORT] Table services with optimistic concurrency control and multiple writers

jtmzheng commented on issue #8362: URL: https://github.com/apache/hudi/issues/8362#issuecomment-1527065290 Yep, seems fine so far, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [hudi] codope commented on issue #4618: [SUPPORT] When querying a hudi table in hive, there have duplicated records.

codope commented on issue #4618: URL: https://github.com/apache/hudi/issues/4618#issuecomment-1527065053 I could not reproduce the issue in latest master with Spark (original issue reported with Flink but main issue lies in hive input format). I think it was fixed by https://github.com/apa

[GitHub] [hudi] nsivabalan commented on issue #8132: [SUPPORT] data loss in new base file after compaction

nsivabalan commented on issue #8132: URL: https://github.com/apache/hudi/issues/8132#issuecomment-1527027664 hey @coffee34 : can you help us w/ any more info on this end. we are taking a serious look into all data consistency issues. So, interested in getting to the bottom of it. -- Thi

[GitHub] [hudi] nsivabalan closed issue #7733: [SUPPORT] Duplicate rows found in Hudi non partitioned table.

nsivabalan closed issue #7733: [SUPPORT] Duplicate rows found in Hudi non partitioned table. URL: https://github.com/apache/hudi/issues/7733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [hudi] nsivabalan commented on issue #7733: [SUPPORT] Duplicate rows found in Hudi non partitioned table.

nsivabalan commented on issue #7733: URL: https://github.com/apache/hudi/issues/7733#issuecomment-1527022328 The linked patch contains unit tests. also, tried reproducing locally w/ some failures as well. could not reproduce. closing the issue as not valid anymore. -- This is an auto

[GitHub] [hudi] hudi-bot commented on pull request #8592: [MINOR] Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyBatch

hudi-bot commented on PR #8592: URL: https://github.com/apache/hudi/pull/8592#issuecomment-1527011496 ## CI report: * f3c45931ee531eb35d339a0844cda68cb297cc7c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] ad1happy2go commented on issue #6869: [SUPPORT] Incremental upsert or merge is not working

ad1happy2go commented on issue #6869: URL: https://github.com/apache/hudi/issues/6869#issuecomment-1527009817 Closing this issue As we were not able to reproduce even with earlier version. Please reopen if you see the issue. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] nsivabalan closed issue #6869: [SUPPORT] Incremental upsert or merge is not working

nsivabalan closed issue #6869: [SUPPORT] Incremental upsert or merge is not working URL: https://github.com/apache/hudi/issues/6869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[jira] [Updated] (HUDI-6072) Fix NPE when upsert merger and null map or array

[ https://issues.apache.org/jira/browse/HUDI-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6072: - Fix Version/s: 0.13.1 0.14.0 > Fix NPE when upsert merger and null map or array > -

[jira] [Closed] (HUDI-6072) Fix NPE when upsert merger and null map or array

[ https://issues.apache.org/jira/browse/HUDI-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6072. Resolution: Fixed Fixed via master branch: 9f8d4d0130dbe78598f24f00e7fa75c13737fc79 > Fix NPE when upsert m

[hudi] branch master updated: [HUDI-6072] Fix NPE when upsert merger and null map or array (#8432)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9f8d4d0130d [HUDI-6072] Fix NPE when upsert mer

[GitHub] [hudi] danny0405 merged pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

danny0405 merged PR #8432: URL: https://github.com/apache/hudi/pull/8432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] danny0405 closed issue #8431: [SUPPORT] NPE with MapType and new hudi merger

danny0405 closed issue #8431: [SUPPORT] NPE with MapType and new hudi merger URL: https://github.com/apache/hudi/issues/8431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] nsivabalan closed issue #5869: [SUPPORT] There are duplicate values in HUDI MOR table for different partition and not updating values in same partition for GLOBAL_BLOOM

nsivabalan closed issue #5869: [SUPPORT] There are duplicate values in HUDI MOR table for different partition and not updating values in same partition for GLOBAL_BLOOM URL: https://github.com/apache/hudi/issues/5869 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] nsivabalan commented on issue #5869: [SUPPORT] There are duplicate values in HUDI MOR table for different partition and not updating values in same partition for GLOBAL_BLOOM

nsivabalan commented on issue #5869: URL: https://github.com/apache/hudi/issues/5869#issuecomment-1527003611 From what I can glean from the description, looks like the query is a RO query and update partition path is set to true. So, w/ 2nd commit, the delete record went to a log file in pa

[GitHub] [hudi] nsivabalan commented on issue #5869: [SUPPORT] There are duplicate values in HUDI MOR table for different partition and not updating values in same partition for GLOBAL_BLOOM

nsivabalan commented on issue #5869: URL: https://github.com/apache/hudi/issues/5869#issuecomment-1527002010 Not reproducible w/ https://github.com/apache/hudi/pull/8490 ``` import org.apache.spark.sql.SaveMode._ import org.apache.hudi.DataSourceReadOptions._ import org.ap

[GitHub] [hudi] nsivabalan commented on issue #6591: [SUPPORT]Duplicate records in MOR

nsivabalan commented on issue #6591: URL: https://github.com/apache/hudi/issues/6591#issuecomment-1526991688 Its already fixed w/ this patch https://github.com/apache/hudi/pull/8490 ``` scala> spark.read.format("hudi").load(basePath).show(false) +---+

[GitHub] [hudi] nsivabalan closed issue #6591: [SUPPORT]Duplicate records in MOR

nsivabalan closed issue #6591: [SUPPORT]Duplicate records in MOR URL: https://github.com/apache/hudi/issues/6591 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hudi] hudi-bot commented on pull request #8594: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers

hudi-bot commented on PR #8594: URL: https://github.com/apache/hudi/pull/8594#issuecomment-1526984708 ## CI report: * 4cae8a025b6427876f2bdc3d618e315ca59c547d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1672

[GitHub] [hudi] hudi-bot commented on pull request #8594: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers

hudi-bot commented on PR #8594: URL: https://github.com/apache/hudi/pull/8594#issuecomment-1526979733 ## CI report: * 4cae8a025b6427876f2bdc3d618e315ca59c547d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xccui opened a new pull request, #8594: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers

xccui opened a new pull request, #8594: URL: https://github.com/apache/hudi/pull/8594 ### Change Logs When a global failover for a Flink job is triggered, it's safer to recreate a new `StreamWriteOperatorCoordinator`. Otherwise, all exceptions caused by the coordinator itself will no

[GitHub] [hudi] prm-xingcan closed pull request #8593: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers

prm-xingcan closed pull request #8593: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers URL: https://github.com/apache/hudi/pull/8593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[jira] [Updated] (HUDI-6148) Recreate StreamWriteOperatorCoordinator for global failovers

[ https://issues.apache.org/jira/browse/HUDI-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6148: - Labels: pull-request-available (was: ) > Recreate StreamWriteOperatorCoordinator for global failo

[GitHub] [hudi] prm-xingcan opened a new pull request, #8593: [HUDI-6148] Recreate StreamWriteOperatorCoordinator for global failovers

prm-xingcan opened a new pull request, #8593: URL: https://github.com/apache/hudi/pull/8593 ### Change Logs When a global failover for a Flink job is triggered, it's safer to recreate a new `StreamWriteOperatorCoordinator`. Otherwise, all exceptions caused by the coordinator itself w

[jira] [Created] (HUDI-6148) Recreate StreamWriteOperatorCoordinator for global failovers

Xingcan Cui created HUDI-6148: - Summary: Recreate StreamWriteOperatorCoordinator for global failovers Key: HUDI-6148 URL: https://issues.apache.org/jira/browse/HUDI-6148 Project: Apache Hudi Iss

[GitHub] [hudi] hudi-bot commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

hudi-bot commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1526944107 ## CI report: * 4ae48627662446f99d5aa84dae43725ea4e7a579 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526943843 ## CI report: * 6d15b09d5d9fff2e0cdbb15faa73f62ad2e5e852 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=167

[GitHub] [hudi] hudi-bot commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

hudi-bot commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1526943724 ## CI report: * 8a67f296a325ef968f5a29ac5cd0c75a0f7c83c6 UNKNOWN * 22fb6052ee84211a79a360d04600c98697d80afa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field

hudi-bot commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1526942779 ## CI report: * 6c5e047614106881b40c05cbaf4972e82a7c9440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

hudi-bot commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1526942681 ## CI report: * 650d19166b9d80abed32b07abf49dd4fb087aeeb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1657

[GitHub] [hudi] danny0405 commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

danny0405 commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1526941833 The failed test case: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=16711&view=logs&j=3b6e910d-b98f-5de6-b9cb-1e5ff571f5de&t=30b5aae4-0ea0-5566-42d0-febf71a

[GitHub] [hudi] hudi-bot commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

hudi-bot commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1526939303 ## CI report: * 4ae48627662446f99d5aa84dae43725ea4e7a579 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

hudi-bot commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1526938835 ## CI report: * 8a67f296a325ef968f5a29ac5cd0c75a0f7c83c6 UNKNOWN * 22fb6052ee84211a79a360d04600c98697d80afa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field

hudi-bot commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1526937817 ## CI report: * 6c5e047614106881b40c05cbaf4972e82a7c9440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

hudi-bot commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1526937691 ## CI report: * 650d19166b9d80abed32b07abf49dd4fb087aeeb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1657

[jira] [Closed] (HUDI-6114) Rollback handling in AbstractHoodieLogRecordReader may not work correctly when multi-writer is enabled

[ https://issues.apache.org/jira/browse/HUDI-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6114. Resolution: Fixed Fixed via master branch: a8c7d48325a74663dd4a4de8d1d8c0407eb7c258 > Rollback handling in

[jira] [Updated] (HUDI-6114) Rollback handling in AbstractHoodieLogRecordReader may not work correctly when multi-writer is enabled

[ https://issues.apache.org/jira/browse/HUDI-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6114: - Fix Version/s: 0.13.1 0.14.0 > Rollback handling in AbstractHoodieLogRecordReader may n

[GitHub] [hudi] danny0405 commented on pull request #8523: [HUDI-6114] Fixed rollback of blocks in scanInternalV1

danny0405 commented on PR #8523: URL: https://github.com/apache/hudi/pull/8523#issuecomment-1526936486 The failed test: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=16702&view=logs&j=3b6e910d-b98f-5de6-b9cb-1e5ff571f5de&t=30b5aae4-0ea0-5566-42d0-febf71a7061a

[GitHub] [hudi] rohan-uptycs commented on a diff in pull request #8503: [HUDI-6047] Clustering operation on consistent hashing index resulting in duplicate data

rohan-uptycs commented on code in PR #8503: URL: https://github.com/apache/hudi/pull/8503#discussion_r1179895882 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -509,7 +509,15 @@ private Stream getCommitInstantsToArchive(

[hudi] branch master updated (77039ae734a -> a8c7d48325a)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 77039ae734a [HUDI-5517] HoodieTimeline support filter instants by state transition time (#7627) add a8c7d48325a

[GitHub] [hudi] danny0405 merged pull request #8523: [HUDI-6114] Fixed rollback of blocks in scanInternalV1

danny0405 merged PR #8523: URL: https://github.com/apache/hudi/pull/8523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1526934343 ## CI report: * 75d4c5b67703349573569677d88deb2b1eb647fd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526934101 ## CI report: * 6d15b09d5d9fff2e0cdbb15faa73f62ad2e5e852 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=167

[GitHub] [hudi] danny0405 commented on issue #7602: [SUPPORT] When does the Spark engine's bulk insert mode support bucket index

danny0405 commented on issue #7602: URL: https://github.com/apache/hudi/issues/7602#issuecomment-1526921599 Is this the fix you want? https://github.com/apache/hudi/pull/7834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] hudi-bot commented on pull request #8592: [MINOR] Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyBatch

hudi-bot commented on PR #8592: URL: https://github.com/apache/hudi/pull/8592#issuecomment-1526903880 ## CI report: * f3c45931ee531eb35d339a0844cda68cb297cc7c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526903651 ## CI report: * e529b3409ce663965bd925e7389a99f323d3ef2d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] slfan1989 commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

slfan1989 commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526902500 @danny0405 Thank you very much for helping to review the code and providing suggestions for improvement! I have refactored this part of the code using `StringBuilder`, as we discussed, to

[GitHub] [hudi] hudi-bot commented on pull request #8592: [MINOR] Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyBatch

hudi-bot commented on PR #8592: URL: https://github.com/apache/hudi/pull/8592#issuecomment-1526899039 ## CI report: * f3c45931ee531eb35d339a0844cda68cb297cc7c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526898655 ## CI report: * e529b3409ce663965bd925e7389a99f323d3ef2d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #8591: [MINOR] Enhancing validate staged bundles script

hudi-bot commented on PR #8591: URL: https://github.com/apache/hudi/pull/8591#issuecomment-1526892070 ## CI report: * 3f368d58afd0982615405d965d29422b57a1 UNKNOWN * d7a6f5fbb3c280f2c2baaa76cc77d33b247730a3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7632: [HUDI-3775] Allow for offline compaction of MOR tables via spark streaming

hudi-bot commented on PR #7632: URL: https://github.com/apache/hudi/pull/7632#issuecomment-1526891136 ## CI report: * 07f5163e41234081bb0c9f9cd9885ed150487d06 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] stream2000 opened a new pull request, #8592: [MINOR] Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyBatch

stream2000 opened a new pull request, #8592: URL: https://github.com/apache/hudi/pull/8592 ### Change Logs Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyBatch. Now will send all write events before committing the empty instant ### Impact _Describe any publi

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1526863201 ## CI report: * a79402c2cb4e8482b02cd8aaad613774703ceb3d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1663

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1526858818 ## CI report: * a79402c2cb4e8482b02cd8aaad613774703ceb3d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1663

[GitHub] [hudi] hudi-bot commented on pull request #8590: [HUDI-3545] [UBER] Make HoodieAvroWriteSupport class configurable

hudi-bot commented on PR #8590: URL: https://github.com/apache/hudi/pull/8590#issuecomment-1526854211 ## CI report: * c9ffac46b68ebb82eb65cb867d579345a2abeac9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8591: [MINOR] Enhancing validate staged bundles script

hudi-bot commented on PR #8591: URL: https://github.com/apache/hudi/pull/8591#issuecomment-1526814179 ## CI report: * 3f368d58afd0982615405d965d29422b57a1 UNKNOWN * d7a6f5fbb3c280f2c2baaa76cc77d33b247730a3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1526812836 ## CI report: * 3cfef7fc92a6c5ce9bb078a7186e04614c11647f UNKNOWN * 76394b7ed5df01286d5085e5f1b43a47e52baa5d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8591: [MINOR] Enhancing validate staged bundles script

hudi-bot commented on PR #8591: URL: https://github.com/apache/hudi/pull/8591#issuecomment-1526782981 ## CI report: * 3f368d58afd0982615405d965d29422b57a1 UNKNOWN * d7a6f5fbb3c280f2c2baaa76cc77d33b247730a3 UNKNOWN Bot commands @hudi-bot supports the following

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1526782528 ## CI report: * c6908a16bf2f1fb46735781f8d969177eadc23a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1666

[GitHub] [hudi] hudi-bot commented on pull request #8591: [MINOR] Enhancing validate staged bundles script

hudi-bot commented on PR #8591: URL: https://github.com/apache/hudi/pull/8591#issuecomment-1526772068 ## CI report: * 3f368d58afd0982615405d965d29422b57a1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

hudi-bot commented on PR #8303: URL: https://github.com/apache/hudi/pull/8303#issuecomment-1526770534 ## CI report: * c6908a16bf2f1fb46735781f8d969177eadc23a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1666

[GitHub] [hudi] nsivabalan opened a new pull request, #8591: [MINOR] Enhancing validate staged bundles script

nsivabalan opened a new pull request, #8591: URL: https://github.com/apache/hudi/pull/8591 ### Change Logs Enhancing validate staged bundles script ### Impact Enhancing validate staged bundles script ### Risk level (write none, low medium or high below) none

[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion

hudi-bot commented on PR #8589: URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526758337 ## CI report: * 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #7632: [HUDI-3775] Allow for offline compaction of MOR tables via spark streaming

hudi-bot commented on PR #7632: URL: https://github.com/apache/hudi/pull/7632#issuecomment-1526754977 ## CI report: * c7fce967ece6f3cef5cb34ddc2ff7ddffa66727f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1656

[GitHub] [hudi] hudi-bot commented on pull request #7632: [HUDI-3775] Allow for offline compaction of MOR tables via spark streaming

hudi-bot commented on PR #7632: URL: https://github.com/apache/hudi/pull/7632#issuecomment-1526671395 ## CI report: * c7fce967ece6f3cef5cb34ddc2ff7ddffa66727f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1656

[GitHub] [hudi] hudi-bot commented on pull request #8590: [HUDI-3545] [UBER] Make HoodieAvroWriteSupport class configurable

hudi-bot commented on PR #8590: URL: https://github.com/apache/hudi/pull/8590#issuecomment-1526659953 ## CI report: * c9ffac46b68ebb82eb65cb867d579345a2abeac9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8590: [HUDI-3545] [UBER] Make HoodieAvroWriteSupport class configurable

hudi-bot commented on PR #8590: URL: https://github.com/apache/hudi/pull/8590#issuecomment-1526645612 ## CI report: * c9ffac46b68ebb82eb65cb867d579345a2abeac9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-3545) Make HoodieAvroWriteSupport class configurable

[ https://issues.apache.org/jira/browse/HUDI-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3545: - Labels: pull-request-available (was: ) > Make HoodieAvroWriteSupport class configurable > ---

[GitHub] [hudi] suryaprasanna opened a new pull request, #8590: [HUDI-3545] [UBER] Make HoodieAvroWriteSupport class configurable

suryaprasanna opened a new pull request, #8590: URL: https://github.com/apache/hudi/pull/8590 ### Change Logs All write support classes should extend HoodieAvoWriteSupport, the change uses Reflection APIs, to load sub classes of HoodieAvroWriteSupport. This approach is useful to over

[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion

hudi-bot commented on PR #8589: URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526415903 ## CI report: * 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

hudi-bot commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1526414671 ## CI report: * 8a67f296a325ef968f5a29ac5cd0c75a0f7c83c6 UNKNOWN * 22fb6052ee84211a79a360d04600c98697d80afa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

hudi-bot commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1526399286 ## CI report: * 4ae48627662446f99d5aa84dae43725ea4e7a579 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1671

[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion

hudi-bot commented on PR #8589: URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526399416 ## CI report: * 9ee4505209c15331987695061f05d0b8e6d06848 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] jonvex opened a new pull request, #8589: [6147] Deltastreamer finish failed compaction before ingestion

jonvex opened a new pull request, #8589: URL: https://github.com/apache/hudi/pull/8589 ### Change Logs Deltastreamer rolls back and does compaction after the deltacommit if it failed previously. Now, if the config `--retry-last-pending-inline-compaction` is set it will do the compact

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1526246719 ## CI report: * 080184949c2e1ece7e272e71f0844597a41c8a0b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[jira] [Created] (HUDI-6147) Retry Pending Compactions before ingestion in Deltastreamer

Jonathan Vexler created HUDI-6147: - Summary: Retry Pending Compactions before ingestion in Deltastreamer Key: HUDI-6147 URL: https://issues.apache.org/jira/browse/HUDI-6147 Project: Apache Hudi

[GitHub] [hudi] sydneyhoran commented on pull request #7913: Adding support for EPOCHMICROSECONDS in TimestampBasedAvroKeyGenerator

sydneyhoran commented on PR #7913: URL: https://github.com/apache/hudi/pull/7913#issuecomment-1526194783 @bvaradar Hi there I just realized I made this PR using my old github account so I've missed these notifications. I will look at addressing this test case as soon as I have a chance. -

[GitHub] [hudi] nsivabalan commented on pull request #8588: [MINOR] Release 0.12.3 script update

nsivabalan commented on PR #8588: URL: https://github.com/apache/hudi/pull/8588#issuecomment-1526166427 When we go for 0.12.4, we might need these scripts since 0.12.x line has diff bundles compared to 0.13.0. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] nsivabalan opened a new pull request, #8588: [MINOR] Release 0.12.3 script update

nsivabalan opened a new pull request, #8588: URL: https://github.com/apache/hudi/pull/8588 ### Change Logs Some release scripts updates for 0.12.3 ### Impact Some release scripts updates for 0.12.3 ### Risk level (write none, low medium or high below) low

[GitHub] [hudi] hudi-bot commented on pull request #8514: [HUDI-6113] Support multiple transformers using the same config keys in DeltaStreamer

hudi-bot commented on PR #8514: URL: https://github.com/apache/hudi/pull/8514#issuecomment-1526142239 ## CI report: * 272e1bd85c19968502784da9eadc14624059ed7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7632: [HUDI-3775] Allow for offline compaction of MOR tables via spark streaming

nsivabalan commented on code in PR #7632: URL: https://github.com/apache/hudi/pull/7632#discussion_r1179510832 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala: ## @@ -483,6 +483,15 @@ object DataSourceWriteOptions { + "Thi

[GitHub] [hudi] hudi-bot commented on pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

hudi-bot commented on PR #8478: URL: https://github.com/apache/hudi/pull/8478#issuecomment-1526072712 ## CI report: * e529b3409ce663965bd925e7389a99f323d3ef2d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #7355: [HUDI-5308] Hive query returns null when the where clause has a partition field

hudi-bot commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1526070638 ## CI report: * 6c5e047614106881b40c05cbaf4972e82a7c9440 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] vedantkhandelwal closed issue #8396: [SUPPORT] Cleaner configs not working . Need to clean .hoodie files after certain interval/batch

vedantkhandelwal closed issue #8396: [SUPPORT] Cleaner configs not working . Need to clean .hoodie files after certain interval/batch URL: https://github.com/apache/hudi/issues/8396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] vedantkhandelwal commented on issue #8396: [SUPPORT] Cleaner configs not working . Need to clean .hoodie files after certain interval/batch

vedantkhandelwal commented on issue #8396: URL: https://github.com/apache/hudi/issues/8396#issuecomment-1526014493 I've figure out the issue. We were running our data pipelines on hudi version 8 (from feb 2022 to july 2022) then we migrated it to hudi version 9 (july to mid feb 2023). We no

[GitHub] [hudi] the-other-tim-brown commented on issue #8519: [SUPPORT] Deltastreamer AvroDeserializer failing with java.lang.NullPointerException

the-other-tim-brown commented on issue #8519: URL: https://github.com/apache/hudi/issues/8519#issuecomment-1526004722 @sydneyhoran I'm still trying to come up to speed on the errors you are seeing but I can chime in on the behavior for the PostgresDebeziumSource and the payload. The [sourc

[GitHub] [hudi] hudi-bot commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

hudi-bot commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1525933114 ## CI report: * 8db0bfcd2ce5aee94771f35ceb8c0eeb905d2003 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

hudi-bot commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1525920793 ## CI report: * 8db0bfcd2ce5aee94771f35ceb8c0eeb905d2003 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

hudi-bot commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1525919896 ## CI report: * 3e9388ee9a6edaa6caab4f738b093f82744bc7dc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1665

[GitHub] [hudi] hudi-bot commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

hudi-bot commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1525905963 ## CI report: * 3e9388ee9a6edaa6caab4f738b093f82744bc7dc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1665

[GitHub] [hudi] parisni commented on pull request #8432: [HUDI-6072] Fix NPE when upsert merger and null map or array

parisni commented on PR #8432: URL: https://github.com/apache/hudi/pull/8432#issuecomment-1525833564 @danny0405 added a commit to apply the exact same context as the issue. The previous tests did not fail w/o the patch. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [hudi] xiarixiaoyao commented on pull request #8587: [HUDI-6145] Fix the flink table create schema to be compatible with S…

xiarixiaoyao commented on PR #8587: URL: https://github.com/apache/hudi/pull/8587#issuecomment-1525755093 nice work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[GitHub] [hudi] hudi-bot commented on pull request #8523: [HUDI-6114] Fixed rollback of blocks in scanInternalV1

hudi-bot commented on PR #8523: URL: https://github.com/apache/hudi/pull/8523#issuecomment-1525735385 ## CI report: * 1450f0a041a5b2b5e8730ccc34c320ea7847abf6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1670

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1525693379 ## CI report: * 5e6936d61afac645b8372b9a2564981c5aad9dca Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=167

[GitHub] [hudi] hudi-bot commented on pull request #8503: [HUDI-6047] Clustering operation on consistent hashing index resulting in duplicate data

hudi-bot commented on PR #8503: URL: https://github.com/apache/hudi/pull/8503#issuecomment-1525692925 ## CI report: * 0738d975df341763e384b9ac9bcad14b006c9c47 UNKNOWN * f6a34e0433d9a3f2f7d7c618fe1e94edd3fc82cb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] SteNicholas commented on a diff in pull request #8503: [HUDI-6047] Clustering operation on consistent hashing index resulting in duplicate data

SteNicholas commented on code in PR #8503: URL: https://github.com/apache/hudi/pull/8503#discussion_r1179114163 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -509,7 +509,15 @@ private Stream getCommitInstantsToArchive()

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1525630390 ## CI report: * cf4e7358763e10aab951d16d7270f7592f7c62b0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1664

  1   2   >