[GitHub] [hudi] bigdata-spec opened a new issue, #8662: [SUPPORT]Spark on K8s can not stop pod by setting set hoodie.write.concurrency.mode=optimistic_concurrency_control;

bigdata-spec opened a new issue, #8662: URL: https://github.com/apache/hudi/issues/8662 Hello, I have meet some error for **spark-on-k8s-operator**. My hudi version is 0.13,spark version is 3.3.2 I find the pod log show the task is Successfully，but pod status is still Running?

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8631: [HUDI-6170] Use correct zone id while calculating earliestTimeToRetain

nsivabalan commented on code in PR #8631: URL: https://github.com/apache/hudi/pull/8631#discussion_r1187089107 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java: ## @@ -510,7 +510,7 @@ public Option getEarliestCommitToRetain() {

[GitHub] [hudi] ad1happy2go commented on issue #8644: [SUPPORT] Data loss while ingesting multiple hudi tables via one glue/spark job with clustering and metadata properties

ad1happy2go commented on issue #8644: URL: https://github.com/apache/hudi/issues/8644#issuecomment-1537836015 @d4r3topk If I have understood it correctly, You are only losing data when you are trying to insert into 3 hudi tables using a single job. Are you using separate checkpoin

[GitHub] [hudi] cxyoob opened a new pull request, #8661: Hudi 6187 modify checkpoints to checkpoint

cxyoob opened a new pull request, #8661: URL: https://github.com/apache/hudi/pull/8661 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[jira] [Created] (HUDI-6187) Modify checkpoints to checkpoint in Procedures doc

bo wen created HUDI-6187: Summary: Modify checkpoints to checkpoint in Procedures doc Key: HUDI-6187 URL: https://issues.apache.org/jira/browse/HUDI-6187 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

hudi-bot commented on PR #7826: URL: https://github.com/apache/hudi/pull/7826#issuecomment-1537819683 ## CI report: * e76f56f4223f346ab0a7cb17ae67beb75d89fa67 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1681

[GitHub] [hudi] stream2000 commented on a diff in pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

stream2000 commented on code in PR #7826: URL: https://github.com/apache/hudi/pull/7826#discussion_r1187063352 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java: ## @@ -545,6 +546,116 @@ public void testMultiWriterWithAsyncTa

[GitHub] [hudi] weimingdiit commented on pull request #8301: [HUDI-5988] Add a param, Implement a full partition sync operation wh…

weimingdiit commented on PR #8301: URL: https://github.com/apache/hudi/pull/8301#issuecomment-1537812570 @XuQianJin-Stars Hi, could you please take a review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hudi] hudi-bot commented on pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

hudi-bot commented on PR #7826: URL: https://github.com/apache/hudi/pull/7826#issuecomment-1537811250 ## CI report: * e76f56f4223f346ab0a7cb17ae67beb75d89fa67 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1681

[GitHub] [hudi] yihua commented on a diff in pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

yihua commented on code in PR #8388: URL: https://github.com/apache/hudi/pull/8388#discussion_r1187059837 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieSyncClient.java: ## @@ -112,44 +113,101 @@ public MessageType getStorageSchema(boolean include

[GitHub] [hudi] stream2000 commented on a diff in pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

stream2000 commented on code in PR #7826: URL: https://github.com/apache/hudi/pull/7826#discussion_r1187056760 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java: ## @@ -545,6 +546,116 @@ public void testMultiWriterWithAsyncTa

[GitHub] [hudi] codope commented on a diff in pull request #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer.

codope commented on code in PR #8520: URL: https://github.com/apache/hudi/pull/8520#discussion_r1187029695 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/ErrorTableUtils.java: ## @@ -28,32 +28,39 @@ import org.apache.hudi.exception.HoodieException; i

[GitHub] [hudi] rahil-c commented on pull request #8082: [HUDI-5868] Upgrade Spark to 3.3.2

rahil-c commented on PR #8082: URL: https://github.com/apache/hudi/pull/8082#issuecomment-1537768291 @xiarixiaoyao Thanks for your analysis. Ive tried adding that code block you linked in this pr. The one thing I am seeing from the tests is a new failure since this "returning_batch"

[GitHub] [hudi] hudi-bot commented on pull request #8399: [HUDI-4630] Add transformer capability to individual feeds in MultiTableDeltaStreamer

hudi-bot commented on PR #8399: URL: https://github.com/apache/hudi/pull/8399#issuecomment-1537765273 ## CI report: * 201246664a237a780795e99897a584eab25b3ee5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8399: [HUDI-4630] Add transformer capability to individual feeds in MultiTableDeltaStreamer

hudi-bot commented on PR #8399: URL: https://github.com/apache/hudi/pull/8399#issuecomment-1537758089 ## CI report: * 201246664a237a780795e99897a584eab25b3ee5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] harsh1231 commented on a diff in pull request #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer.

harsh1231 commented on code in PR #8520: URL: https://github.com/apache/hudi/pull/8520#discussion_r1187019450 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/ErrorTableUtils.java: ## @@ -63,4 +70,27 @@ public static HoodieErrorTableConfig.ErrorWriteFailu

[hudi] branch master updated: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync (#8388)

This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6ef00d147c1 [HUDI-5816] List all partitions as the

[GitHub] [hudi] codope merged pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

codope merged PR #8388: URL: https://github.com/apache/hudi/pull/8388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537736137 yes,It has come into effect，the clustering task appears in DAG,and the replace_commit appears in .hoodie directory. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] harsh1231 opened a new pull request, #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer.

harsh1231 opened a new pull request, #8520: URL: https://github.com/apache/hudi/pull/8520 ### Change Logs This pr adds `ERROR_TABLE_CURRUPT_RECORD_COL_NAME` as null value column if error table is eanbled for transformers and column does not exists in dataset . Adds validation for `ERRO

[GitHub] [hudi] harsh1231 closed pull request #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer.

harsh1231 closed pull request #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer. URL: https://github.com/apache/hudi/pull/8520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hudi] harsh1231 commented on pull request #8520: [HUDI-6115] Hardening expectation of corruptRecordColumn in ChainedTransformer.

harsh1231 commented on PR #8520: URL: https://github.com/apache/hudi/pull/8520#issuecomment-1537727478 > @harsh1231 The newly added tests are failing. Can you please check? https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=16684&view=logs&j=9273dbcb-a208-5cd0-554

[GitHub] [hudi] hudi-bot commented on pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

hudi-bot commented on PR #8660: URL: https://github.com/apache/hudi/pull/8660#issuecomment-1537727259 ## CI report: * a080f12934335a1000ffbeca5ec2ff6f9e4054a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537726867 ## CI report: * b043a8845b6a4d12148b9f45d92638cbe4c1cba7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8190: [HUDI-5936] Fix serialization problem when FileStatus is not serializable

hudi-bot commented on PR #8190: URL: https://github.com/apache/hudi/pull/8190#issuecomment-1537726679 ## CI report: * 687ea9631daa66ba4f0144146b2c73636891bb04 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] danny0405 commented on a diff in pull request #5168: [HUDI-3729][SPARK] fixed the per regression by enable vectorizeReader for parquet file

danny0405 commented on code in PR #5168: URL: https://github.com/apache/hudi/pull/5168#discussion_r1187009757 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala: ## @@ -56,6 +56,11 @@ class MergeOnReadSnapshotRelation(sqlC

[GitHub] [hudi] danny0405 commented on a diff in pull request #5168: [HUDI-3729][SPARK] fixed the per regression by enable vectorizeReader for parquet file

danny0405 commented on code in PR #5168: URL: https://github.com/apache/hudi/pull/5168#discussion_r1187009323 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala: ## @@ -56,6 +56,11 @@ class MergeOnReadSnapshotRelation(sqlC

[GitHub] [hudi] danny0405 commented on pull request #8308: [HUDI-5994] Spark bulk insert with bucket index

danny0405 commented on PR #8308: URL: https://github.com/apache/hudi/pull/8308#issuecomment-1537722512 Reviewing now~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [hudi] hudi-bot commented on pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

hudi-bot commented on PR #8660: URL: https://github.com/apache/hudi/pull/8660#issuecomment-1537722344 ## CI report: * a080f12934335a1000ffbeca5ec2ff6f9e4054a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] danny0405 commented on pull request #8308: [HUDI-5994] Bucket index supports bulk insert mode.

danny0405 commented on PR #8308: URL: https://github.com/apache/hudi/pull/8308#issuecomment-1537719771 Start reviewing now~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[hudi] branch master updated: [HUDI-6186] Fix lock identity in InProcessLockProvider (#8658)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 79412ed2a63 [HUDI-6186] Fix lock identity in InProc

[GitHub] [hudi] yihua merged pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua merged PR #8658: URL: https://github.com/apache/hudi/pull/8658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r118716 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] danny0405 commented on issue #8651: [SUPPORT]How to resolve small file?

danny0405 commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537696123 Does the async clustering happens in normal progress? Can you past the DAG of the running job? -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hudi] danny0405 commented on a diff in pull request #8659: [HUDI-6155] Fix cleaner based on hours for earliest commit to retain

danny0405 commented on code in PR #8659: URL: https://github.com/apache/hudi/pull/8659#discussion_r1186997845 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstantTimeGenerator.java: ## @@ -129,7 +129,7 @@ public static String getInstantForDateString(S

[GitHub] [hudi] danny0405 commented on issue #8653: [SUPPORT] Why we create the partition path if it's doesn't exist?

danny0405 commented on issue #8653: URL: https://github.com/apache/hudi/issues/8653#issuecomment-1537693429 In general we can see it as a bug, the reader should not make any modifications to the table. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] danny0405 commented on pull request #8657: [HUDI-6150] Support bucketing for each hive client

danny0405 commented on PR #8657: URL: https://github.com/apache/hudi/pull/8657#issuecomment-1537691855 > but so far I am not sure what the current status of hudi hashing It uses only simple Java hashcode: https://github.com/apache/hudi/blob/20938c30b168d63cf4e520c6b4e1d7b930bed1ab/

[GitHub] [hudi] codope commented on a diff in pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

codope commented on code in PR #8388: URL: https://github.com/apache/hudi/pull/8388#discussion_r1186990025 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieSyncClient.java: ## @@ -112,44 +113,101 @@ public MessageType getStorageSchema(boolean includ

[GitHub] [hudi] watermelon12138 commented on pull request #8308: [HUDI-5994] Bucket index supports bulk insert mode.

watermelon12138 commented on PR #8308: URL: https://github.com/apache/hudi/pull/8308#issuecomment-1537680201 @XuQianJin-Stars Can you review my code? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] voonhous commented on issue #8653: [SUPPORT] Why we create the partition path if it's doesn't exist?

voonhous commented on issue #8653: URL: https://github.com/apache/hudi/issues/8653#issuecomment-1537680061 https://github.com/apache/hudi/pull/3769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] hudi-bot commented on pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

hudi-bot commented on PR #8658: URL: https://github.com/apache/hudi/pull/8658#issuecomment-1537674038 ## CI report: * 6b40dd8376d8874e224be04c4a4667f23546692d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8190: [HUDI-5936] Fix serialization problem when FileStatus is not serializable

hudi-bot commented on PR #8190: URL: https://github.com/apache/hudi/pull/8190#issuecomment-1537673187 ## CI report: * 7c71b63797be01ee91268c2520f82b18b3f13b7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1676

[GitHub] [hudi] danny0405 commented on pull request #8472: [HUDI-5298] Optimize WriteStatus storing HoodieRecord

danny0405 commented on PR #8472: URL: https://github.com/apache/hudi/pull/8472#issuecomment-1537670179 Cool, I think we are good to close this issue.. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] danny0405 closed pull request #8472: [HUDI-5298] Optimize WriteStatus storing HoodieRecord

danny0405 closed pull request #8472: [HUDI-5298] Optimize WriteStatus storing HoodieRecord URL: https://github.com/apache/hudi/pull/8472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] danny0405 commented on a diff in pull request #8596: [MINOR] Use try with resource to close stream

danny0405 commented on code in PR #8596: URL: https://github.com/apache/hudi/pull/8596#discussion_r1186981669 ## hudi-cli/src/main/java/org/apache/hudi/cli/commands/RepairsCommand.java: ## @@ -71,10 +72,8 @@ public class RepairsCommand { @ShellMethod(key = "repair deduplicate

[GitHub] [hudi] danny0405 commented on a diff in pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

danny0405 commented on code in PR #8660: URL: https://github.com/apache/hudi/pull/8660#discussion_r1186980699 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RunBootstrapProcedure.scala: ## @@ -110,9 +111,7 @@ class RunBootstrapPro

[GitHub] [hudi] danny0405 commented on pull request #8107: [HUDI-5514][HUDI-5574][HUDI-5604][HUDI-5535] Adding auto generation of record keys support to Hudi/Spark

danny0405 commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1537657119 Looks good! It is great if we can move the decision ` !props.containsKey(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key()) into a tool method in KeyGenUtils.autoGenerateRecordKeys(TypedP

[GitHub] [hudi] hudi-bot commented on pull request #8190: [HUDI-5936] Fix serialization problem when FileStatus is not serializable

hudi-bot commented on PR #8190: URL: https://github.com/apache/hudi/pull/8190#issuecomment-1537649378 ## CI report: * 7c71b63797be01ee91268c2520f82b18b3f13b7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1676

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537638295 ## CI report: * 3f12c145c40172e1fb5139e9bc5a392bd989c859 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537612605 ## CI report: * 3f12c145c40172e1fb5139e9bc5a392bd989c859 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1537601707 ## CI report: * 36ccd4fddc346692d11803f5673c60d4e5811dc6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8107: [HUDI-5514][HUDI-5574][HUDI-5604][HUDI-5535] Adding auto generation of record keys support to Hudi/Spark

hudi-bot commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1537601087 ## CI report: * f6b8d65052236818a43eb9a81efe7b6bdd9ff1b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

hudi-bot commented on PR #8658: URL: https://github.com/apache/hudi/pull/8658#issuecomment-1537583085 ## CI report: * b34cfea7a18dfc7d235f710f1f62a35af8378e28 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

hudi-bot commented on PR #8658: URL: https://github.com/apache/hudi/pull/8658#issuecomment-1537579748 ## CI report: * b34cfea7a18dfc7d235f710f1f62a35af8378e28 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1537579606 ## CI report: * 36ccd4fddc346692d11803f5673c60d4e5811dc6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8557: [HUDI-5895] Remove bootstrap key generator configs

hudi-bot commented on PR #8557: URL: https://github.com/apache/hudi/pull/8557#issuecomment-1537577012 ## CI report: * 36ccd4fddc346692d11803f5673c60d4e5811dc6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537576877 ## CI report: * 3f12c145c40172e1fb5139e9bc5a392bd989c859 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186932947 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186933157 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186932947 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186932552 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] yihua commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

yihua commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186932514 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvider()

[GitHub] [hudi] hudi-bot commented on pull request #8107: [HUDI-5514][HUDI-5574][HUDI-5604][HUDI-5535] Adding auto generation of record keys support to Hudi/Spark

hudi-bot commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1537553376 ## CI report: * 31e6dc5d1d1d60a8d6465abbfb6549790861a476 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8107: [HUDI-5514][HUDI-5574][HUDI-5604][HUDI-5535] Adding auto generation of record keys support to Hudi/Spark

hudi-bot commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1537552167 ## CI report: * 31e6dc5d1d1d60a8d6465abbfb6549790861a476 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] nsivabalan commented on pull request #7469: [HUDI-5386] Cleaning conflicts when write concurrency mode is OCC

nsivabalan commented on PR #7469: URL: https://github.com/apache/hudi/pull/7469#issuecomment-1537547742 hey @LinMingQiang : Do you think even after landing https://github.com/apache/hudi/pull/7826, we still need this locking ? -- This is an automated message from the Apache Git Servi

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

nsivabalan commented on code in PR #7826: URL: https://github.com/apache/hudi/pull/7826#discussion_r1186918090 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java: ## @@ -545,6 +546,116 @@ public void testMultiWriterWithAsyncTa

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537543698 ## CI report: * 18e3f3140ba960834c0a3fb4285c7eabb7bbb0d7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

hudi-bot commented on PR #8388: URL: https://github.com/apache/hudi/pull/8388#issuecomment-1537542366 ## CI report: * 18e3f3140ba960834c0a3fb4285c7eabb7bbb0d7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] yihua commented on a diff in pull request #8388: [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync

yihua commented on code in PR #8388: URL: https://github.com/apache/hudi/pull/8388#discussion_r1186910826 ## azure-pipelines.yml: ## @@ -200,7 +203,7 @@ stages: inputs: mavenPomFile: 'pom.xml' goals: 'test' - options: $(MVN

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

nsivabalan commented on code in PR #8658: URL: https://github.com/apache/hudi/pull/8658#discussion_r1186893387 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/transaction/TestInProcessLockProvider.java: ## @@ -52,6 +54,107 @@ public TestInProcessLockProvid

[GitHub] [hudi] hudi-bot commented on pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

hudi-bot commented on PR #8660: URL: https://github.com/apache/hudi/pull/8660#issuecomment-1537498797 ## CI report: * a080f12934335a1000ffbeca5ec2ff6f9e4054a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537480172 ## CI report: * 72ce7d8709a80603ca8a963f461ddab22bf64e89 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

hudi-bot commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537467752 ## CI report: * 869dbe6a349700c3a1f3c8686c078edae139bdd9 UNKNOWN * f2b991d31006109e54ca98499e91ad9ea4bfdd30 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

hudi-bot commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537465510 ## CI report: * 26956ff23fcbd6806cca21a420158675f53a2265 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=169

[GitHub] [hudi] hudi-bot commented on pull request #8198: [HUDI-5943] Support bootstrap produce to synchronize to multiple metastores

hudi-bot commented on PR #8198: URL: https://github.com/apache/hudi/pull/8198#issuecomment-1537465475 ## CI report: * 2f3468c00a766cfb9a5fdb641fb98114aa572e99 UNKNOWN * dd775c219835665f9c679e9dbf5dcffbccf70313 UNKNOWN * 6ff74f85b57dc51565e77d9992226830d33b2685 Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

hudi-bot commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537463721 ## CI report: * ab3c9eb91ad9711c66f8ed2727e2a004d1485385 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1581

[GitHub] [hudi] danielfordfc commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

danielfordfc commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537458573 @nsivabalan I've taken your thoughts and moved the schemaprovider.refresh call into the SyncOnce call. I've then added overrides to FileBasedSchemaProvider and SchemaRegistryProvider

[GitHub] [hudi] hudi-bot commented on pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

hudi-bot commented on PR #8660: URL: https://github.com/apache/hudi/pull/8660#issuecomment-1537454456 ## CI report: * a080f12934335a1000ffbeca5ec2ff6f9e4054a1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1692

[GitHub] [hudi] hudi-bot commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

hudi-bot commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537454235 ## CI report: * ab3c9eb91ad9711c66f8ed2727e2a004d1485385 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1581

[GitHub] [hudi] hudi-bot commented on pull request #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

hudi-bot commented on PR #8660: URL: https://github.com/apache/hudi/pull/8660#issuecomment-1537452785 ## CI report: * a080f12934335a1000ffbeca5ec2ff6f9e4054a1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8246: [HUDI-5973] Add cachedSchema per batch, fix idempotency with getSourceSchema calls

hudi-bot commented on PR #8246: URL: https://github.com/apache/hudi/pull/8246#issuecomment-1537452543 ## CI report: * ab3c9eb91ad9711c66f8ed2727e2a004d1485385 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1581

[GitHub] [hudi] hudi-bot commented on pull request #8198: [HUDI-5943] Support bootstrap produce to synchronize to multiple metastores

hudi-bot commented on PR #8198: URL: https://github.com/apache/hudi/pull/8198#issuecomment-1537452495 ## CI report: * 2f3468c00a766cfb9a5fdb641fb98114aa572e99 UNKNOWN * dd775c219835665f9c679e9dbf5dcffbccf70313 UNKNOWN * 0c3708ac17b88a9cfd6a6639d234ed2d6949919b Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537451140 ## CI report: * d40bfb64e99a27dfe61654dd3ad4d361b037895f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=169

[GitHub] [hudi] hudi-bot commented on pull request #8198: [HUDI-5943] Support bootstrap produce to synchronize to multiple metastores

hudi-bot commented on PR #8198: URL: https://github.com/apache/hudi/pull/8198#issuecomment-1537450953 ## CI report: * 2f3468c00a766cfb9a5fdb641fb98114aa572e99 UNKNOWN * dd775c219835665f9c679e9dbf5dcffbccf70313 UNKNOWN * 0c3708ac17b88a9cfd6a6639d234ed2d6949919b Azure: [FAILUR

[GitHub] [hudi] huangxiaopingRD opened a new pull request, #8660: [MINOR] Fix RunBootstrapProcedure doesn't has database default value

huangxiaopingRD opened a new pull request, #8660: URL: https://github.com/apache/hudi/pull/8660 ### Change Logs Add default database value in RunBootstrapProcedure. There will has a NPE if it doesn't has default. ```scala java.lang.NullPointerException at java.util.Hash

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537441980 ## CI report: * d40bfb64e99a27dfe61654dd3ad4d361b037895f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=169

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537440276 ## CI report: * 5bece022bf8894b72e2f2834ae61de1747cad823 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8659: [HUDI-6155] Fix cleaner based on hours for earliest commit to retain

hudi-bot commented on PR #8659: URL: https://github.com/apache/hudi/pull/8659#issuecomment-1537438756 ## CI report: * 74a4018e52c5a9e1026547b34bf373307962b88d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537438695 ## CI report: * 5bece022bf8894b72e2f2834ae61de1747cad823 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537428711 ## CI report: * 5bece022bf8894b72e2f2834ae61de1747cad823 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537425034 ## CI report: * 5bece022bf8894b72e2f2834ae61de1747cad823 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8658: [HUDI-6186] Fix lock identity in InProcessLockProvider

hudi-bot commented on PR #8658: URL: https://github.com/apache/hudi/pull/8658#issuecomment-1537412802 ## CI report: * b34cfea7a18dfc7d235f710f1f62a35af8378e28 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion

hudi-bot commented on PR #8589: URL: https://github.com/apache/hudi/pull/8589#issuecomment-1537412498 ## CI report: * d31955ef5b729029897444b0aa5c30c242cfadb9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8659: [HUDI-6155] Fix cleaner based on hours for earliest commit to retain

hudi-bot commented on PR #8659: URL: https://github.com/apache/hudi/pull/8659#issuecomment-1537378571 ## CI report: * 74a4018e52c5a9e1026547b34bf373307962b88d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] hudi-bot commented on pull request #8659: [HUDI-6155] Fix cleaner based on hours for earliest commit to retain

hudi-bot commented on PR #8659: URL: https://github.com/apache/hudi/pull/8659#issuecomment-1537377131 ## CI report: * 74a4018e52c5a9e1026547b34bf373307962b88d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8107: [HUDI-5514][HUDI-5574][HUDI-5604][HUDI-5535] Adding auto generation of record keys support to Hudi/Spark

hudi-bot commented on PR #8107: URL: https://github.com/apache/hudi/pull/8107#issuecomment-1537376890 ## CI report: * 31e6dc5d1d1d60a8d6465abbfb6549790861a476 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1691

[GitHub] [hudi] clownxc commented on pull request #8472: [HUDI-5298] Optimize WriteStatus storing HoodieRecord

clownxc commented on PR #8472: URL: https://github.com/apache/hudi/pull/8472#issuecomment-1537368074 > @clownxc : For failed records, we need to have them logged elsewhere and so no need to deflate. For exception cases, the write status should be marked as failure. So, I don't see any reaso

[jira] [Updated] (HUDI-6155) Fix cleaner based on hours for earliest commit to retain

[ https://issues.apache.org/jira/browse/HUDI-6155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6155: - Labels: pull-request-available (was: ) > Fix cleaner based on hours for earliest commit to retain

[GitHub] [hudi] clownxc opened a new pull request, #8659: [HUDI-6155] Fix cleaner based on hours for earliest commit to retain

clownxc opened a new pull request, #8659: URL: https://github.com/apache/hudi/pull/8659 ### Change Logs HoodieTableConfig has a config to let users to override timezone for commit time generation. but looks like there are some places where we use current system's zone instead of hono

[GitHub] [hudi] hudi-bot commented on pull request #8596: [MINOR] Use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1537366987 ## CI report: * 5ee1f5c3af94d920cfb1e186b2896dfe25533c2a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1690

  1   2   >