Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438475893 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -765,6 +765,14 @@ public class HoodieWriteConfig extends

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

bvaradar commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438474702 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -765,6 +765,14 @@ public class HoodieWriteConfig extends

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

hudi-bot commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1872442526 ## CI report: * 55a5a675e4ee4eb813804f9a7216ab1812be9bb1 Azure:

Re: [PR] [HUDI-7246] Fix Data Skipping Issue: No Results When Query Conditions Involve Both Columns with and without Column Stats [hudi]

danny0405 commented on code in PR #10389: URL: https://github.com/apache/hudi/pull/10389#discussion_r1438465730 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/ColumnStatsIndexSupport.scala: ## @@ -272,11 +272,13 @@ class ColumnStatsIndexSupport(spark:

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

danny0405 commented on code in PR #10413: URL: https://github.com/apache/hudi/pull/10413#discussion_r1438465451 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RepairOverwriteHoodiePropsProcedure.scala: ## @@ -57,7 +67,7 @@ class

Re: [PR] [HUDI-7266] add clustering metric for flink [hudi]

danny0405 commented on code in PR #10420: URL: https://github.com/apache/hudi/pull/10420#discussion_r1438465298 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/metrics/FlinkClusteringMetrics.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software

Re: [PR] [HUDI-7266] add clustering metric for flink [hudi]

danny0405 commented on code in PR #10420: URL: https://github.com/apache/hudi/pull/10420#discussion_r1438465244 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/ClusteringPlanOperator.java: ## @@ -88,10 +97,20 @@ public void

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438464674 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,6 +294,9 @@ protected void

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

hudi-bot commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1872436241 ## CI report: * 55a5a675e4ee4eb813804f9a7216ab1812be9bb1 Azure:

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

beyond1920 commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1872435971 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

hudi-bot commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1872409882 ## CI report: * a04fd58813acf95fcc02aac9a4abe6860abf1fe9 Azure:

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

hudi-bot commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1872408232 ## CI report: * a04fd58813acf95fcc02aac9a4abe6860abf1fe9 Azure:

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

empcl commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1872397230 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872355904 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * ea196eb78876cb4761ccf181131a179ed1c25fa5 Azure:

Re: [PR] [HUDI-7261] TVF to query hudi table's filesystem state through spark-sql [hudi]

bvaradar commented on PR #10414: URL: https://github.com/apache/hudi/pull/10414#issuecomment-1872341655 @bhat-vinay : Landed the other PR. Please resolve conflicts and rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872333827 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 4b31fd3e3eed64c7a859c4135b0c21579fe7e014 Azure:

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872330969 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 1fed4235017a82cfb77a389bc2b6cace60ea340d Azure:

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872328041 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 1fed4235017a82cfb77a389bc2b6cace60ea340d Azure:

(hudi) branch master updated (3b3ca961d9e -> f39cb1d2e3e)

This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 3b3ca961d9e [HUDI-7249] Disable mor compaction scheduling when using append mode (#10388) add f39cb1d2e3e

Re: [PR] [HUDI-7243] TVF to query hudi timeline through spark sql [hudi]

bvaradar merged PR #10355: URL: https://github.com/apache/hudi/pull/10355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872309523 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * a73fa68a2658fb6cff2630604948ba6c186f98ed Azure:

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872306140 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * a73fa68a2658fb6cff2630604948ba6c186f98ed Azure:

Re: [PR] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1872302826 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * a73fa68a2658fb6cff2630604948ba6c186f98ed Azure:

Re: [PR] Improve datadog reporter [hudi]

hudi-bot commented on PR #10433: URL: https://github.com/apache/hudi/pull/10433#issuecomment-1872271587 ## CI report: * bfffa0ecaba7867705ec3ef9d36731ecacf8add1 Azure:

Re: [PR] Improve datadog reporter [hudi]

hudi-bot commented on PR #10433: URL: https://github.com/apache/hudi/pull/10433#issuecomment-1872245263 ## CI report: * bfffa0ecaba7867705ec3ef9d36731ecacf8add1 Azure:

Re: [PR] Improve datadog reporter [hudi]

hudi-bot commented on PR #10433: URL: https://github.com/apache/hudi/pull/10433#issuecomment-1872241628 ## CI report: * bfffa0ecaba7867705ec3ef9d36731ecacf8add1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

[PR] Fix/improve dd [hudi]

parisni opened a new pull request, #10433: URL: https://github.com/apache/hudi/pull/10433 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

Re: [I] [SUPPORT] Bulk insert does not handle hoodie metrics with datadog reporter [hudi]

parisni commented on issue #10395: URL: https://github.com/apache/hudi/issues/10395#issuecomment-1872216614 Changing the datadog host did help to bulk_insert metrics on my local setup. However in production the metrics still won't work. Some warning are missing when the api key is not

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1872139372 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * 0bf07811af39bc66fa42ea2fe0fee65e3525f26c Azure:

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

hudi-bot commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1872099273 ## CI report: * a04fd58813acf95fcc02aac9a4abe6860abf1fe9 Azure:

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

hudi-bot commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1872048097 ## CI report: * 55a5a675e4ee4eb813804f9a7216ab1812be9bb1 Azure:

[I] [SUPPORT] Failed to create marker file Exception when trying to write data on Hudi [hudi]

gsudhanshu opened a new issue, #10432: URL: https://github.com/apache/hudi/issues/10432 My Setup: My Django application on app server uses pyspark 3.4.2 to connect to spark-hudi bundle (3.4.2:2_12:0.14.0) on db server spark config: ``` spark = SparkSession.builder \

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

hudi-bot commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1871979104 ## CI report: * 9804f94e4afc6990f9d565e7bfeddb33f4c60027 Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871978751 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * e16c1436b39b856f47564d7c02fb09c3ffdf8e0b Azure:

Re: [PR] [HUDI-7260] fix call repair_overwrite_hoodie_props failure error due to specify h… [hudi]

hudi-bot commented on PR #10413: URL: https://github.com/apache/hudi/pull/10413#issuecomment-1871948951 ## CI report: * 9804f94e4afc6990f9d565e7bfeddb33f4c60027 Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871948621 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * c03b1658a59d5064acc05cf9a92b6130999f7ee6 Azure:

Re: [PR] [HUDI-7243] TVF to query hudi timeline through spark sql [hudi]

hudi-bot commented on PR #10355: URL: https://github.com/apache/hudi/pull/10355#issuecomment-1871943323 ## CI report: * bb48cd2b3363f1a83248fa8d8ff655115c43a14e Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871943188 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * c03b1658a59d5064acc05cf9a92b6130999f7ee6 Azure:

Re: [I] [ERROR] hudi-0.14.0/hudi-kafka-connect/src/main/resources/._ControlMessage.proto [1:1]: Invalid control characters encountered in text. [hudi]

lxxstarii closed issue #10429: [ERROR] hudi-0.14.0/hudi-kafka-connect/src/main/resources/._ControlMessage.proto [1:1]: Invalid control characters encountered in text. URL: https://github.com/apache/hudi/issues/10429 -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871937709 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * 814e79d8e5f756f0dac433164a0862a426874ac0 Azure:

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

hudi-bot commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1871938150 ## CI report: * c22260ad096257d8b433b847d7c11ed64bc0f1db Azure:

Re: [PR] [HUDI-7246] Fix Data Skipping Issue: No Results When Query Conditions Involve Both Columns with and without Column Stats [hudi]

hudi-bot commented on PR #10389: URL: https://github.com/apache/hudi/pull/10389#issuecomment-1871938027 ## CI report: * 248df7c04d611c5f521f309732aa21351161fa8b UNKNOWN * b02cd220b9a58268c224b7faf6f06ce95ed283a7 Azure:

Re: [PR] [HUDI-7243] TVF to query hudi timeline through spark sql [hudi]

hudi-bot commented on PR #10355: URL: https://github.com/apache/hudi/pull/10355#issuecomment-1871937854 ## CI report: * 5ec834ec0158eea935caa1870a714d80fca8fecc Azure:

Re: [PR] [HUDI-7265] Support schema evolution by Flink SQL using HoodieHiveCatalog [hudi]

hudi-bot commented on PR #10426: URL: https://github.com/apache/hudi/pull/10426#issuecomment-1871893096 ## CI report: * d2064b4cddad70fb951d129b0c9057becd556c19 Azure:

Re: [PR] [HUDI-7261] TVF to query hudi table's filesystem state through spark-sql [hudi]

hudi-bot commented on PR #10414: URL: https://github.com/apache/hudi/pull/10414#issuecomment-1871882269 ## CI report: * 898fd87176b5546dbceb7062998ff517b2ec347e Azure:

Re: [I] The Schema Evolution Not working For Hudi 0.12.3 [hudi]

ad1happy2go commented on issue #10309: URL: https://github.com/apache/hudi/issues/10309#issuecomment-1871879825 @Amar1404 Sorry for the delay here I was OOO. Can you ping me on slack so we can work on this together? -- This is an automated message from the Apache Git Service. To

Re: [I] [SUPPORT] How to skip some partitions in a table when readStreaming in Spark at the init stage [hudi]

ad1happy2go commented on issue #10315: URL: https://github.com/apache/hudi/issues/10315#issuecomment-1871878668 @lei-su-awx Did you see the behaviour I mentioned? Any more updates here please? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Concurrent and Roll back commit issue [hudi]

ad1happy2go commented on issue #10356: URL: https://github.com/apache/hudi/issues/10356#issuecomment-1871877913 @khajaasmath786 Any updates on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [SUPPORT]0.12.3 upgrade to 0.14.0 data duplication [hudi]

ad1happy2go commented on issue #10407: URL: https://github.com/apache/hudi/issues/10407#issuecomment-1871874792 @zyclove Also, Any reason why you are setting DataSourceReadOptions.EXTRACT_PARTITION_VALUES_FROM_PARTITION_PATH. This config will extract the partition values from physical

Re: [I] hoodie.bulkinsert.shuffle.parallelism Not activated [hudi]

ad1happy2go commented on issue #10418: URL: https://github.com/apache/hudi/issues/10418#issuecomment-1871871612 @zhangjw123321 This is for number of partitions after shuffle stage. Can you show the spark DAG once. How many stages it is creating? -- This is an automated message from the

Re: [I] [ERROR] hudi-0.14.0/hudi-kafka-connect/src/main/resources/._ControlMessage.proto [1:1]: Invalid control characters encountered in text. [hudi]

ad1happy2go commented on issue #10429: URL: https://github.com/apache/hudi/issues/10429#issuecomment-1871865474 @lxxstarii Can you let us more details on when you are getting this error? Adding @yihua for more visibility. -- This is an automated message from the Apache Git Service. To

Re: [PR] [HUDI-7246] Fix Data Skipping Issue: No Results When Query Conditions Involve Both Columns with and without Column Stats [hudi]

hudi-bot commented on PR #10389: URL: https://github.com/apache/hudi/pull/10389#issuecomment-1871852576 ## CI report: * 248df7c04d611c5f521f309732aa21351161fa8b UNKNOWN * daa335eb9234834ded995c1cc27480b94d2bb0e3 Azure:

Re: [PR] [HUDI-7243] TVF to query hudi timeline through spark sql [hudi]

hudi-bot commented on PR #10355: URL: https://github.com/apache/hudi/pull/10355#issuecomment-1871852424 ## CI report: * 52a8b7ef64709e5c63797299c76e19827a124083 Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871852318 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * 814e79d8e5f756f0dac433164a0862a426874ac0 Azure:

Re: [PR] [HUDI-7246] Fix Data Skipping Issue: No Results When Query Conditions Involve Both Columns with and without Column Stats [hudi]

hudi-bot commented on PR #10389: URL: https://github.com/apache/hudi/pull/10389#issuecomment-1871848010 ## CI report: * 248df7c04d611c5f521f309732aa21351161fa8b UNKNOWN * daa335eb9234834ded995c1cc27480b94d2bb0e3 Azure:

Re: [PR] [HUDI-7243] TVF to query hudi timeline through spark sql [hudi]

hudi-bot commented on PR #10355: URL: https://github.com/apache/hudi/pull/10355#issuecomment-1871847922 ## CI report: * 52a8b7ef64709e5c63797299c76e19827a124083 Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

hudi-bot commented on PR #10297: URL: https://github.com/apache/hudi/pull/10297#issuecomment-1871847811 ## CI report: * 05bed31829f2362de479344215d29ccca99bd449 UNKNOWN * 814e79d8e5f756f0dac433164a0862a426874ac0 Azure:

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438091481 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,6 +294,9 @@ protected void

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438091481 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,6 +294,9 @@ protected void

Re: [I] [SUPPORT] Kafka connect sink to S3 authentification parameters [hudi]

ad1happy2go commented on issue #10428: URL: https://github.com/apache/hudi/issues/10428#issuecomment-1871834313 @akolyaga Did you tried setting env variables like below and adding hadoop-aws and aws-java-sdk bundle jar - export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY=

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1437448466 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,6 +294,9 @@ protected void

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

xuzifu666 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438079654 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,8 +294,16 @@ protected void

Re: [PR] [HUDI-7208] Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details [hudi]

beyond1920 commented on code in PR #10297: URL: https://github.com/apache/hudi/pull/10297#discussion_r1438079373 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -292,8 +294,16 @@ protected void