[GitHub] [hudi] stream2000 opened a new pull request, #9181: Revert "[HUDI-6521] Disable failing test case."

stream2000 opened a new pull request, #9181: URL: https://github.com/apache/hudi/pull/9181 Reverts #9169 since #9156 merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] Tomccat3 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633577643 ok, i will try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[hudi] branch master updated: [HUDI-6315] Feature flag for disabling optimized update/delete code path. (#9131)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2c9bd80cc8a [HUDI-6315] Feature flag for

[GitHub] [hudi] nsivabalan merged pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

nsivabalan merged PR #9131: URL: https://github.com/apache/hudi/pull/9131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633563635 ## CI report: * e53131f6d1daa203fbb36fd8346d9a72c665b857 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633557115 ## CI report: * 25a9b77fa1b48dd27e8ec0f88165278216bc08b3 Azure:

[GitHub] [hudi] danny0405 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

danny0405 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633529140 yeah, possible, you can pass around the schema together with the RowData, before each commit, you can overwrite the schema with the latest schema, only feasible for Flink DataStream

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633526009 ## CI report: * 25a9b77fa1b48dd27e8ec0f88165278216bc08b3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633525872 ## CI report: * 5f98b8a27f2948e7f3a0f8673606b793b90729db Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633521100 ## CI report: * 25a9b77fa1b48dd27e8ec0f88165278216bc08b3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633520963 ## CI report: * 5f98b8a27f2948e7f3a0f8673606b793b90729db Azure:

[hudi] branch master updated (cfc1bc5bb7c -> e5a9ce55f47)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from cfc1bc5bb7c [HUDI-6156] prevent leaving tmp file in timeline when multi process try to complete the same instant

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633516645 ## CI report: * 42d7e09b27e34bbaa1f17d9cea78be06c8e8b9a2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633516353 ## CI report: * 25a9b77fa1b48dd27e8ec0f88165278216bc08b3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633516216 ## CI report: * 5f98b8a27f2948e7f3a0f8673606b793b90729db Azure:

[GitHub] [hudi] ad1happy2go commented on issue #9180: Hudi metadata to hive metastore Get the parquet schema for this table looking at the latest commit OOM

ad1happy2go commented on issue #9180: URL: https://github.com/apache/hudi/issues/9180#issuecomment-1633515696 @SendDreams Are you running hive sync job? This is failing with Java Heap Space issue. You might consider trying increasing the memory. -- This is an automated message from the

[GitHub] [hudi] Tomccat3 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633507531 > > be made automatic in flink datastream job > > No, the open source Flink does no support pass the schema around to the sink, ok, what if we implement a RowData with

[GitHub] [hudi] SendDreams commented on issue #9180: Hudi metadata to hive metastore Get the parquet schema for this table looking at the latest commit OOM

SendDreams commented on issue #9180: URL: https://github.com/apache/hudi/issues/9180#issuecomment-1633492408 Is there any god can help me anwser zhe question? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1633491501 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * 8ea8e3a853152fcc0d1cf4d0f53a38565ac33990 Azure:

[GitHub] [hudi] SendDreams opened a new issue, #9180: Hudi metadata to hive metastore Get the parquet schema for this table looking at the latest commit OOM

SendDreams opened a new issue, #9180: URL: https://github.com/apache/hudi/issues/9180 Trying to sync hoodie table acs_camain_huo with base path hdfs://GWNS:8020/user/hudi/acs_camain_hu of type MERGE_ON_READ [INFO ] 2023-07-13 11:09:14,065

[GitHub] [hudi] codope commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

codope commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1261929493 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter { if

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1633482191 ## CI report: * 570f60de57b432bdd88d7f1dd83f239834dbd579 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633481898 ## CI report: * a0e53d8ef3f506afca5f25fd22adb70581fae426 Azure:

[GitHub] [hudi] KnightChess commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

KnightChess commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1633468958 @danny0405 Thank you for reviewing the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] danny0405 commented on a diff in pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

danny0405 commented on code in PR #9175: URL: https://github.com/apache/hudi/pull/9175#discussion_r1261916555 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java: ## @@ -408,8 +408,13 @@ private void initInstant(String

[GitHub] [hudi] danny0405 commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

danny0405 commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1261912747 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter { if

[GitHub] [hudi] danny0405 commented on issue #9143: [SUPPORT] Failure to delete records with missing attributes from PostgresDebeziumSource

danny0405 commented on issue #9143: URL: https://github.com/apache/hudi/issues/9143#issuecomment-1633455688 @ad1happy2go Can you help reproduce this issue, this seems a blocker issue for release 0.14.0. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] danny0405 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

danny0405 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633452375 > be made automatic in flink datastream job No, the open source Flink does no support pass the schema around to the sink, that is a tricky part for Hudi to track the schema

[jira] [Closed] (HUDI-6156) prevent leaving tmp file in timeline when multi task try to complete the same instant

[ https://issues.apache.org/jira/browse/HUDI-6156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6156. Resolution: Fixed Fixed via master branch: cfc1bc5bb7c653457587dcff57951d3a30254f7d > prevent leaving tmp

[hudi] branch master updated: [HUDI-6156] prevent leaving tmp file in timeline when multi process try to complete the same instant (#8610)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new cfc1bc5bb7c [HUDI-6156] prevent leaving tmp

[GitHub] [hudi] danny0405 merged pull request #8610: [HUDI-6156] Prevent leaving tmp file in timeline when multi process t…

danny0405 merged PR #8610: URL: https://github.com/apache/hudi/pull/8610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] eric9204 commented on pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer

eric9204 commented on PR #9176: URL: https://github.com/apache/hudi/pull/9176#issuecomment-1633448383 > whats the bug here. can you please write a Unit test @nsivabalan Thank you for your reply. Every batch, we get checkpoint message from timeline firstly, if

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1633445941 ## CI report: * 2be85a0a57f0a8d5962e4c41acaead0d8db1a559 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8949: [DNM] Testing Java 17

hudi-bot commented on PR #8949: URL: https://github.com/apache/hudi/pull/8949#issuecomment-1633445571 ## CI report: * a0e53d8ef3f506afca5f25fd22adb70581fae426 Azure:

[hudi] branch master updated (acc63fc4193 -> 6db4c01cf3d)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from acc63fc4193 [HUDI-6526] Hoodie Hive catalog sync timestamp(3) as timestamp type into Hive (#9177) add

[GitHub] [hudi] danny0405 merged pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

danny0405 merged PR #9158: URL: https://github.com/apache/hudi/pull/9158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

danny0405 commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1633435313 Tests passed: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=18510=results -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] Tomccat3 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633423097 > @Tomccat3 Hudi should support it. Are you facing issues? What kind of schema changes you are having? For example: 1. I have a flink datastream job consume mysql binlog and

[GitHub] [hudi] SteNicholas commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

SteNicholas commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1633418730 @bvaradar, could you help to review this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] Tomccat3 commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

Tomccat3 commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1633417713 > @Tomccat3 , schema evolution in Hudi is supported, but not automatically, you still need to alter the table schema manually and maybe restart the writing job. Thanks for you

[GitHub] [hudi] amrishlal opened a new pull request, #9179: Rename Delta useage.

amrishlal opened a new pull request, #9179: URL: https://github.com/apache/hudi/pull/9179 ### Change Logs Change instances of `DeltaStreamer` to `Streamer`. ### Impact None ### Risk level (write none, low medium or high below) Low ### Documentation

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1633397100 ## CI report: * 9debfe30921875081c9760dd08655a72e863fcb2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1633392748 ## CI report: * 4162ef786e793a3476010858c3175434803c4341 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1633392589 ## CI report: * 5b48833e4d759cf072a3da49f91c7476bbf09b48 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

hudi-bot commented on PR #9175: URL: https://github.com/apache/hudi/pull/9175#issuecomment-1633366033 ## CI report: * 4162ef786e793a3476010858c3175434803c4341 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9053: [HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 rows only is incoming

hudi-bot commented on PR #9053: URL: https://github.com/apache/hudi/pull/9053#issuecomment-1633355211 ## CI report: * ff5cd07154d48f18d8034075c8dfc3990b204cbe Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633328836 ## CI report: * 3b80c6f2e2620bd56328e4935a4085a97ef23f0a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633328438 ## CI report: * 89a1c11b7cdf452c37c190ee11934c7546c34c25 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633323756 ## CI report: * 3b80c6f2e2620bd56328e4935a4085a97ef23f0a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-163332 ## CI report: * 89a1c11b7cdf452c37c190ee11934c7546c34c25 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633318367 ## CI report: * 3b80c6f2e2620bd56328e4935a4085a97ef23f0a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1633318338 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * 8ea8e3a853152fcc0d1cf4d0f53a38565ac33990 Azure:

[GitHub] [hudi] amrishlal commented on a diff in pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

amrishlal commented on code in PR #9145: URL: https://github.com/apache/hudi/pull/9145#discussion_r1261790210 ## hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/adapter/Spark3_3Adapter.scala: ## @@ -124,8 +124,4 @@ class Spark3_3Adapter extends

[GitHub] [hudi] CTTY commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

CTTY commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1633289362 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #9173: Dynamic Partition Pruning Port

hudi-bot commented on PR #9173: URL: https://github.com/apache/hudi/pull/9173#issuecomment-1633272250 ## CI report: * bee3d635bca523618671908924de929bf8d9dce6 UNKNOWN * 836b7eb07f2641c35da3583a7313b4f8f33bd454 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1633272088 ## CI report: * 2be85a0a57f0a8d5962e4c41acaead0d8db1a559 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1633271156 ## CI report: * 1fc455198964e104536b807792dde341c960de88 Azure:

[GitHub] [hudi] amrishlal commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

amrishlal commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1633260806 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1633232415 ## CI report: * beb523cca98b4b62964c485e464d5eb1ce6e25a2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9053: [HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 rows only is incoming

hudi-bot commented on PR #9053: URL: https://github.com/apache/hudi/pull/9053#issuecomment-1633232195 ## CI report: * 92ee666bdfd82f71489a45eac3c789cf223c86a9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1633231348 ## CI report: * 1fc455198964e104536b807792dde341c960de88 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1633224365 ## CI report: * beb523cca98b4b62964c485e464d5eb1ce6e25a2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9053: [HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 rows only is incoming

hudi-bot commented on PR #9053: URL: https://github.com/apache/hudi/pull/9053#issuecomment-1633224102 ## CI report: * 92ee666bdfd82f71489a45eac3c789cf223c86a9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633215795 ## CI report: * 9e3c05bdf4ef8cb5bb800ccde85fde085b0d07af Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633215123 ## CI report: * 89a1c11b7cdf452c37c190ee11934c7546c34c25 Azure:

[GitHub] [hudi] nsivabalan commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

nsivabalan commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1633199465 hey @codope : not sure I understand your question here. `I think this config is also used for datasource inserts. So, now the behavior of datasource and sql will differ for the insert

[GitHub] [hudi] amrishlal commented on a diff in pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

amrishlal commented on code in PR #9145: URL: https://github.com/apache/hudi/pull/9145#discussion_r1261702203 ## hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/adapter/Spark3_3Adapter.scala: ## @@ -124,8 +124,4 @@ class Spark3_3Adapter extends

[GitHub] [hudi] amrishlal commented on a diff in pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

amrishlal commented on code in PR #9145: URL: https://github.com/apache/hudi/pull/9145#discussion_r1261691400 ## hudi-spark-datasource/hudi-spark3.2.x/src/main/scala/org/apache/spark/sql/adapter/Spark3_2Adapter.scala: ## @@ -123,8 +123,4 @@ class Spark3_2Adapter extends

[GitHub] [hudi] hudi-bot commented on pull request #9173: Dynamic Partition Pruning Port

hudi-bot commented on PR #9173: URL: https://github.com/apache/hudi/pull/9173#issuecomment-1633160518 ## CI report: * 0d0d0cde8aa18c5e93ce256208ea3b27b210a815 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633160218 ## CI report: * 9e3c05bdf4ef8cb5bb800ccde85fde085b0d07af Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9173: Dynamic Partition Pruning Port

hudi-bot commented on PR #9173: URL: https://github.com/apache/hudi/pull/9173#issuecomment-1633148201 ## CI report: * 0d0d0cde8aa18c5e93ce256208ea3b27b210a815 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9173: Dynamic Partition Pruning Port

hudi-bot commented on PR #9173: URL: https://github.com/apache/hudi/pull/9173#issuecomment-1633137871 ## CI report: * 0d0d0cde8aa18c5e93ce256208ea3b27b210a815 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1633137640 ## CI report: * 2be85a0a57f0a8d5962e4c41acaead0d8db1a559 Azure:

[GitHub] [hudi] github-code-scanning[bot] commented on pull request #9111: Add CodeQL workflow

github-code-scanning[bot] commented on PR #9111: URL: https://github.com/apache/hudi/pull/9111#issuecomment-1633112191 This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request

[GitHub] [hudi] nsivabalan commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

nsivabalan commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1633096348 hey @zhuanshenbsj1 I know we are changing the behavior. But we looked at few other systems in similar space and everywhere INSERT_INTO can result in duplicates. And we are taking a hit

[GitHub] [hudi] yihua commented on a diff in pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

yihua commented on code in PR #9145: URL: https://github.com/apache/hudi/pull/9145#discussion_r1261618581 ## hudi-spark-datasource/hudi-spark3.0.x/src/main/scala/org/apache/spark/sql/catalyst/analysis/HoodieSpark30Analysis.scala: ## @@ -197,11 +170,7 @@ object

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

nsivabalan commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1261637106 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -187,29 +218,38 @@ trait ProvidesHoodieConfig

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

nsivabalan commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1261636806 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter { if

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1633061715 ## CI report: * 9e3c05bdf4ef8cb5bb800ccde85fde085b0d07af Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1633008402 ## CI report: * 37dd6d9ce0a638d4274d66b67bd0799ee7d3daf5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1632996425 ## CI report: * 37dd6d9ce0a638d4274d66b67bd0799ee7d3daf5 Azure:

[GitHub] [hudi] ad1happy2go commented on issue #9073: Spark-3.2 Insert Into Hudi Table UnsupportedOperationException: S3A streams are not Syncable

ad1happy2go commented on issue #9073: URL: https://github.com/apache/hudi/issues/9073#issuecomment-1632922916 @sunneebaby Can you provide some reproducible steps in order to reproduce it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1632900338 ## CI report: * 37dd6d9ce0a638d4274d66b67bd0799ee7d3daf5 Azure:

[GitHub] [hudi] sydneyhoran commented on issue #9143: [SUPPORT] Failure to delete records with missing attributes from PostgresDebeziumSource

sydneyhoran commented on issue #9143: URL: https://github.com/apache/hudi/issues/9143#issuecomment-1632896844 Hey @ad1happy2go @danny0405 @soumilshah1995 unfortunately with the latest master branch, the non-op deletes are still happening and the job is not processing our deletes

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

Zouxxyy commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261417413 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RunCompactionProcedure.scala: ## @@ -59,76 +61,63 @@ class

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632845653 ## CI report: * c6e27b201d5fe4ca6eff34f0efdd5a8da5a3a45d Azure:

[GitHub] [hudi] voonhous commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

voonhous commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261394742 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RunCompactionProcedure.scala: ## @@ -59,76 +61,63 @@ class

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

Zouxxyy commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261417681 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/HoodieProcedureUtils.scala: ## @@ -43,4 +47,79 @@ object

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

Zouxxyy commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261417413 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RunCompactionProcedure.scala: ## @@ -59,76 +61,63 @@ class

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632829931 ## CI report: * c6e27b201d5fe4ca6eff34f0efdd5a8da5a3a45d Azure:

[GitHub] [hudi] voonhous commented on issue #9178: [SUPPORT]In the scenario of writing from mysql cdc to hudi use flink，does hudi already supported schema evolution？

voonhous commented on issue #9178: URL: https://github.com/apache/hudi/issues/9178#issuecomment-1632821508 Commenting for visibility as this might be a feature-request for implicit comprehensive schema evolution for Hudi-on-Flink. -- This is an automated message from the Apache Git

[GitHub] [hudi] voonhous commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

voonhous commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-163281 @big-doudou Thank you so much for the details! This looks like an issue with partial-failover and recovery. Apologies, am still trying to understand this, can you give me the

[GitHub] [hudi] voonhous commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

voonhous commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261394742 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/RunCompactionProcedure.scala: ## @@ -59,76 +61,63 @@ class

[GitHub] [hudi] voonhous commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

voonhous commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261388037 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/HoodieProcedureUtils.scala: ## @@ -43,4 +47,79 @@ object

[GitHub] [hudi] voonhous commented on a diff in pull request #9068: [HUDI-6447] Unify operation in compaction and clustering procedure

voonhous commented on code in PR #9068: URL: https://github.com/apache/hudi/pull/9068#discussion_r1261388037 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/HoodieProcedureUtils.scala: ## @@ -43,4 +47,79 @@ object

[GitHub] [hudi] SteNicholas commented on a diff in pull request #9175: [MINOR] Flink Coordinator stop current instant heartbeat before create a new instant

SteNicholas commented on code in PR #9175: URL: https://github.com/apache/hudi/pull/9175#discussion_r1261384632 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java: ## @@ -407,16 +408,25 @@ private void startInstant() {

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] Codreview changes for Spark SQL Merge Into for pkless tables'

hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1632712812 ## CI report: * 9e3c05bdf4ef8cb5bb800ccde85fde085b0d07af Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632712598 ## CI report: * c6e27b201d5fe4ca6eff34f0efdd5a8da5a3a45d Azure:

[GitHub] [hudi] amrishlal commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

amrishlal commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1632710887 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1632622262 ## CI report: * 74451cd8ee26909fac98e88850ba352c8c415ef5 Azure:

  1   2   >