Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-25 Thread via GitHub
yihua merged PR #9876: URL: https://github.com/apache/hudi/pull/9876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-25 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1780092114 ## CI report: * 3672dea3c9d2512071dc27b99e24dfb3922a3b38 UNKNOWN * eb5b62e94807c1b2b6942402b117fe9dc57d425b Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-25 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1779778741 ## CI report: * 3672dea3c9d2512071dc27b99e24dfb3922a3b38 UNKNOWN * bfdb36f31ef0b8670c82c308494f9af2f7ef1272 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-25 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1779765466 ## CI report: * 3672dea3c9d2512071dc27b99e24dfb3922a3b38 UNKNOWN * bfdb36f31ef0b8670c82c308494f9af2f7ef1272 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1371173652 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -261,7 +262,8 @@ class TestMergeIntoTable extends

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1371170669 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -261,7 +262,8 @@ class TestMergeIntoTable extends

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1371157540 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -261,7 +262,8 @@ class TestMergeIntoTable extends

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778503193 ## CI report: * 3672dea3c9d2512071dc27b99e24dfb3922a3b38 UNKNOWN * bfdb36f31ef0b8670c82c308494f9af2f7ef1272 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778461940 ## CI report: * 3672dea3c9d2512071dc27b99e24dfb3922a3b38 UNKNOWN * d96a7423b1c1bae13148744547726ed95ee5c6b7 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1371105049 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestMergeIntoTable.scala: ## @@ -261,7 +262,8 @@ class TestMergeIntoTable extends

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778351985 ## CI report: * b8bc65dc87cfd1305634bf16f96a97944ce85816 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778343787 ## CI report: * b8bc65dc87cfd1305634bf16f96a97944ce85816 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778336139 ## CI report: * b8bc65dc87cfd1305634bf16f96a97944ce85816 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778296319 ## CI report: * b8bc65dc87cfd1305634bf16f96a97944ce85816 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
yihua commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1778076328 I discussed the comments with @danny0405 offline. Two things to address in this PR: (1) Instead of putting both partial and full schemas in the log block header, when partial updates

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-24 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1370828538 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload {

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-23 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1369447936 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-23 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1369060242 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload {

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-23 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1369039030 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java: ## @@ -652,6 +660,16 @@ private static Map getUpdatedHeader(Map

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-22 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1368076982 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-22 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1368076982 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-22 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1368075473 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java: ## @@ -652,6 +660,16 @@ private static Map getUpdatedHeader(Map

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1773965020 ## CI report: * b8bc65dc87cfd1305634bf16f96a97944ce85816 Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1773948275 ## CI report: * 794904512405851fa42c10927c315ca55d82fbdc Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1773947285 ## CI report: * 794904512405851fa42c10927c315ca55d82fbdc Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1367807590 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala: ## @@ -449,21 +466,58 @@ case class

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1367807326 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala: ## @@ -411,10 +414,14 @@ object ExpressionPayload {

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1367806982 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java: ## @@ -652,6 +660,16 @@ private static Map getUpdatedHeader(Map

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-21 Thread via GitHub
yihua commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1367806866 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -755,6 +756,24 @@ public class HoodieWriteConfig extends HoodieConfig {

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-19 Thread via GitHub
danny0405 commented on code in PR #9876: URL: https://github.com/apache/hudi/pull/9876#discussion_r1365081774 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala: ## @@ -449,21 +466,58 @@ case class

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-17 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1767619892 ## CI report: * 794904512405851fa42c10927c315ca55d82fbdc Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-17 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1767534035 ## CI report: * 794904512405851fa42c10927c315ca55d82fbdc Azure:

Re: [PR] [HUDI-6800] Support writing partial updates to the data blocks in MOR tables [hudi]

2023-10-17 Thread via GitHub
hudi-bot commented on PR #9876: URL: https://github.com/apache/hudi/pull/9876#issuecomment-1767526069 ## CI report: * 794904512405851fa42c10927c315ca55d82fbdc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the