[GitHub] [hudi] codope commented on pull request #9324: [HUDI-6619] [WIP] Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

2023-07-31 Thread via GitHub
codope commented on PR #9324: URL: https://github.com/apache/hudi/pull/9324#issuecomment-1659687966 @xushiyan Can you please review it as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] codope commented on a diff in pull request #9324: [HUDI-6619] [WIP] Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

2023-07-31 Thread via GitHub
codope commented on code in PR #9324: URL: https://github.com/apache/hudi/pull/9324#discussion_r1280185595 ## pom.xml: ## @@ -2479,10 +2465,7 @@ ${fasterxml.spark3.version} ${fasterxml.spark3.version} ${fasterxml.spark3.version} -${fasterxml.

[jira] [Updated] (HUDI-6624) Return Empty split input array when there is no commit instant for batch read

2023-07-31 Thread hehuiyuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hehuiyuan updated HUDI-6624: Description: ```java org.apache.flink.runtime.rest.handler.RestHandlerException: The main method caused an

[jira] [Created] (HUDI-6624) 1

2023-07-31 Thread hehuiyuan (Jira)
hehuiyuan created HUDI-6624: --- Summary: 1 Key: HUDI-6624 URL: https://issues.apache.org/jira/browse/HUDI-6624 Project: Apache Hudi Issue Type: Bug Reporter: hehuiyuan ```java org.apach

[GitHub] [hudi] hudi-bot commented on pull request #9331: [HUDI-6623] Move the sorting of HoodieMetaserverClient instants from …

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9331: URL: https://github.com/apache/hudi/pull/9331#issuecomment-1659663415 ## CI report: * 75502f79cd6d81f7f297b60ad4a591ed7f76215e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1895

[GitHub] [hudi] hudi-bot commented on pull request #9330: [HUDI-6622] Reuse the table config from HoodieTableMetaClient in the …

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9330: URL: https://github.com/apache/hudi/pull/9330#issuecomment-1659663379 ## CI report: * 72a29231ecc147a87700ef70a524a40cf70a8840 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1895

[GitHub] [hudi] hudi-bot commented on pull request #9331: [HUDI-6623] Move the sorting of HoodieMetaserverClient instants from …

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9331: URL: https://github.com/apache/hudi/pull/9331#issuecomment-1659653066 ## CI report: * 75502f79cd6d81f7f297b60ad4a591ed7f76215e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9330: [HUDI-6622] Reuse the table config from HoodieTableMetaClient in the …

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9330: URL: https://github.com/apache/hudi/pull/9330#issuecomment-1659652999 ## CI report: * 72a29231ecc147a87700ef70a524a40cf70a8840 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Commented] (HUDI-6605) Add compaction/logcompaction writestatus errors check and advance it

2023-07-31 Thread kwang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749477#comment-17749477 ] kwang commented on HUDI-6605: - Hi [~danny0405], you merged error, this pull request is linked

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants from meta server side to client side

2023-07-31 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-6623: --- Issue Type: Bug (was: Improvement) > Move the sorting of HoodieMetaserverClient instants from meta server side to >

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants from meta server side to client side

2023-07-31 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-6623: --- Issue Type: Improvement (was: Bug) > Move the sorting of HoodieMetaserverClient instants from meta server side to >

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants from meta server side to client side

2023-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6623: - Labels: pull-request-available (was: ) > Move the sorting of HoodieMetaserverClient instants from

[GitHub] [hudi] nylqd commented on issue #9269: [SUPPORT] Hudi HMS Catalog hive_sync.conf.dir

2023-07-31 Thread via GitHub
nylqd commented on issue #9269: URL: https://github.com/apache/hudi/issues/9269#issuecomment-1659637537 > we apply a fix as follow, you may implement a general solution for the community ![snapshot](https://github.com/apache/hudi/assets/3632490/53bbba09-af43-417d-8025-9abdfa6

[GitHub] [hudi] eric9204 opened a new pull request, #9331: [HUDI-6623] Move the sorting of HoodieMetaserverClient instants from …

2023-07-31 Thread via GitHub
eric9204 opened a new pull request, #9331: URL: https://github.com/apache/hudi/pull/9331 …meta server side to client side ### Change Logs Move the sorting of HoodieMetaserverClient instants from meta server side to client side. Avoid potential disorder problems. ### Impa

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants to meta server client side.

2023-07-31 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-6623: --- Description: Move the sorting of HoodieMetaserverClient instants from meta server side to client side. Avoid potential

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants from meta server side to client side

2023-07-31 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-6623: --- Summary: Move the sorting of HoodieMetaserverClient instants from meta server side to client side (was: Move the sort

[jira] [Updated] (HUDI-6623) Move the sorting of HoodieMetaserverClient instants to meta server client side.

2023-07-31 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-6623: --- Summary: Move the sorting of HoodieMetaserverClient instants to meta server client side. (was: Move the sorting of in

[jira] [Created] (HUDI-6623) Move the sorting of instants to meta server client side.

2023-07-31 Thread eric (Jira)
eric created HUDI-6623: -- Summary: Move the sorting of instants to meta server client side. Key: HUDI-6623 URL: https://issues.apache.org/jira/browse/HUDI-6623 Project: Apache Hudi Issue Type: Bug Af

[GitHub] [hudi] empcl commented on a diff in pull request #9297: Generate test jars for hudi-utilities and hudi-hive-sync modules

2023-07-31 Thread via GitHub
empcl commented on code in PR #9297: URL: https://github.com/apache/hudi/pull/9297#discussion_r1280149897 ## hudi-sync/hudi-hive-sync/pom.xml: ## @@ -200,6 +200,9 @@ + + false + Review Comment: I'm also surprised t

[jira] [Updated] (HUDI-6622) Reuse the table config from HoodieTableMetaClient in the HoodieTableMetaserverClient

2023-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6622: - Labels: pull-request-available (was: ) > Reuse the table config from HoodieTableMetaClient in the

[GitHub] [hudi] eric9204 opened a new pull request, #9330: [HUDI-6622] Reuse the table config from HoodieTableMetaClient in the …

2023-07-31 Thread via GitHub
eric9204 opened a new pull request, #9330: URL: https://github.com/apache/hudi/pull/9330 …HoodieTableMetaserverClient ### Change Logs Reuse the table config from HoodieTableMetaClient in the HoodieTableMetaserverClient, to avoid the loss of table parameters ### Impact

[jira] [Created] (HUDI-6622) Reuse the table config from HoodieTableMetaClient in the HoodieTableMetaserverClient

2023-07-31 Thread eric (Jira)
eric created HUDI-6622: -- Summary: Reuse the table config from HoodieTableMetaClient in the HoodieTableMetaserverClient Key: HUDI-6622 URL: https://issues.apache.org/jira/browse/HUDI-6622 Project: Apache Hudi

[GitHub] [hudi] hudi-bot commented on pull request #9325: Fix to explain the no commit instant exception when batch read

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9325: URL: https://github.com/apache/hudi/pull/9325#issuecomment-1659598490 ## CI report: * d75dcbd912872534fedf4b5f23542fdf7e6be993 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1893

[GitHub] [hudi] hudi-bot commented on pull request #9328: [MINOR] Remove irrelevant comments

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9328: URL: https://github.com/apache/hudi/pull/9328#issuecomment-1659584553 ## CI report: * a1bcf36dad6927b928e1839e7aaf7c0fb3c4967f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1894

[GitHub] [hudi] hudi-bot commented on pull request #9324: [HUDI-6619] [WIP] Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9324: URL: https://github.com/apache/hudi/pull/9324#issuecomment-1659584496 ## CI report: * 98e49fad21b4c7b1151e96c7a72b18caf5014a7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1893

[GitHub] [hudi] xushiyan commented on a diff in pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-07-31 Thread via GitHub
xushiyan commented on code in PR #8697: URL: https://github.com/apache/hudi/pull/8697#discussion_r1280116139 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -429,6 +416,40 @@ object HoodieSparkSqlWriter { } }

[GitHub] [hudi] amrishlal commented on pull request #9324: [HUDI-6619] [WIP] Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

2023-07-31 Thread via GitHub
amrishlal commented on PR #9324: URL: https://github.com/apache/hudi/pull/9324#issuecomment-1659571910 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] chandu-1101 opened a new issue, #9329: [SUPPORT] Hudi upsert takes more time than merging using spark sql

2023-07-31 Thread via GitHub
chandu-1101 opened a new issue, #9329: URL: https://github.com/apache/hudi/issues/9329 Issue: 1. I have 39GB parquet file on s3 which is ingested into Apache hudi. This is snappy compressed. 2. I have 147GB json file-s on s3 representing CDC data. This is CDC from mongo db. 3

[GitHub] [hudi] hudi-bot commented on pull request #9328: [MINOR] Remove irrelevant comments

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9328: URL: https://github.com/apache/hudi/pull/9328#issuecomment-1659550754 ## CI report: * a1bcf36dad6927b928e1839e7aaf7c0fb3c4967f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1659544576 ## CI report: * b6afe889ca6b47f4d1d934bb552cc1c489f9d0af UNKNOWN * c0b4aa4f64c37ee755284ce1b905459830d27ba9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280086564 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] SteNicholas commented on pull request #9211: [HUDI-6540] Support failed writes clean policy for Flink

2023-07-31 Thread via GitHub
SteNicholas commented on PR #9211: URL: https://github.com/apache/hudi/pull/9211#issuecomment-1659531558 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280086564 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280085427 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] eric9204 opened a new pull request, #9328: [MINOR] Remove irrelevant comments

2023-07-31 Thread via GitHub
eric9204 opened a new pull request, #9328: URL: https://github.com/apache/hudi/pull/9328 ### Change Logs NONE ### Impact NONE ### Risk level (write none, low medium or high below) NONE ### Documentation Update NONE - _The config descript

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280082412 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280082319 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280082204 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] yihua commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
yihua commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659516558 cc @vinothchandar @nsivabalan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] hudi-bot commented on pull request #9327: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9327: URL: https://github.com/apache/hudi/pull/9327#issuecomment-1659516360 ## CI report: * 4466920375cd45408723c7f9c5de65082832d502 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1894

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280075324 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9280: [HUDI-6587] Handle hollow commit for time travel query

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9280: URL: https://github.com/apache/hudi/pull/9280#discussion_r1280073349 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java: ## @@ -343,7 +343,8 @@ public static HoodieTimeline handleHollowCommitIfNeeded(H

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280074900 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] hudi-bot commented on pull request #9327: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9327: URL: https://github.com/apache/hudi/pull/9327#issuecomment-1659511241 ## CI report: * 4466920375cd45408723c7f9c5de65082832d502 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9324: [HUDI-6619] [WIP] Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9324: URL: https://github.com/apache/hudi/pull/9324#issuecomment-1659511199 ## CI report: * 98e49fad21b4c7b1151e96c7a72b18caf5014a7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1893

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280074068 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java: ## @@ -18,75 +18,125 @@ package org.apache.hudi.common.table.timel

[GitHub] [hudi] vinothchandar commented on a diff in pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
vinothchandar commented on code in PR #9209: URL: https://github.com/apache/hudi/pull/9209#discussion_r1280073859 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/ActiveInstant.java: ## @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9278: [HUDI-6312] Rename enum values of `HollowCommitHandling`

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9278: URL: https://github.com/apache/hudi/pull/9278#discussion_r1280073650 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieCommonConfig.java: ## @@ -78,18 +78,18 @@ public class HoodieCommonConfig extends HoodieConfig { p

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9278: [HUDI-6312] Rename enum values of `HollowCommitHandling`

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9278: URL: https://github.com/apache/hudi/pull/9278#discussion_r1280072375 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieCommonConfig.java: ## @@ -78,18 +78,18 @@ public class HoodieCommonConfig extends HoodieConfig { p

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #9275: [HUDI-6584] Abstract commit in CommitActionExecutor

2023-07-31 Thread via GitHub
Zouxxyy commented on code in PR #9275: URL: https://github.com/apache/hudi/pull/9275#discussion_r1280071700 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java: ## @@ -223,47 +221,11 @@ protected void comm

[GitHub] [hudi] hudi-bot commented on pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9209: URL: https://github.com/apache/hudi/pull/9209#issuecomment-1659505428 ## CI report: * 8f2dc4ec3e26f1908ae5d15f194bf70ca7dab27e UNKNOWN * 9e7266d31156421a129d11dbde82be2fc4fd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] danny0405 commented on a diff in pull request #9325: Fix to explain the no commit instant exception when batch read

2023-07-31 Thread via GitHub
danny0405 commented on code in PR #9325: URL: https://github.com/apache/hudi/pull/9325#discussion_r1280065719 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java: ## @@ -334,6 +334,9 @@ private List buildInputSplits() { HoodieTable

[GitHub] [hudi] eric9204 opened a new pull request, #9327: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable

2023-07-31 Thread via GitHub
eric9204 opened a new pull request, #9327: URL: https://github.com/apache/hudi/pull/9327 ### Change Logs make `HoodieRecordDelegate ` implement `KryoSerializable` ### Impact Improve serialize/deserialize performance. ### Risk level (write none, low medium or high b

[GitHub] [hudi] hudi-bot commented on pull request #9317: [MINOR] Simplify CreateHoodieTableCommand logWarning

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9317: URL: https://github.com/apache/hudi/pull/9317#issuecomment-1659477620 ## CI report: * 4e8de54b6fc64ee0a728e721510dfe73ace9c6cb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1891

[GitHub] [hudi] hudi-bot commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659477588 ## CI report: * 7aa72de28882934a1ae1d93f18822bd8edc80d9d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=189

[GitHub] [hudi] hudi-bot commented on pull request #9312: [HUDI-6609] Reverting multi writer checkpointing with HoodieStreamer

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9312: URL: https://github.com/apache/hudi/pull/9312#issuecomment-1659477543 ## CI report: * a31095e3549d2a60698b8f10351e96cfc7c02298 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1888

[GitHub] [hudi] hudi-bot commented on pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9311: URL: https://github.com/apache/hudi/pull/9311#issuecomment-1659477529 ## CI report: * 426d3cbffbd114cc644c92fd7ee24e6dcc997990 UNKNOWN * f3239197e53a3f0e5c223e9fe709d8d336542967 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #9209: [HUDI-6539] New LSM tree style archived timeline

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9209: URL: https://github.com/apache/hudi/pull/9209#issuecomment-1659477292 ## CI report: * 8f2dc4ec3e26f1908ae5d15f194bf70ca7dab27e UNKNOWN * 9e7266d31156421a129d11dbde82be2fc4fd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #9229: [HUDI-6565] Spark offline compaction add failed retry mechanism

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9229: URL: https://github.com/apache/hudi/pull/9229#issuecomment-1659477358 ## CI report: * cac84b6d8d4244c5d33af16e9982338b119937a7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1887

[GitHub] [hudi] eric9204 closed pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer

2023-07-31 Thread via GitHub
eric9204 closed pull request #9176: [HUDI-6523] fix get valid checkpoint for current writer URL: https://github.com/apache/hudi/pull/9176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] eric9204 closed pull request #9326: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable

2023-07-31 Thread via GitHub
eric9204 closed pull request #9326: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable URL: https://github.com/apache/hudi/pull/9326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #9317: [MINOR] Simplify CreateHoodieTableCommand logWarning

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9317: URL: https://github.com/apache/hudi/pull/9317#issuecomment-1659471792 ## CI report: * 4e8de54b6fc64ee0a728e721510dfe73ace9c6cb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1891

[GitHub] [hudi] hudi-bot commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659471755 ## CI report: * 8366afa4faf338c62f28d3277e246a754e7a6a86 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=189

[GitHub] [hudi] hudi-bot commented on pull request #9312: [HUDI-6609] Reverting multi writer checkpointing with HoodieStreamer

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9312: URL: https://github.com/apache/hudi/pull/9312#issuecomment-1659471695 ## CI report: * a31095e3549d2a60698b8f10351e96cfc7c02298 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1888

[GitHub] [hudi] hudi-bot commented on pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9311: URL: https://github.com/apache/hudi/pull/9311#issuecomment-1659471635 ## CI report: * 426d3cbffbd114cc644c92fd7ee24e6dcc997990 UNKNOWN * f3239197e53a3f0e5c223e9fe709d8d336542967 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #9229: [HUDI-6565] Spark offline compaction add failed retry mechanism

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9229: URL: https://github.com/apache/hudi/pull/9229#issuecomment-1659471420 ## CI report: * cac84b6d8d4244c5d33af16e9982338b119937a7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1887

[GitHub] [hudi] eric9204 opened a new pull request, #9326: [HUDI-6617] make HoodieRecordDelegate implement KryoSerializable

2023-07-31 Thread via GitHub
eric9204 opened a new pull request, #9326: URL: https://github.com/apache/hudi/pull/9326 ### Change Logs make HoodieRecordDelegate implement KryoSerializable ### Impact improve serialize/deserialize performance. _ ### Risk level (write none, low medium or high belo

[GitHub] [hudi] hudi-bot commented on pull request #9325: Fix to explain the no commit instant exception when batch read

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9325: URL: https://github.com/apache/hudi/pull/9325#issuecomment-1659465111 ## CI report: * d75dcbd912872534fedf4b5f23542fdf7e6be993 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1893

[GitHub] [hudi] danny0405 commented on a diff in pull request #9275: [HUDI-6584] Abstract commit in CommitActionExecutor

2023-07-31 Thread via GitHub
danny0405 commented on code in PR #9275: URL: https://github.com/apache/hudi/pull/9275#discussion_r1280043171 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java: ## @@ -223,47 +221,11 @@ protected void co

[GitHub] [hudi] yihua commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
yihua commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659457425 We also need to add a downgrade step: for downgrading a table from v6 to v5, we need to check any v3 delete blocks using the new format and ask user to manually restore to a commit before any

[jira] [Updated] (HUDI-6621) Add a downgrade step from 6 to 5 to detect new delete blocks

2023-07-31 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6621: Description: In table version 6, we introduce a new delete block format (v3) with Avro serde (HUDI-5760).  F

[jira] [Assigned] (HUDI-6621) Add a downgrade step from 6 to 5 to detect new delete blocks

2023-07-31 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-6621: --- Assignee: Ethan Guo > Add a downgrade step from 6 to 5 to detect new delete blocks >

[jira] [Updated] (HUDI-6621) Add a downgrade step from 6 to 5 to detect new delete blocks

2023-07-31 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6621: Fix Version/s: 0.14.0 > Add a downgrade step from 6 to 5 to detect new delete blocks > -

[jira] [Created] (HUDI-6621) Add a downgrade step from 6 to 5 to detect new delete blocks

2023-07-31 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-6621: --- Summary: Add a downgrade step from 6 to 5 to detect new delete blocks Key: HUDI-6621 URL: https://issues.apache.org/jira/browse/HUDI-6621 Project: Apache Hudi Issue T

[GitHub] [hudi] yihua commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
yihua commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659453598 > Do we also need to upgrade table version? I know we're already handling different versions. But, jsut asking what's the convention here? Did we update table version last time? We need

[GitHub] [hudi] danny0405 commented on a diff in pull request #9317: [MINOR] Simplify CreateHoodieTableCommand logWarning

2023-07-31 Thread via GitHub
danny0405 commented on code in PR #9317: URL: https://github.com/apache/hudi/pull/9317#discussion_r1280038324 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableCommand.scala: ## @@ -84,11 +84,7 @@ case class CreateHoodie

[hudi] branch master updated (32c9a633f6c -> f46db1d4a31)

2023-07-31 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 32c9a633f6c [HUDI-6605] Write handlers not close with try block in ClientIds (#9309) add f46db1d4a31 [MINOR] Se

[GitHub] [hudi] danny0405 merged pull request #9323: [MINOR] Set sort memory only when sortClusteringEnabled

2023-07-31 Thread via GitHub
danny0405 merged PR #9323: URL: https://github.com/apache/hudi/pull/9323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] hudi-bot commented on pull request #9325: Fix to explain the no commit instant exception when batch read

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9325: URL: https://github.com/apache/hudi/pull/9325#issuecomment-1659434895 ## CI report: * d75dcbd912872534fedf4b5f23542fdf7e6be993 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659434838 ## CI report: * 8366afa4faf338c62f28d3277e246a754e7a6a86 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=189

[GitHub] [hudi] hudi-bot commented on pull request #9261: [HUDI-6579] Adding support for upsert and deletes with spark datasource for pk less table

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9261: URL: https://github.com/apache/hudi/pull/9261#issuecomment-1659434634 ## CI report: * 84ef26635cf016f0e4466754ad89f8b70b66d889 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1893

[jira] [Closed] (HUDI-6605) Add compaction/logcompaction writestatus errors check and advance it

2023-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6605. Resolution: Fixed Fixed via master branch: 32c9a633f6cf83faf7e3c4ad32e28c52cce0049f > Add compaction/logcom

[hudi] branch master updated: [HUDI-6605] Write handlers not close with try block in ClientIds (#9309)

2023-07-31 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 32c9a633f6c [HUDI-6605] Write handlers not clos

[GitHub] [hudi] danny0405 merged pull request #9309: [HUDI-6605] Write handlers not close with try block in ClientIds

2023-07-31 Thread via GitHub
danny0405 merged PR #9309: URL: https://github.com/apache/hudi/pull/9309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] danny0405 commented on pull request #9318: [HUDI-6617] Fix HoodieRecordDelegate NotSerializableException

2023-07-31 Thread via GitHub
danny0405 commented on PR #9318: URL: https://github.com/apache/hudi/pull/9318#issuecomment-1659430643 Does Spark uses Kryo serialization for this objects, it it is, that makes sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] hudi-bot commented on pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9315: URL: https://github.com/apache/hudi/pull/9315#issuecomment-1659428849 ## CI report: * ae048ad8f890264db36b52091ce639747f2a9f2c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1891

[GitHub] [hudi] hudi-bot commented on pull request #9261: [HUDI-6579] Adding support for upsert and deletes with spark datasource for pk less table

2023-07-31 Thread via GitHub
hudi-bot commented on PR #9261: URL: https://github.com/apache/hudi/pull/9261#issuecomment-1659428628 ## CI report: * f67df4f8bac07f798b0c376440cc30b878dcff6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1876

[jira] [Closed] (HUDI-6567) ExpressionEvaluators numeric types conversion support

2023-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6567. Resolution: Fixed Fixed via master branch: 34afa6784eab08fa994f7d58f0c37f197d47a3f1 > ExpressionEvaluators

[hudi] branch master updated (f7c07ec3194 -> 34afa6784ea)

2023-07-31 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from f7c07ec3194 [MINOR] fix update metric when record not be updated (#9316) add 34afa6784ea [HUDI-6567] Expression

[GitHub] [hudi] danny0405 merged pull request #9230: [HUDI-6567] ExpressionEvaluators numeric types conversion support

2023-07-31 Thread via GitHub
danny0405 merged PR #9230: URL: https://github.com/apache/hudi/pull/9230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] waywtdcc commented on pull request #9228: [HUDI-6563]Supports flink lookup join

2023-07-31 Thread via GitHub
waywtdcc commented on PR #9228: URL: https://github.com/apache/hudi/pull/9228#issuecomment-1659426459 > > > Thanks for the contribution @waywtdcc , can you explain in high level how the hudi table is loaded and what is the refresh strategy of the table ? > > > > > > The FileSystem

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9311: URL: https://github.com/apache/hudi/pull/9311#discussion_r1280016059 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java: ## Review Comment: Entire RLI is a new addition. we did not have one with 0.13.0.

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9311: URL: https://github.com/apache/hudi/pull/9311#discussion_r1280015823 ## hudi-common/src/main/avro/HoodieMetadata.avsc: ## Review Comment: we need more jamming on rowId. thats why -- This is an automated message from the Apache G

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9311: URL: https://github.com/apache/hudi/pull/9311#discussion_r1280015743 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java: ## @@ -166,11 +166,15 @@ public class HoodieMetadataPayload implements HoodieRecordPa

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9311: [HUDI-6607] Fixing RLI schema to support different fileID formats

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9311: URL: https://github.com/apache/hudi/pull/9311#discussion_r1280015250 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -727,6 +728,11 @@ public class HoodieWriteConfig extends HoodieConf

[GitHub] [hudi] nsivabalan commented on pull request #9261: [HUDI-6579] Adding support for upsert and deletes with spark datasource for pk less table

2023-07-31 Thread via GitHub
nsivabalan commented on PR #9261: URL: https://github.com/apache/hudi/pull/9261#issuecomment-1659410591 @codope : addressed all comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9261: [HUDI-6579] Adding support for upsert and deletes with spark datasource for pk less table

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9261: URL: https://github.com/apache/hudi/pull/9261#discussion_r1280014102 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala: ## @@ -164,6 +164,18 @@ class DefaultSource extends RelationProvider

[GitHub] [hudi] yihua commented on a diff in pull request #9315: [HUDI-5760] Use Avro as serde for delete log blocks

2023-07-31 Thread via GitHub
yihua commented on code in PR #9315: URL: https://github.com/apache/hudi/pull/9315#discussion_r1280013405 ## hudi-common/src/test/java/org/apache/hudi/common/table/log/block/TestHoodieDeleteBlock.java: ## Review Comment: I wrote a test by writing both V2 and V3 delete block

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9261: [HUDI-6579] Adding support for upsert and deletes with spark datasource for pk less table

2023-07-31 Thread via GitHub
nsivabalan commented on code in PR #9261: URL: https://github.com/apache/hudi/pull/9261#discussion_r1280012474 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -172,6 +172,9 @@ object HoodieSparkSqlWriter { operat

[GitHub] [hudi] nsivabalan closed pull request #9057: [HUDI-6446][WIP] Fixing MDT commit time parsing and RLI instantiation with MDT

2023-07-31 Thread via GitHub
nsivabalan closed pull request #9057: [HUDI-6446][WIP] Fixing MDT commit time parsing and RLI instantiation with MDT URL: https://github.com/apache/hudi/pull/9057 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

  1   2   >