[GitHub] [hudi] XuQianJin-Stars commented on issue #5686: [DISCUSS] Is it possible to create a Chinese meeting?

2022-05-25 Thread GitBox
XuQianJin-Stars commented on issue #5686: URL: https://github.com/apache/hudi/issues/5686#issuecomment-1138183915 > @cdmikechen that's good idea. we need to have someone driving and organizing these events. We already have dingding and wechat groups. I think it only needs the

[GitHub] [hudi] codope merged pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
codope merged PR #5641: URL: https://github.com/apache/hudi/pull/5641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle (#5641)

2022-05-25 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 31e13db1f0 [HUDI-4023] Decouple hudi-spark from

[GitHub] [hudi] codope commented on a diff in pull request #5637: [HUDI-4124] Add valid check in Spark Datasource configs

2022-05-25 Thread GitBox
codope commented on code in PR #5637: URL: https://github.com/apache/hudi/pull/5637#discussion_r882337916 ## hudi-common/src/main/java/org/apache/hudi/common/config/ConfigProperty.java: ## @@ -95,33 +99,46 @@ Option>> getInferFunc() { return inferFunction; } + public

[GitHub] [hudi] XuQianJin-Stars commented on issue #5687: [SUPPORT]hudi sql parser ignores all exceptions of spark sql parser

2022-05-25 Thread GitBox
XuQianJin-Stars commented on issue #5687: URL: https://github.com/apache/hudi/issues/5687#issuecomment-1138181072 > > To convert values from int to timestamp, you can use functions > > In this case, the exception thrown should be the spark exception information, not the hudi parsing

[GitHub] [hudi] hudi-bot commented on pull request #4958: [HUDI-3558] Consistent bucket index: bucket resizing (split) & concurrent write during resizing

2022-05-25 Thread GitBox
hudi-bot commented on PR #4958: URL: https://github.com/apache/hudi/pull/4958#issuecomment-1138180221 ## CI report: * 9a74c2b306326453e1d1160da8c378a05711c3f8 Azure:

[GitHub] [hudi] melin commented on issue #5687: [SUPPORT]hudi sql parser ignores all exceptions of spark sql parser

2022-05-25 Thread GitBox
melin commented on issue #5687: URL: https://github.com/apache/hudi/issues/5687#issuecomment-1138179782 > To convert values from int to timestamp, you can use functions In this case, the exception thrown should be the spark exception information, not the hudi parsing failure

[GitHub] [hudi] hudi-bot commented on pull request #4958: [HUDI-3558] Consistent bucket index: bucket resizing (split) & concurrent write during resizing

2022-05-25 Thread GitBox
hudi-bot commented on PR #4958: URL: https://github.com/apache/hudi/pull/4958#issuecomment-1138178623 ## CI report: * 9a74c2b306326453e1d1160da8c378a05711c3f8 Azure:

[GitHub] [hudi] xushiyan commented on issue #5686: [DISCUSS] Is it possible to create a Chinese meeting?

2022-05-25 Thread GitBox
xushiyan commented on issue #5686: URL: https://github.com/apache/hudi/issues/5686#issuecomment-1138178486 @cdmikechen that's good idea. we need to have someone driving and organizing these events. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] xushiyan commented on issue #5687: [SUPPORT]hudi sql parser ignores all exceptions of spark sql parser

2022-05-25 Thread GitBox
xushiyan commented on issue #5687: URL: https://github.com/apache/hudi/issues/5687#issuecomment-1138176406 @melin what exactly are you suggesting here? can you elaborate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] YuweiXiao commented on pull request #4958: [HUDI-3558] Consistent bucket index: bucket resizing (split) & concurrent write during resizing

2022-05-25 Thread GitBox
YuweiXiao commented on PR #4958: URL: https://github.com/apache/hudi/pull/4958#issuecomment-1138162762 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-25 Thread GitBox
hudi-bot commented on PR #5436: URL: https://github.com/apache/hudi/pull/5436#issuecomment-1138160471 ## CI report: * 83759b5efe62aa91aa396a3fbee94aa41afe458d UNKNOWN * c7d28244fb8a8583d127be91b10bf1bf8536a88c UNKNOWN * 8fc0e6862a3e8c7f3a53c9c2d6857c4e237bb5ff UNKNOWN *

[jira] [Assigned] (HUDI-4040) Add customerColumnPartitionerRow support

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-4040: - Assignee: Hui An > Add customerColumnPartitionerRow support >

[jira] [Updated] (HUDI-4040) Add customerColumnPartitionerRow support

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4040: -- Fix Version/s: 0.12.0 > Add customerColumnPartitionerRow support >

[jira] [Closed] (HUDI-4040) Add customerColumnPartitionerRow support

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4040. - Resolution: Done > Add customerColumnPartitionerRow support > >

[jira] [Updated] (HUDI-4040) Add customerColumnPartitionerRow support

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4040: -- Status: In Progress (was: Open) > Add customerColumnPartitionerRow support >

[jira] [Updated] (HUDI-4040) Add customerColumnPartitionerRow support

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4040: -- Status: Patch Available (was: In Progress) > Add customerColumnPartitionerRow support >

[hudi] branch master updated: [HUDI-4040] Bulk insert Support CustomColumnsSortPartitioner with Row (#5502)

2022-05-25 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 98c5c6c654 [HUDI-4040] Bulk insert Support

[GitHub] [hudi] codope merged pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-25 Thread GitBox
codope merged PR #5502: URL: https://github.com/apache/hudi/pull/5502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #5659: [HUDI-4137] SnowflakeSyncTool MVP implementation to integrate with Snowflake

2022-05-25 Thread GitBox
hudi-bot commented on PR #5659: URL: https://github.com/apache/hudi/pull/5659#issuecomment-1138157534 ## CI report: * 223068cbc3deeac3f39b5592f3812d5b4643ad2d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5659: [HUDI-4137] SnowflakeSyncTool MVP implementation to integrate with Snowflake

2022-05-25 Thread GitBox
hudi-bot commented on PR #5659: URL: https://github.com/apache/hudi/pull/5659#issuecomment-1138156062 ## CI report: * 223068cbc3deeac3f39b5592f3812d5b4643ad2d Azure:

[GitHub] [hudi] codope commented on a diff in pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
codope commented on code in PR #5641: URL: https://github.com/apache/hudi/pull/5641#discussion_r882315881 ## packaging/hudi-utilities-slim-bundle/README.md: ## @@ -17,6 +17,89 @@ # Usage of hudi-utilities-slim-bundle -Starting from versions 0.11, Hudi provides

[GitHub] [hudi] codope commented on a diff in pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
codope commented on code in PR #5641: URL: https://github.com/apache/hudi/pull/5641#discussion_r882315243 ## packaging/hudi-utilities-slim-bundle/pom.xml: ## @@ -136,13 +134,6 @@ org.apache.kafka:kafka_${scala.binary.version}

[GitHub] [hudi] codope commented on a diff in pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
codope commented on code in PR #5641: URL: https://github.com/apache/hudi/pull/5641#discussion_r882315127 ## packaging/hudi-utilities-slim-bundle/pom.xml: ## @@ -92,9 +92,7 @@ org.apache.hudi:hudi-common

[GitHub] [hudi] hudi-bot commented on pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-25 Thread GitBox
hudi-bot commented on PR #5436: URL: https://github.com/apache/hudi/pull/5436#issuecomment-1138151727 ## CI report: * 83759b5efe62aa91aa396a3fbee94aa41afe458d UNKNOWN * c7d28244fb8a8583d127be91b10bf1bf8536a88c UNKNOWN * 8fc0e6862a3e8c7f3a53c9c2d6857c4e237bb5ff UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #5688: [HUDI-4157] fix Problem that IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread GitBox
hudi-bot commented on PR #5688: URL: https://github.com/apache/hudi/pull/5688#issuecomment-1138150552 ## CI report: * fa54c98bc27837b31de301152b8043455337b917 Azure:

[jira] [Closed] (HUDI-4134) Fix Method naming consistency issues in FSUtils

2022-05-25 Thread Heap (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heap closed HUDI-4134. -- Resolution: Fixed > Fix Method naming consistency issues in FSUtils > --- >

[GitHub] [hudi] eshu opened a new issue, #5689: [SUPPORT] Duplicates appears on the some attempts of rewrite

2022-05-25 Thread GitBox
eshu opened a new issue, #5689: URL: https://github.com/apache/hudi/issues/5689 Dataset has two stages: the initial upload from a snapshot (insert operations) and after that updates happens on demand from Kafka (upserts). I checked the snapshot, it does not contain any duplicates.

[GitHub] [hudi] qianchutao commented on issue #5553: [SUPPORT] java.io.EOFException: Invalid position - exceeds the bounds of the stream

2022-05-25 Thread GitBox
qianchutao commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1138124161 > Okey, our user feedback shows that the bug is fixed, maybe there are some other cases that cause this problem too, did you use the MOR table and what is your checkpoint interval ?

[GitHub] [hudi] qianchutao commented on issue #5553: [SUPPORT] java.io.EOFException: Invalid position - exceeds the bounds of the stream

2022-05-25 Thread GitBox
qianchutao commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1138123328 > Okey, our user feedback shows that the bug is fixed, maybe there are some other cases that cause this problem too, did you use the MOR table and what is your

[GitHub] [hudi] YannByron commented on a diff in pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-25 Thread GitBox
YannByron commented on code in PR #5436: URL: https://github.com/apache/hudi/pull/5436#discussion_r882291071 ## rfc/rfc-51/rfc-51.md: ## @@ -0,0 +1,234 @@ + + +# RFC-50: Hudi CDC + +# Proposers + +- @Yann Byron + +# Approvers + +- @Raymond + +# Statue +JIRA:

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1138123017 ## CI report: * b0d09ce1d2fcff16407a64f3127e1ce5218e6be5 UNKNOWN * 538fdfbd216be49ea3652af8460333622a68d0ff Azure:

[jira] [Resolved] (HUDI-4083) Fix the flink application fails to start due to uncompleted archiving deletion

2022-05-25 Thread Ying Lin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Lin resolved HUDI-4083. > Fix the flink application fails to start due to uncompleted archiving deletion >

[GitHub] [hudi] danny0405 commented on issue #5553: [SUPPORT] java.io.EOFException: Invalid position - exceeds the bounds of the stream

2022-05-25 Thread GitBox
danny0405 commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1138118194 Okey, our user feedback shows that the bug is fixed, maybe there are some other cases that cause this problem too, did you use the MOR table and what is your checkpoint interval ?

[GitHub] [hudi] BruceKellan closed pull request #5556: [HUDI-4083] Fix the flink application fails to start due to uncomplet…

2022-05-25 Thread GitBox
BruceKellan closed pull request #5556: [HUDI-4083] Fix the flink application fails to start due to uncomplet… URL: https://github.com/apache/hudi/pull/5556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] BruceKellan commented on pull request #5556: [HUDI-4083] Fix the flink application fails to start due to uncomplet…

2022-05-25 Thread GitBox
BruceKellan commented on PR #5556: URL: https://github.com/apache/hudi/pull/5556#issuecomment-1138117817 I will close this pr because other prs have solved similar problems. Related to [HUDI-4145](https://github.com/apache/hudi/pull/5676). -- This is an automated message from the

[GitHub] [hudi] danny0405 commented on pull request #5556: [HUDI-4083] Fix the flink application fails to start due to uncomplet…

2022-05-25 Thread GitBox
danny0405 commented on PR #5556: URL: https://github.com/apache/hudi/pull/5556#issuecomment-1138115108 Close because HUDI-4145 already resolved it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (HUDI-4145) Archives the metadata file in HoodieInstant.State sequence

2022-05-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17542323#comment-17542323 ] Danny Chen commented on HUDI-4145: -- A following patch via master branch:

[hudi] branch master updated: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequence (part2) (#5676)

2022-05-25 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 4e42ed5eae [HUDI-4145] Archives the metadata

[GitHub] [hudi] danny0405 merged pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-25 Thread GitBox
danny0405 merged PR #5676: URL: https://github.com/apache/hudi/pull/5676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #5688: [HUDI-4157] fix Problem that IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread GitBox
hudi-bot commented on PR #5688: URL: https://github.com/apache/hudi/pull/5688#issuecomment-1138094352 ## CI report: * fa54c98bc27837b31de301152b8043455337b917 Azure:

[jira] [Updated] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4157: --- Description: 1、Run the “commit rollback --commit “ command for a completed commit by hudi-cli. 2、We can

[GitHub] [hudi] hudi-bot commented on pull request #5688: [HUDI-4157] fix Problem that IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread GitBox
hudi-bot commented on PR #5688: URL: https://github.com/apache/hudi/pull/5688#issuecomment-1138091978 ## CI report: * fa54c98bc27837b31de301152b8043455337b917 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4157: --- Description: 1、Run the “commit rollback --commit “ command for a completed commit by hudi-cli. 2、We can

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-25 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1138089408 ## CI report: * d84585bf3018a6cb9795059cfa24a8f61affa023 Azure:

[jira] [Updated] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4157: - Labels: pull-request-available (was: ) > IllegalArgumentException is thrown when the rollback is

[GitHub] [hudi] watermelon12138 opened a new pull request, #5688: [HUDI-4157] fix Problem that IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread GitBox
watermelon12138 opened a new pull request, #5688: URL: https://github.com/apache/hudi/pull/5688 …the rollback is executed for a completed commit. ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review

[jira] [Updated] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4157: --- Description: 1、Run the “rollback --commit “ command for a completed commit by hudi-cli. 2、We can find

[jira] [Updated] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan updated HUDI-4157: --- Description: 1、Run the “rollback --commit “ command for a completed commit by hudi-cli. 2、We can find

[GitHub] [hudi] melin opened a new issue, #5687: [SUPPORT]hudi sql parser ignores all exceptions of spark sql parser

2022-05-25 Thread GitBox
melin opened a new issue, #5687: URL: https://github.com/apache/hudi/issues/5687 example: > spark.sql.ansi.enabled = true select CAST(-123456789 AS TIMESTAMP) as de, exception: ``` Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve

[jira] [Created] (HUDI-4157) IllegalArgumentException is thrown when the rollback is executed for a completed commit.

2022-05-25 Thread YangXuan (Jira)
YangXuan created HUDI-4157: -- Summary: IllegalArgumentException is thrown when the rollback is executed for a completed commit. Key: HUDI-4157 URL: https://issues.apache.org/jira/browse/HUDI-4157 Project:

[GitHub] [hudi] zhengminhz commented on issue #1887: [SUPPORT] docker-demo : Earliest write inflight instant time must be later than compaction time

2022-05-25 Thread GitBox
zhengminhz commented on issue #1887: URL: https://github.com/apache/hudi/issues/1887#issuecomment-1138074885 @bvaradar so how to solve this problem?there is a realtime job.there is another batch job need to delete some old data,it can't do the compact -- This is an automated message

[GitHub] [hudi] boneanxs commented on a diff in pull request #5679: [MINOR] Fix the issue when handling conf hoodie.datasource.write.operation=bulk_insert in sql mode

2022-05-25 Thread GitBox
boneanxs commented on code in PR #5679: URL: https://github.com/apache/hudi/pull/5679#discussion_r882258541 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -129,7 +129,8 @@ trait ProvidesHoodieConfig extends

[GitHub] [hudi] YannByron commented on a diff in pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-25 Thread GitBox
YannByron commented on code in PR #5436: URL: https://github.com/apache/hudi/pull/5436#discussion_r882254049 ## rfc/rfc-51/rfc-51.md: ## @@ -0,0 +1,234 @@ + + +# RFC-50: Hudi CDC + +# Proposers + +- @Yann Byron + +# Approvers + +- @Raymond + +# Statue +JIRA:

[GitHub] [hudi] xccui commented on issue #5553: [SUPPORT] java.io.EOFException: Invalid position - exceeds the bounds of the stream

2022-05-25 Thread GitBox
xccui commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1138062379 > Have fired a fix for flink here: #5660 > > https://issues.apache.org/jira/browse/HUDI-3782 and https://issues.apache.org/jira/browse/HUDI-4138 may cause this bug. > > The

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1138061542 ## CI report: * 479159c38df30bc0aad7ac2ead1ba972477fbf72 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1138059723 ## CI report: * 479159c38df30bc0aad7ac2ead1ba972477fbf72 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1138057915 ## CI report: * 479159c38df30bc0aad7ac2ead1ba972477fbf72 Azure:

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-25 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1138057358 > Another detail is that we tweaked the archive strategy when metadata table is enabled for 0.11, #4821, maybe it is relevant with your error. After the program runs for

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-25 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1138056176 ## CI report: * 36161bf2dea866c5aecc745af9109e66b2af0726 Azure:

[GitHub] [hudi] sunke38 commented on issue #5684: [SUPPORT]CreateHoodieTableCommand.orgsetter$nodePatterns_$eq(Lscala/collection/Seq;)V is abstract

2022-05-25 Thread GitBox
sunke38 commented on issue #5684: URL: https://github.com/apache/hudi/issues/5684#issuecomment-1138055328 @xushiyan sorry for that, I modify style. Here is. I change key to spark.sql.extensions but still show it. By the way, sql of create table is work in spark-shell -- This is an

[GitHub] [hudi] sunke38 commented on issue #5673: [SUPPORT]scala.MatchError for rename colum

2022-05-25 Thread GitBox
sunke38 commented on issue #5673: URL: https://github.com/apache/hudi/issues/5673#issuecomment-1138047204 @xushiyan I did this in spark shell ,so only code is ''' spark.sql("ALTER TABLE hudi_cow_pt_tbl RENAME COLUMN ts TO tts") ''' here is the command for spark shell '''

[GitHub] [hudi] qianchutao commented on issue #5553: [SUPPORT] java.io.EOFException: Invalid position - exceeds the bounds of the stream

2022-05-25 Thread GitBox
qianchutao commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1138040724 i had the same problem, i used flink 1.14.0 stream write to hudi 0.11.0

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-25 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1138033419 ## CI report: * 36161bf2dea866c5aecc745af9109e66b2af0726 Azure:

[jira] [Created] (HUDI-4156) AsyncIndexer fails for column stats partition

2022-05-25 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4156: - Summary: AsyncIndexer fails for column stats partition Key: HUDI-4156 URL: https://issues.apache.org/jira/browse/HUDI-4156 Project: Apache Hudi

[GitHub] [hudi] cdmikechen opened a new issue, #5686: [DISCUSS] Is it possible to create a Chinese meeting?

2022-05-25 Thread GitBox
cdmikechen opened a new issue, #5686: URL: https://github.com/apache/hudi/issues/5686 At present, many Chinese companies are using Hudi, and there are many Chinese contributors/commiters in the community. If we can provide a channel that can communicate in Chinese, I think many people

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1137989271 ## CI report: * 479159c38df30bc0aad7ac2ead1ba972477fbf72 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-05-25 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1137984546 ## CI report: * 479159c38df30bc0aad7ac2ead1ba972477fbf72 Azure:

[GitHub] [hudi] xushiyan commented on issue #5673: [SUPPORT]scala.MatchError for rename colum

2022-05-25 Thread GitBox
xushiyan commented on issue #5673: URL: https://github.com/apache/hudi/issues/5673#issuecomment-1137951296 @sunke38 can you share your complete command to start the spark session -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] xushiyan commented on issue #5684: [SUPPORT]CreateHoodieTableCommand.orgsetter$nodePatterns_$eq(Lscala/collection/Seq;)V is abstract

2022-05-25 Thread GitBox
xushiyan commented on issue #5684: URL: https://github.com/apache/hudi/issues/5684#issuecomment-1137943564 @sunke38 it's hard to read. can you pls format your code and stacktrace separately with proper block code format like ```java print ``` -- This is an automated message from

[GitHub] [hudi] lanyuanxiaoyao commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-25 Thread GitBox
lanyuanxiaoyao commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1137941736 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] Gatsby-Lee commented on issue #5636: [SUPPORT]Need info on the versions of hudi dependent jars which can be used with Glue 3.0

2022-05-25 Thread GitBox
Gatsby-Lee commented on issue #5636: URL: https://github.com/apache/hudi/issues/5636#issuecomment-1137922706 @tjtoll + anyone who use AWS Glue If you have to use Glue3, then go with Hudi 0.9 that you can get from AWS Marketplace. you don't need to download anything manually since

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-25 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1137892349 > @xushiyan I tried with sync mode as glue and it also didn't work. "hoodie.datasource.hive_sync.mode" : "glue" > >

[GitHub] [hudi] hudi-bot commented on pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5641: URL: https://github.com/apache/hudi/pull/5641#issuecomment-1137750178 ## CI report: * 6cbe9ef01b44d6897578ebe6203d0670d2629e19 Azure:

[GitHub] [hudi] tchunchu commented on issue #5636: [SUPPORT]Need info on the versions of hudi dependent jars which can be used with Glue 3.0

2022-05-25 Thread GitBox
tchunchu commented on issue #5636: URL: https://github.com/apache/hudi/issues/5636#issuecomment-1137734544 @tjtoll possible duplicate of https://github.com/apache/hudi/issues/5484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] tchunchu commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-25 Thread GitBox
tchunchu commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1137733672 @xushiyan I tried with sync mode as glue and it also didn't work. "hoodie.datasource.hive_sync.mode" : "glue"

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1137654784 ## CI report: * b35aa8a0bfb61febc72ab39e15f4bd3eb8ef210d Azure:

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-25 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1137575389 @xushiyan hi. after I had a chat with you yesterday, I did some self-question / answer, and also some investigation in Glue and Hudi and Hive code base. Here is the my

[GitHub] [hudi] hudi-bot commented on pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5641: URL: https://github.com/apache/hudi/pull/5641#issuecomment-1137563180 ## CI report: * b6bee5ad68425261adff75c480b0d78ae663e927 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5641: URL: https://github.com/apache/hudi/pull/5641#issuecomment-1137558259 ## CI report: * b6bee5ad68425261adff75c480b0d78ae663e927 Azure:

[GitHub] [hudi] codope commented on pull request #5449: [HUDI-3911][DOCS][WIP] Add async indexing doc

2022-05-25 Thread GitBox
codope commented on PR #5449: URL: https://github.com/apache/hudi/pull/5449#issuecomment-1137550267 Closing this PR #5476 already merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] codope closed pull request #5449: [HUDI-3911][DOCS][WIP] Add async indexing doc

2022-05-25 Thread GitBox
codope closed pull request #5449: [HUDI-3911][DOCS][WIP] Add async indexing doc URL: https://github.com/apache/hudi/pull/5449 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #5678: [HUDI-4153]StreamReadOperator supports async opeing of foramt.

2022-05-25 Thread GitBox
hudi-bot commented on PR #5678: URL: https://github.com/apache/hudi/pull/5678#issuecomment-1137543203 ## CI report: * 70f47485c23aca09d1ff04489b41875c7a5c096d UNKNOWN * 72f368df6c8d7dafd5389ef10931a91fbd9dd771 Azure:

[GitHub] [hudi] tjtoll commented on issue #5636: [SUPPORT]Need info on the versions of hudi dependent jars which can be used with Glue 3.0

2022-05-25 Thread GitBox
tjtoll commented on issue #5636: URL: https://github.com/apache/hudi/issues/5636#issuecomment-1137491859 I’m not sure what the next thing to try is, we do not convert to dynamic frames before we write > On May 25, 2022, at 8:20 AM, Kapil Kumar Joshi ***@***.***> wrote: > > 

[GitHub] [hudi] codope commented on a diff in pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-25 Thread GitBox
codope commented on code in PR #5641: URL: https://github.com/apache/hudi/pull/5641#discussion_r881831831 ## pom.xml: ## @@ -557,6 +558,11 @@ jersey-container-servlet-core ${glassfish.version} + Review Comment: not needed for compilation..

[GitHub] [hudi] hudi-bot commented on pull request #5681: [HUDI-4148] Preparations and client for hudi table manager service

2022-05-25 Thread GitBox
hudi-bot commented on PR #5681: URL: https://github.com/apache/hudi/pull/5681#issuecomment-1137460677 ## CI report: * 6566d731e9d462a78a2a85a37c09d5a24ffc45fb UNKNOWN * f486cddda77a38dc493e042986d30f0f1ae5d687 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-25 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1137460568 ## CI report: * 7212b3a0ad469dd36321b385ff7b02ad1995224c Azure:

[GitHub] [hudi] xushiyan closed issue #5550: [SUPPORT] Issues with Spark3_2Adapter while using spark streaming to write to hudi

2022-05-25 Thread GitBox
xushiyan closed issue #5550: [SUPPORT] Issues with Spark3_2Adapter while using spark streaming to write to hudi URL: https://github.com/apache/hudi/issues/5550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-25 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1137413084 ## CI report: * 36161bf2dea866c5aecc745af9109e66b2af0726 Azure:

[GitHub] [hudi] codope commented on a diff in pull request #5449: [HUDI-3911][DOCS][WIP] Add async indexing doc

2022-05-25 Thread GitBox
codope commented on code in PR #5449: URL: https://github.com/apache/hudi/pull/5449#discussion_r881770184 ## website/blog/2022-05-25-async-indexing.md: ## @@ -0,0 +1,142 @@ +--- +title: "Asynchronous Indexing using Hudi" +excerpt: "How to setup Hudi for asynchronous indexing"

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1137384484 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-25 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1137376253 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure:

[jira] [Updated] (HUDI-4155) Support optional Source schema config for S3EventsHoodieIncrSource

2022-05-25 Thread Harshal Patil (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patil updated HUDI-4155: Fix Version/s: 0.12.0 > Support optional Source schema config for S3EventsHoodieIncrSource >

[jira] [Created] (HUDI-4155) Support optional Source schema config for S3EventsHoodieIncrSource

2022-05-25 Thread Harshal Patil (Jira)
Harshal Patil created HUDI-4155: --- Summary: Support optional Source schema config for S3EventsHoodieIncrSource Key: HUDI-4155 URL: https://issues.apache.org/jira/browse/HUDI-4155 Project: Apache Hudi

[jira] [Assigned] (HUDI-4155) Support optional Source schema config for S3EventsHoodieIncrSource

2022-05-25 Thread Harshal Patil (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patil reassigned HUDI-4155: --- Assignee: Harshal Patil > Support optional Source schema config for S3EventsHoodieIncrSource

[GitHub] [hudi] MikeTipico opened a new issue, #5685: [SUPPORT] Loading older data with old schema version into Hudi

2022-05-25 Thread GitBox
MikeTipico opened a new issue, #5685: URL: https://github.com/apache/hudi/issues/5685 **Problem Description** In our systems we have scenarios that require us to reload some data (sent to us backdated or fixed from source) into existing Hudi tables, however we are facing an issue when

[GitHub] [hudi] lanyuanxiaoyao commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-25 Thread GitBox
lanyuanxiaoyao commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1137304189 I don't know how to re-run the IT test, the bot command looks not working. The IT test show me a exception but it is no problem on my local environment and the error case is

[jira] [Closed] (HUDI-3193) Move hudi-aws dependency out of hudi-client-common

2022-05-25 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-3193. - Resolution: Done > Move hudi-aws dependency out of hudi-client-common >

[hudi] branch master updated (a6bc9e8e81 -> cf837b4900)

2022-05-25 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from a6bc9e8e81 [HUDI-4146] Claim RFC-55 for Improve Hive/Meta sync class design and hierachies (#5682) add cf837b4900

[GitHub] [hudi] codope merged pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-25 Thread GitBox
codope merged PR #5666: URL: https://github.com/apache/hudi/pull/5666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

  1   2   3   >