[jira] [Closed] (HUDI-4052) Move AWS Cloudwatch metric configs to hudi-client-common instead of hudi-aws

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4052. - Resolution: Fixed > Move AWS Cloudwatch metric configs to hudi-client-common instead of hudi-aws > ---

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Status: Patch Available (was: In Progress) > Add a Hudi AWS bundle > - > >

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Status: In Progress (was: Open) > Add a Hudi AWS bundle > - > > Key

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Story Points: 3 > Add a Hudi AWS bundle > - > > Key: HUDI-4011 >

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136099029 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136094876 ## CI report: * 1999f03275ed0c9da3db0ceb17b39e9943d78017 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8876

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4011: - Labels: pull-request-available (was: ) > Add a Hudi AWS bundle > - > >

[GitHub] [hudi] codope opened a new pull request, #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
codope opened a new pull request, #5674: URL: https://github.com/apache/hudi/pull/5674 ## What is the purpose of the pull request Stacked on top of #5666 Adds a new bundle where aws-specific dependencies are shaded. It is meant to be used along with other bundles e.g. hudi-uti

[GitHub] [hudi] danny0405 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
danny0405 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136083838 If you can connect to the HDFS on your local env, you can use the HoodieTimelineArchiver to test it locally and you can quickly find the reason -- This is an automated message from

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
nsivabalan commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880650541 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +- @v

[GitHub] [hudi] fengjian428 commented on pull request #5549: [Hudi 4065] Implement a self-releasing FileLock for multiple writers scenario

2022-05-24 Thread GitBox
fengjian428 commented on PR #5549: URL: https://github.com/apache/hudi/pull/5549#issuecomment-1136053079 > can you please fill in the details in the description of the patch. would be easy for reviewers done, what's more, add description in https://issues.apache.org/jira/browse/HUDI-

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136052812 ## CI report: * 1999f03275ed0c9da3db0ceb17b39e9943d78017 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8876

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135966032 ## CI report: * d679d17404397e420fea70dbb5449e299ab66a76 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8897

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135961449 ## CI report: * d679d17404397e420fea70dbb5449e299ab66a76 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8897

[GitHub] [hudi] sunke38 opened a new issue, #5673: [SUPPORT]

2022-05-24 Thread GitBox
sunke38 opened a new issue, #5673: URL: https://github.com/apache/hudi/issues/5673 I am try to rename a colum by spark-sql.but it show scala.MatchError spark.sql("ALTER TABLE hudi_cow_pt_tbl RENAME COLUMN ts TO tts") Environment Description Hudi version : 0.11 Spa

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1135956820 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8901

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1135946579 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135944763 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN * e8

[jira] [Updated] (HUDI-4149) Drop-Table fails when underlying table directory is broken

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4149: - Labels: pull-request-available (was: ) > Drop-Table fails when underlying table directory is brok

[GitHub] [hudi] jinxing64 opened a new pull request, #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
jinxing64 opened a new pull request, #5672: URL: https://github.com/apache/hudi/pull/5672 ## What is the purpose of the pull request If a hudi table directory is lost on DFS due to some maloperation and cannot restore, user cannot drop it from SQL interface any more. We may provide an

[GitHub] [hudi] miomiocat commented on pull request #5111: [HUDI-3695] Add a ORC reader in HoodieBaseRelation

2022-05-24 Thread GitBox
miomiocat commented on PR #5111: URL: https://github.com/apache/hudi/pull/5111#issuecomment-1135899058 > hi @miomiocat fix the CI failed. fixed, PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] codope commented on pull request #5518: [HUDI-4052] Move HoodieMetricsCloudWatchConfig to hudi-client-common

2022-05-24 Thread GitBox
codope commented on PR #5518: URL: https://github.com/apache/hudi/pull/5518#issuecomment-1135893978 Closing it in favor of #5666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] codope closed pull request #5518: [HUDI-4052] Move HoodieMetricsCloudWatchConfig to hudi-client-common

2022-05-24 Thread GitBox
codope closed pull request #5518: [HUDI-4052] Move HoodieMetricsCloudWatchConfig to hudi-client-common URL: https://github.com/apache/hudi/pull/5518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-113531 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[jira] [Updated] (HUDI-4149) Drop-Table fails when underlying table directory is broken

2022-05-24 Thread Jin Xing (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jin Xing updated HUDI-4149: --- Description: If a hudi table directory is lost on DFS due to some maloperation and cannot restore, user cannot

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135876047 ## CI report: * 089a9288320928473bfcf7a713882c2fd135c1dd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8896

[jira] [Created] (HUDI-4149) Drop-Table fails when underlying table directory is broken

2022-05-24 Thread Jin Xing (Jira)
Jin Xing created HUDI-4149: -- Summary: Drop-Table fails when underlying table directory is broken Key: HUDI-4149 URL: https://issues.apache.org/jira/browse/HUDI-4149 Project: Apache Hudi Issue Type:

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135874464 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN * e8

[GitHub] [hudi] codope commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
codope commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880447051 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +- @vinot

[GitHub] [hudi] codope commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
codope commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880446719 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +- @vinot

[GitHub] [hudi] codope commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
codope commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880446386 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +- @vinot

[GitHub] [hudi] codope commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
codope commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880440161 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +- @vinot

[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135853543 Thanks for suggest, Danny! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135852605 > +1, thanks for contribution @yuzhaojing , we may need to fix the clustering plan scheduling issue in the following PR. Sure, I will fix the issue in following PR to support cluste

[GitHub] [hudi] xushiyan commented on a diff in pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
xushiyan commented on code in PR #5666: URL: https://github.com/apache/hudi/pull/5666#discussion_r880435101 ## hudi-aws/src/main/java/org/apache/hudi/aws/sync/AwsGlueCatalogSyncTool.java: ## @@ -65,6 +63,56 @@ public static void main(String[] args) { FileSystem fs = FSUtils

[GitHub] [hudi] codope commented on a diff in pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
codope commented on code in PR #5666: URL: https://github.com/apache/hudi/pull/5666#discussion_r880432062 ## hudi-aws/src/main/java/org/apache/hudi/aws/sync/AwsGlueCatalogSyncTool.java: ## @@ -65,6 +63,56 @@ public static void main(String[] args) { FileSystem fs = FSUtils.g

[GitHub] [hudi] nsivabalan merged pull request #5648: [HUDI-4132] Fixing determining target table schema for delta sync with empty batch

2022-05-24 Thread GitBox
nsivabalan merged PR #5648: URL: https://github.com/apache/hudi/pull/5648 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated: [HUDI-4132] Fixing determining target table schema for delta sync with empty batch (#5648)

2022-05-24 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 10363c1412 [HUDI-4132] Fixing determining targe

[GitHub] [hudi] danny0405 commented on a diff in pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
danny0405 commented on code in PR #3599: URL: https://github.com/apache/hudi/pull/3599#discussion_r880428258 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java: ## @@ -399,6 +404,59 @@ public HoodieWriteMetadata> cluster(final Stri

[jira] [Updated] (HUDI-4148) Preparations for hudi table manager service

2022-05-24 Thread Zhaojing Yu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaojing Yu updated HUDI-4148: -- Summary: Preparations for hudi table manager service (was: Realize the main process of TMS) > Preparat

[jira] [Created] (HUDI-4148) Realize the main process of TMS

2022-05-24 Thread Zhaojing Yu (Jira)
Zhaojing Yu created HUDI-4148: - Summary: Realize the main process of TMS Key: HUDI-4148 URL: https://issues.apache.org/jira/browse/HUDI-4148 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] codope commented on pull request #5641: [HUDI-4023] Decouple hudi-spark from hudi-utilities-slim-bundle

2022-05-24 Thread GitBox
codope commented on PR #5641: URL: https://github.com/apache/hudi/pull/5641#issuecomment-1135827322 CI succeeded https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=8777&view=results -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Created] (HUDI-4147) [Umbrella] Hudi table manager service

2022-05-24 Thread Zhaojing Yu (Jira)
Zhaojing Yu created HUDI-4147: - Summary: [Umbrella] Hudi table manager service Key: HUDI-4147 URL: https://issues.apache.org/jira/browse/HUDI-4147 Project: Apache Hudi Issue Type: Epic

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135820940 ## CI report: * 145542b26e6dae731fb155ee048073c568065941 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8869

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135817505 ## CI report: * 145542b26e6dae731fb155ee048073c568065941 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8869

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135809973 ## CI report: * dea6229e0070b824af4c2c0838b9b1deb3a06d41 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8891

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135806876 ## CI report: * dea6229e0070b824af4c2c0838b9b1deb3a06d41 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8891

[jira] [Updated] (HUDI-4135) Verify if netty dependencies can be removed

2022-05-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4135: - Fix Version/s: 0.11.1 (was: 0.12.0) > Verify if netty dependencies can be removed >

[jira] [Closed] (HUDI-4135) Verify if netty dependencies can be removed

2022-05-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-4135. Resolution: Fixed > Verify if netty dependencies can be removed > --

[GitHub] [hudi] xushiyan commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
xushiyan commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135769113 @fengjian428 @XuQianJin-Stars as we discussed, this feature is to enable partial update during precombine. I suggest put this on-hold and adapt to implementations in https://github.com/apac

[GitHub] [hudi] xushiyan commented on a diff in pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
xushiyan commented on code in PR #5666: URL: https://github.com/apache/hudi/pull/5666#discussion_r880378853 ## hudi-aws/src/main/java/org/apache/hudi/aws/sync/AwsGlueCatalogSyncTool.java: ## @@ -65,6 +63,56 @@ public static void main(String[] args) { FileSystem fs = FSUtils

[GitHub] [hudi] JerryYue-M commented on pull request #5445: [HUDI-3953]Flink Hudi module should support low-level source and sink…

2022-05-24 Thread GitBox
JerryYue-M commented on PR #5445: URL: https://github.com/apache/hudi/pull/5445#issuecomment-1135783133 @danny0405 can you review it again? it had resolved all the discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135773647 ## CI report: * dea6229e0070b824af4c2c0838b9b1deb3a06d41 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8891

[hudi] branch master updated: [HUDI-4135] remove netty and netty-all (#5663)

2022-05-24 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0caa55ecb4 [HUDI-4135] remove netty and netty-al

[GitHub] [hudi] xushiyan merged pull request #5663: [HUDI-4135] Remove netty and netty-all

2022-05-24 Thread GitBox
xushiyan merged PR #5663: URL: https://github.com/apache/hudi/pull/5663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135761423 ## CI report: * dea6229e0070b824af4c2c0838b9b1deb3a06d41 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8891

[GitHub] [hudi] hudi-bot commented on pull request #5111: [HUDI-3695] Add a ORC reader in HoodieBaseRelation

2022-05-24 Thread GitBox
hudi-bot commented on PR #5111: URL: https://github.com/apache/hudi/pull/5111#issuecomment-1135754657 ## CI report: * c6c3f827126de639ec6b05f32a3751abe7d8d457 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8890

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135754286 ## CI report: * e7e16a239220f67ef90a0ecd59dbba27ad695d4d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8892

[GitHub] [hudi] garyli1019 commented on issue #4887: [SUPPORT] Unexpected behaviour with partitioned hudi tables with impala as query engine

2022-05-24 Thread GitBox
garyli1019 commented on issue #4887: URL: https://github.com/apache/hudi/issues/4887#issuecomment-1135751072 @nsivabalan https://issues.apache.org/jira/browse/HUDI-3537 I created a jira for this issue. To solve this we need to bump the Hudi version on impala codebase. -- This is an autom

[GitHub] [hudi] tjtoll commented on issue #5636: [SUPPORT]Need info on the versions of hudi dependent jars which can be used with Glue 3.0

2022-05-24 Thread GitBox
tjtoll commented on issue #5636: URL: https://github.com/apache/hudi/issues/5636#issuecomment-1135737992 Since you went to .11 you might need this specific jar instead: hudi-spark3.1-bundle_2.12-0.11.0.jar Glue is spark 3.1 and I’m guessing the jar you have is 3.2 > On May 24,

[GitHub] [hudi] codope commented on a diff in pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
codope commented on code in PR #5666: URL: https://github.com/apache/hudi/pull/5666#discussion_r880313503 ## hudi-flink-datasource/hudi-flink/pom.xml: ## @@ -89,6 +89,12 @@ hudi-client-common ${project.version} + +org.apach

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135703630 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN * f1

[GitHub] [hudi] codope commented on a diff in pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
codope commented on code in PR #5666: URL: https://github.com/apache/hudi/pull/5666#discussion_r880314331 ## hudi-flink-datasource/hudi-flink/pom.xml: ## @@ -89,6 +89,12 @@ hudi-client-common ${project.version} + +org.apach

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1135694651 Actually,the program runs four days a day,all of compaction were successful and all log files were successfully compacted before the program stops. -- This is an automated message fr

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135686433 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN * f1

[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135682335 > We may need to resolve the conflict. fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] ccchenhe commented on issue #4881: Full incremental Enable index loading to discover duplicate data(index.bootstrap.enabled)

2022-05-24 Thread GitBox
ccchenhe commented on issue #4881: URL: https://github.com/apache/hudi/issues/4881#issuecomment-1135676263 > Thanks, it seems the sink hudi table num is a factor of data duplication. Thank you so much for investigating this root cause, bro -- This is an automated message from the Ap

[jira] [Resolved] (HUDI-4145) Archives the metadata file in HoodieInstant.State sequence

2022-05-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4145. -- > Archives the metadata file in HoodieInstant.State sequence > -

[jira] [Commented] (HUDI-4145) Archives the metadata file in HoodieInstant.State sequence

2022-05-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541383#comment-17541383 ] Danny Chen commented on HUDI-4145: -- Fixed via master branch: eb219010d2491165dbdc00a6b954

[hudi] branch master updated (c05ebf2417 -> eb219010d2)

2022-05-24 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c05ebf2417 [HUDI-2473] Fixing compaction write operation in commit metadata (#5203) add eb219010d2 [HUDI-4145]

[GitHub] [hudi] danny0405 merged pull request #5669: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequence

2022-05-24 Thread GitBox
danny0405 merged PR #5669: URL: https://github.com/apache/hudi/pull/5669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] danny0405 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
danny0405 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1135650145 Did you schedule the compaction but does not compact it actually ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] danny0405 commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
danny0405 commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135647735 Would take some time to review, please do not merge before i give an approve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hudi] danny0405 commented on issue #4881: Full incremental Enable index loading to discover duplicate data(index.bootstrap.enabled)

2022-05-24 Thread GitBox
danny0405 commented on issue #4881: URL: https://github.com/apache/hudi/issues/4881#issuecomment-1135644671 Thanks, it seems the sink hudi table num is a factor of data duplication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [hudi] ccchenhe commented on issue #4881: Full incremental Enable index loading to discover duplicate data(index.bootstrap.enabled)

2022-05-24 Thread GitBox
ccchenhe commented on issue #4881: URL: https://github.com/apache/hudi/issues/4881#issuecomment-1135631326 1. I try to change index type, write.payload.class, not works. 2. I try to remove spark job, only submit flink job consume all of history data, not works. 3. when my application t

[GitHub] [hudi] ccchenhe commented on issue #4881: Full incremental Enable index loading to discover duplicate data(index.bootstrap.enabled)

2022-05-24 Thread GitBox
ccchenhe commented on issue #4881: URL: https://github.com/apache/hudi/issues/4881#issuecomment-1135621886 > @ccchenhe said that COW may lost the index data, let's see if we can have more details :) ### summary my flink application consume same kafka and sink multiple hudi tabl

[GitHub] [hudi] liujinhui1994 commented on pull request #5663: [HUDI-4135] Remove netty and netty-all

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5663: URL: https://github.com/apache/hudi/pull/5663#issuecomment-1135594213 ![image](https://user-images.githubusercontent.com/25769285/169991829-66f80dc4-b6de-4a65-8c65-693e5a81a191.png) ![image](https://user-images.githubusercontent.com/25769285/169991943

[GitHub] [hudi] fengjian428 commented on a diff in pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
fengjian428 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r880242817 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -58,6 +58,20 @@ default T preCombine(T oldValue, Properties properties) {

[GitHub] [hudi] liujinhui1994 commented on pull request #5663: [HUDI-4135] Remove netty and netty-all

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5663: URL: https://github.com/apache/hudi/pull/5663#issuecomment-1135592051 @xushiyan I am in spark2.4.5 environment. Use hudi-spark-bundle_2.11-0.12.0-SNAPSHOT.jar. Consume kafka and write hudi successfully and stably -- This is an automated message from

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135584702 ## CI report: * 06ad01690f6b7ad97ffcae055b5d29543832b74a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=

[GitHub] [hudi] fengjian428 commented on a diff in pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
fengjian428 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r880210547 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -58,6 +58,20 @@ default T preCombine(T oldValue, Properties properties) {

[GitHub] [hudi] fengjian428 commented on a diff in pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
fengjian428 commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r880210547 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -58,6 +58,20 @@ default T preCombine(T oldValue, Properties properties) {

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135546748 ## CI report: * 06ad01690f6b7ad97ffcae055b5d29543832b74a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=

[GitHub] [hudi] xushiyan commented on a diff in pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
xushiyan commented on code in PR #4676: URL: https://github.com/apache/hudi/pull/4676#discussion_r880190065 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java: ## @@ -58,6 +58,20 @@ default T preCombine(T oldValue, Properties properties) { re

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135536920 ## CI report: * bfd19beadbae54ca4981baf5c9b7ceeda7755214 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8857

[GitHub] [hudi] eric9204 opened a new issue, #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 opened a new issue, #5671: URL: https://github.com/apache/hudi/issues/5671 **Describe the problem you faced** Flink sql consume data from kafka and write to hudi, the sql statement enable metadata table, Archive can't be triggered.so ,there are 7044 files in the `.hoodie` di

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1135533727 ## CI report: * bfd19beadbae54ca4981baf5c9b7ceeda7755214 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8857

[GitHub] [hudi] hudi-bot commented on pull request #5111: [HUDI-3695] Add a ORC reader in HoodieBaseRelation

2022-05-24 Thread GitBox
hudi-bot commented on PR #5111: URL: https://github.com/apache/hudi/pull/5111#issuecomment-1135529878 ## CI report: * 344e7f1327ee91a271aaa01e05f4afc35c533041 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8792

[GitHub] [hudi] hudi-bot commented on pull request #5111: [HUDI-3695] Add a ORC reader in HoodieBaseRelation

2022-05-24 Thread GitBox
hudi-bot commented on PR #5111: URL: https://github.com/apache/hudi/pull/5111#issuecomment-1135526685 ## CI report: * 344e7f1327ee91a271aaa01e05f4afc35c533041 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8792

[GitHub] [hudi] hudi-bot commented on pull request #4676: [HUDI-3304] support partial update on mor table

2022-05-24 Thread GitBox
hudi-bot commented on PR #4676: URL: https://github.com/apache/hudi/pull/4676#issuecomment-1135522739 ## CI report: * 06ad01690f6b7ad97ffcae055b5d29543832b74a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=

[GitHub] [hudi] kapjoshi-cisco commented on issue #5636: [SUPPORT]Need info on the versions of hudi dependent jars which can be used with Glue 3.0

2022-05-24 Thread GitBox
kapjoshi-cisco commented on issue #5636: URL: https://github.com/apache/hudi/issues/5636#issuecomment-1135518894 ~> Thank you this works~ Sorry there as an error handling section, which made it look like the job passed, its still failing with the below error Jars used: 1. s3

[jira] [Closed] (HUDI-4084) Add support to test any yaml w/ deltastreamer continuous mode and async table services

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4084. - Resolution: Done > Add support to test any yaml w/ deltastreamer continuous mode and async table > servic

[jira] [Closed] (HUDI-2473) Fix compaction action type in commit metadata

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-2473. - Resolution: Fixed > Fix compaction action type in commit metadata > --

[hudi] branch master updated: [HUDI-2473] Fixing compaction write operation in commit metadata (#5203)

2022-05-24 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c05ebf2417 [HUDI-2473] Fixing compaction write ope

[GitHub] [hudi] codope merged pull request #5203: [HUDI-2473] Fixing compaction write operation in commit metadata

2022-05-24 Thread GitBox
codope merged PR #5203: URL: https://github.com/apache/hudi/pull/5203 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[jira] [Updated] (HUDI-2473) Fix compaction action type in commit metadata

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-2473: -- Fix Version/s: 0.11.1 (was: 0.11.0) > Fix compaction action type in commit metada

[GitHub] [hudi] danny0405 commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
danny0405 commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1135506129 We may need to resolve the conflict. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[jira] [Updated] (HUDI-3764) Allow loading external configs while querying Hudi tables with Spark

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3764: -- Status: Patch Available (was: In Progress) > Allow loading external configs while querying Hudi tables

[jira] [Updated] (HUDI-3764) Allow loading external configs while querying Hudi tables with Spark

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3764: -- Status: In Progress (was: Open) > Allow loading external configs while querying Hudi tables with Spark

[GitHub] [hudi] danny0405 commented on issue #4881: Full incremental Enable index loading to discover duplicate data(index.bootstrap.enabled)

2022-05-24 Thread GitBox
danny0405 commented on issue #4881: URL: https://github.com/apache/hudi/issues/4881#issuecomment-1135504884 @ccchenhe said that COW may lost the index data, let's see if we can have more details :) -- This is an automated message from the Apache Git Service. To respond to the message, ple

<    1   2   3   >