[jira] [Resolved] (HUDI-4134) Fix Method naming consistency issues in FSUtils

2022-05-24 Thread Heap (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heap resolved HUDI-4134. > Fix Method naming consistency issues in FSUtils > --- > >

[GitHub] [hudi] hudi-bot commented on pull request #5678: [HUDI-4153]StreamReadOperator supports async opeing of foramt.

2022-05-24 Thread GitBox
hudi-bot commented on PR #5678: URL: https://github.com/apache/hudi/pull/5678#issuecomment-1136829854 ## CI report: * cb96c0707ff1fea2760b2a94cd58c4a1bbb1fc63 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8915

[GitHub] [hudi] hudi-bot commented on pull request #5678: [HUDI-4153]StreamReadOperator supports async opeing of foramt.

2022-05-24 Thread GitBox
hudi-bot commented on PR #5678: URL: https://github.com/apache/hudi/pull/5678#issuecomment-1136823733 ## CI report: * cb96c0707ff1fea2760b2a94cd58c4a1bbb1fc63 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136823595 ## CI report: * ace699cf034214ea3ec7b1485cb884a9fa46ef72 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8911

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136823089 ## CI report: * b235c452d47ad7de3884ac0928c9b468b2ec5d75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8907

[GitHub] [hudi] cuibo01 commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
cuibo01 commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136818900 ![image](https://user-images.githubusercontent.com/36881649/170197749-4af09cec-0763-48df-8c96-f2a3bc7ade78.png) BitCaskDiskMap require a lot of memory. -- This is an automated message f

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136817746 ## CI report: * ace699cf034214ea3ec7b1485cb884a9fa46ef72 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8911

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136817193 ## CI report: * b235c452d47ad7de3884ac0928c9b468b2ec5d75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8907

[GitHub] [hudi] hudi-bot commented on pull request #4958: [HUDI-3558] Consistent bucket index: bucket resizing (split&merge) & concurrent write during resizing

2022-05-24 Thread GitBox
hudi-bot commented on PR #4958: URL: https://github.com/apache/hudi/pull/4958#issuecomment-1136816063 ## CI report: * fca0c137ca9612904db9444125f093e61b516de9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8913

[jira] [Updated] (HUDI-4153) StreamReadOperator supports asynchronous opening of format

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4153: - Labels: pull-request-available (was: ) > StreamReadOperator supports asynchronous opening of form

[GitHub] [hudi] cuibo01 opened a new pull request, #5678: [HUDI-4153]StreamReadOperator supports async opeing of foramt.

2022-05-24 Thread GitBox
cuibo01 opened a new pull request, #5678: URL: https://github.com/apache/hudi/pull/5678 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[GitHub] [hudi] cuibo01 commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
cuibo01 commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136775144 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] boneanxs commented on a diff in pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
boneanxs commented on code in PR #5502: URL: https://github.com/apache/hudi/pull/5502#discussion_r881254024 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/CustomColumnsSortPartitionerWithRows.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1136768321 ## CI report: * c617d9b6f67c9e567392da9ee9718faa84b1ec02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8912

[GitHub] [hudi] hudi-bot commented on pull request #4958: [HUDI-3558] Consistent bucket index: bucket resizing (split&merge) & concurrent write during resizing

2022-05-24 Thread GitBox
hudi-bot commented on PR #4958: URL: https://github.com/apache/hudi/pull/4958#issuecomment-1136767572 ## CI report: * 33aace381fc3633b52e8321adeab29f47f7a24f8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8670

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136756847 ## CI report: * ace699cf034214ea3ec7b1485cb884a9fa46ef72 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8911

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136754485 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8910

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136754239 ## CI report: * abc9efbeb629e712842e208977a1666a265b4c2a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8909

[jira] [Updated] (HUDI-4063) Update the site doc for flink since release 0.11

2022-05-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4063: - Component/s: docs (was: doc) > Update the site doc for flink since release 0.11 > ---

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136742468 I can't connect to the HDFS ,so I could't test it use HoodieTimelineArchiver. this should be related to compaction.I'm thinking about it,#4812. -- This is an automated message from

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136737349 Maybe You are right , i restart the program only change the table type from mor to cow yesterday,the meta file under `.hoodie` directory was archived .The program has been running for

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1136724250 ## CI report: * c617d9b6f67c9e567392da9ee9718faa84b1ec02 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8912

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1136722761 ## CI report: * c617d9b6f67c9e567392da9ee9718faa84b1ec02 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
danny0405 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136716947 Another detail is that we tweaked the archive strategy when metadata table is enabled for 0.11, https://github.com/apache/hudi/pull/4821, maybe it is relevant with your error. -- T

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-24 Thread GitBox
vinothchandar commented on code in PR #5436: URL: https://github.com/apache/hudi/pull/5436#discussion_r881010366 ## rfc/rfc-51/rfc-51.md: ## @@ -0,0 +1,234 @@ + + +# RFC-50: Hudi CDC + +# Proposers + +- @Yann Byron + +# Approvers + +- @Raymond + +# Statue +JIRA: [https://issues

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lanyuanxiaoyao updated HUDI-4152: - Status: Patch Available (was: In Progress) > Flink offline compaction allow compact multi compact

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4152: - Labels: pull-request-available (was: ) > Flink offline compaction allow compact multi compaction

[GitHub] [hudi] lanyuanxiaoyao opened a new pull request, #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
lanyuanxiaoyao opened a new pull request, #5677: URL: https://github.com/apache/hudi/pull/5677 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136702220 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8901

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136700540 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8901

[GitHub] [hudi] yangzhiyue commented on issue #5658: [SUPPORT] Hudi update will produce duplicated records

2022-05-24 Thread GitBox
yangzhiyue commented on issue #5658: URL: https://github.com/apache/hudi/issues/5658#issuecomment-1136698494 > yes, I have used the org.apache.hudi.keygen.SimpleAvroKeyGenerator as the hoodie.datasource.write.keygenerator.class, thanks for replying -- This is an automated messa

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136696308 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8910

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136694805 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136694689 ## CI report: * 0e41e6e83512a3a8bc57a0f7b70b0c4fc0fbe2f9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8438

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136693485 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8908

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136693354 ## CI report: * b235c452d47ad7de3884ac0928c9b468b2ec5d75 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8907

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136693309 ## CI report: * 0e41e6e83512a3a8bc57a0f7b70b0c4fc0fbe2f9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8438

[GitHub] [hudi] danny0405 opened a new pull request, #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
danny0405 opened a new pull request, #5676: URL: https://github.com/apache/hudi/pull/5676 …e (part2) ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ##

[jira] [Assigned] (HUDI-4153) StreamReadOperator supports asynchronous opening of format

2022-05-24 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HUDI-4153: Assignee: Bo Cui > StreamReadOperator supports asynchronous opening of format > ---

[jira] [Created] (HUDI-4153) StreamReadOperator supports asynchronous opening of format

2022-05-24 Thread Bo Cui (Jira)
Bo Cui created HUDI-4153: Summary: StreamReadOperator supports asynchronous opening of format Key: HUDI-4153 URL: https://issues.apache.org/jira/browse/HUDI-4153 Project: Apache Hudi Issue Type: New

[GitHub] [hudi] boneanxs commented on a diff in pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
boneanxs commented on code in PR #5502: URL: https://github.com/apache/hudi/pull/5502#discussion_r881177283 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/CustomColumnsSortPartitionerWithRows.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the

[GitHub] [hudi] boneanxs commented on a diff in pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
boneanxs commented on code in PR #5502: URL: https://github.com/apache/hudi/pull/5502#discussion_r881177283 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/CustomColumnsSortPartitionerWithRows.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136672546 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136670895 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[GitHub] [hudi] liujinhui1994 commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136670572 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136667885 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8908

[GitHub] [hudi] cuibo01 commented on a diff in pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
cuibo01 commented on code in PR #5554: URL: https://github.com/apache/hudi/pull/5554#discussion_r881169914 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -315,14 +315,14 @@ public static DataStream rowDataToHoodieRecord(Config

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136665826 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136665763 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[jira] [Commented] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541796#comment-17541796 ] lanyuanxiaoyao commented on HUDI-4152: -- I will push a pr to solve this later > Flink

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136663889 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8898

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lanyuanxiaoyao updated HUDI-4152: - Status: In Progress (was: Open) > Flink offline compaction allow compact multi compaction plan at

[jira] [Created] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
lanyuanxiaoyao created HUDI-4152: Summary: Flink offline compaction allow compact multi compaction plan at once Key: HUDI-4152 URL: https://issues.apache.org/jira/browse/HUDI-4152 Project: Apache Hudi

[GitHub] [hudi] wzx140 commented on a diff in pull request #5522: [HUDI-3378][HUDI-3379][HUDI-3381] Rebasing usages of HoodieRecordPayload and raw Avro payload to rely on HoodieRecord instead

2022-05-24 Thread GitBox
wzx140 commented on code in PR #5522: URL: https://github.com/apache/hudi/pull/5522#discussion_r881164109 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java: ## @@ -34,13 +41,51 @@ */ public abstract class HoodieRecord implements Serializable { - p

[jira] [Updated] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4151: - Labels: pull-request-available (was: ) > flink split_reader supports rocksdb > --

[GitHub] [hudi] cuibo01 opened a new pull request, #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
cuibo01 opened a new pull request, #5675: URL: https://github.com/apache/hudi/pull/5675 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[jira] [Assigned] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HUDI-4151: Assignee: Bo Cui > flink split_reader supports rocksdb > --- > >

[jira] [Created] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread Bo Cui (Jira)
Bo Cui created HUDI-4151: Summary: flink split_reader supports rocksdb Key: HUDI-4151 URL: https://issues.apache.org/jira/browse/HUDI-4151 Project: Apache Hudi Issue Type: New Feature Compo

[GitHub] [hudi] liujinhui1994 commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136638397 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [hudi] liujinhui1994 closed pull request #5635: [HUDI-3555] Re-use spark config for parquet timestamp format

2022-05-24 Thread GitBox
liujinhui1994 closed pull request #5635: [HUDI-3555] Re-use spark config for parquet timestamp format URL: https://github.com/apache/hudi/pull/5635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [hudi] danny0405 commented on a diff in pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
danny0405 commented on code in PR #5554: URL: https://github.com/apache/hudi/pull/5554#discussion_r881137895 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -315,14 +315,14 @@ public static DataStream rowDataToHoodieRecord(Conf

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136624766 ## CI report: * 7cee38130e461e5d072702902fd171aac4860e2a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8628

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136621810 ## CI report: * 7cee38130e461e5d072702902fd171aac4860e2a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8628

[GitHub] [hudi] jinxing64 commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
jinxing64 commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136608977 WIP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[jira] [Updated] (HUDI-4039) Make sure builtin key-generators can efficiently fetch record-key, partition-path

2022-05-24 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4039: -- Status: In Progress (was: Open) > Make sure builtin key-generators can efficiently fetch record

[GitHub] [hudi] liuzhuang2017 commented on pull request #5670: [MINOR][DOCS] Update `spark.yarn.driver.memoryOverhead` and `spark.yarn.executor.memoryOverhead` in the tuning-guide.

2022-05-24 Thread GitBox
liuzhuang2017 commented on PR #5670: URL: https://github.com/apache/hudi/pull/5670#issuecomment-1136569835 @yihua,Thanks your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1136568012 @xushiyan thank you for re-opening the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[jira] [Assigned] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-4143: --- Assignee: Ethan Guo > Add build for component archive of Hudi Kafka sink connector >

[jira] [Updated] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4143: Sprint: 2022/05/16 > Add build for component archive of Hudi Kafka sink connector >

[jira] [Updated] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4143: Status: In Progress (was: Open) > Add build for component archive of Hudi Kafka sink connector > --

[jira] [Closed] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-3873. --- Resolution: Done https://www.onehouse.ai/blog/introducing-multi-modal-index-for-the-lakehouse-in-apache-hudi

[jira] [Updated] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3873: Sprint: Hudi-Sprint-Apr-12, Hudi-Sprint-Apr-19, Hudi-Sprint-Apr-25, 2022/05/02, 2022/05/16 (was: Hudi-Sprin

[jira] [Updated] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3873: Summary: Multi-modal index blog for 0.11 release (was: 0.11 release blog) > Multi-modal index blog for 0.11

[GitHub] [hudi] yihua closed pull request #5325: [MINOR][NOMERGE] turn drop partition col on

2022-05-24 Thread GitBox
yihua closed pull request #5325: [MINOR][NOMERGE] turn drop partition col on URL: https://github.com/apache/hudi/pull/5325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [hudi] yihua commented on pull request #5325: [MINOR][NOMERGE] turn drop partition col on

2022-05-24 Thread GitBox
yihua commented on PR #5325: URL: https://github.com/apache/hudi/pull/5325#issuecomment-1136561102 Closing this as it is meant for testing only before 0.11.0 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[hudi] branch asf-site updated: [MINOR][DOCS] Update spark.yarn.driver.memoryOverhead and spark.yarn.executor.memoryOverhead in the tuning-guide. (#5670)

2022-05-24 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 9f2ea8563f [MINOR][DOCS] Update spark.yarn.driv

[GitHub] [hudi] yihua merged pull request #5670: [MINOR][DOCS] Update `spark.yarn.driver.memoryOverhead` and `spark.yarn.executor.memoryOverhead` in the tuning-guide.

2022-05-24 Thread GitBox
yihua merged PR #5670: URL: https://github.com/apache/hudi/pull/5670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] jasondavindev opened a new issue, #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
jasondavindev opened a new issue, #5484: URL: https://github.com/apache/hudi/issues/5484 **Describe the problem you faced** After some issues reported [here](https://github.com/apache/hudi/issues/5469), I upgraded my workload version from Hudi 0.10.0 to 0.11.0. In my applications

[GitHub] [hudi] Gatsby-Lee commented on issue #4839: Hudi upsert doesnt trigger compaction for MOR

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #4839: URL: https://github.com/apache/hudi/issues/4839#issuecomment-1136495929 For anyone who gets here because Async Compaction doesn't work - AWS Glue + Hudi + MoR Table, If you use Spark Write directly, Async compaction won't work. ``` glue_cont

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1136490508 BTW, this issue still exists. I tested with Glue2 and Glue3 ( with the right bundle ), but it still doesn't work. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5522: [HUDI-3378][HUDI-3379][HUDI-3381] Rebasing usages of HoodieRecordPayload and raw Avro payload to rely on HoodieRecord instead

2022-05-24 Thread GitBox
vinothchandar commented on code in PR #5522: URL: https://github.com/apache/hudi/pull/5522#discussion_r880994838 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java: ## @@ -34,13 +41,51 @@ */ public abstract class HoodieRecord implements Serializable {

[jira] [Commented] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541683#comment-17541683 ] sivabalan narayanan commented on HUDI-4150: --- Add new cleaner policy(based on hou

[jira] [Updated] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4150: -- Description: Add comments for missing documentation on our website. It could be faq or c

[jira] [Commented] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541681#comment-17541681 ] sivabalan narayanan commented on HUDI-4150: --- Insert_overwrite_table and delete_p

[jira] [Assigned] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4150: - Assignee: Bhavani Sudha Saktheeswaran > Doc updates for 0.11.1 >

[jira] [Updated] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4150: -- Component/s: docs (was: dos) > Doc updates for 0.11.1 > ---

[jira] [Created] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4150: - Summary: Doc updates for 0.11.1 Key: HUDI-4150 URL: https://issues.apache.org/jira/browse/HUDI-4150 Project: Apache Hudi Issue Type: Improvement

[hudi] branch master updated: [MINOR] Fix a potential NPE and some finer points of hudi cli (#5656)

2022-05-24 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f30b3aef3e [MINOR] Fix a potential NPE and some fin

[GitHub] [hudi] yihua merged pull request #5656: [MINOR] Fix a potential NPE and some finer points of hudi cli

2022-05-24 Thread GitBox
yihua merged PR #5656: URL: https://github.com/apache/hudi/pull/5656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] yihua commented on issue #5422: [SUPPORT] Enabling metadata on MOR table causes FileNotFound exception

2022-05-24 Thread GitBox
yihua commented on issue #5422: URL: https://github.com/apache/hudi/issues/5422#issuecomment-1136269395 @rahil-c ^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136268679 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8904

[GitHub] [hudi] yihua commented on issue #5280: [SUPPORT] Docker Demo: Failed to Connect to namenode

2022-05-24 Thread GitBox
yihua commented on issue #5280: URL: https://github.com/apache/hudi/issues/5280#issuecomment-1136184428 @xushiyan We've built the images for arm64. I remember some native libraries from Hadoop or Hive are not compatible on arm64 arch so at runtime the errors are thrown. It depends on thos

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136182475 ## CI report: * c10517f194c5eeeafa9e7bf60873290c56f114d8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8903

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1136170317 ## CI report: * f9a7a610284b8cefe52fcec55e52f0ea10a01b3e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8902

[hudi] branch master updated: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread yuzhaojing
This is an automated email from the ASF dual-hosted git repository. yuzhaojing pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c20db99a7b [HUDI-2207] Support independent fli

[GitHub] [hudi] yuzhaojing merged pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
yuzhaojing merged PR #3599: URL: https://github.com/apache/hudi/pull/3599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1136162136 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN * c2

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136118694 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8901

[jira] [Updated] (HUDI-4023) de-couple spark from utilities bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4023: -- Story Points: 2 > de-couple spark from utilities bundle > - > >

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136102890 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8904

  1   2   3   >