[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136756847 ## CI report: * ace699cf034214ea3ec7b1485cb884a9fa46ef72 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136754485 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136754239 ## CI report: * abc9efbeb629e712842e208977a1666a265b4c2a Azure:

[jira] [Updated] (HUDI-4063) Update the site doc for flink since release 0.11

2022-05-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4063: - Component/s: docs (was: doc) > Update the site doc for flink since release 0.11 >

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136742468 I can't connect to the HDFS ,so I could't test it use HoodieTimelineArchiver. this should be related to compaction.I'm thinking about it,#4812. -- This is an automated message from

[GitHub] [hudi] eric9204 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
eric9204 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136737349 Maybe You are right , i restart the program only change the table type from mor to cow yesterday,the meta file under `.hoodie` directory was archived .The program has been running for

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1136724250 ## CI report: * c617d9b6f67c9e567392da9ee9718faa84b1ec02 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
hudi-bot commented on PR #5677: URL: https://github.com/apache/hudi/pull/5677#issuecomment-1136722761 ## CI report: * c617d9b6f67c9e567392da9ee9718faa84b1ec02 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] danny0405 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
danny0405 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136716947 Another detail is that we tweaked the archive strategy when metadata table is enabled for 0.11, https://github.com/apache/hudi/pull/4821, maybe it is relevant with your error. --

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5436: [RFC-51] [HUDI-3478] Change Data Capture RFC

2022-05-24 Thread GitBox
vinothchandar commented on code in PR #5436: URL: https://github.com/apache/hudi/pull/5436#discussion_r881010366 ## rfc/rfc-51/rfc-51.md: ## @@ -0,0 +1,234 @@ + + +# RFC-50: Hudi CDC + +# Proposers + +- @Yann Byron + +# Approvers + +- @Raymond + +# Statue +JIRA:

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lanyuanxiaoyao updated HUDI-4152: - Status: Patch Available (was: In Progress) > Flink offline compaction allow compact multi

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4152: - Labels: pull-request-available (was: ) > Flink offline compaction allow compact multi compaction

[GitHub] [hudi] lanyuanxiaoyao opened a new pull request, #5677: [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread GitBox
lanyuanxiaoyao opened a new pull request, #5677: URL: https://github.com/apache/hudi/pull/5677 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136702220 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136700540 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure:

[GitHub] [hudi] yangzhiyue commented on issue #5658: [SUPPORT] Hudi update will produce duplicated records

2022-05-24 Thread GitBox
yangzhiyue commented on issue #5658: URL: https://github.com/apache/hudi/issues/5658#issuecomment-1136698494 > yes, I have used the org.apache.hudi.keygen.SimpleAvroKeyGenerator as the hoodie.datasource.write.keygenerator.class, thanks for replying -- This is an automated

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136696308 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5676: URL: https://github.com/apache/hudi/pull/5676#issuecomment-1136694805 ## CI report: * dd574b5ec3f5e724d6581c84fed8f75dd40bbff4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136694689 ## CI report: * 0e41e6e83512a3a8bc57a0f7b70b0c4fc0fbe2f9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136693485 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136693354 ## CI report: * b235c452d47ad7de3884ac0928c9b468b2ec5d75 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
hudi-bot commented on PR #5502: URL: https://github.com/apache/hudi/pull/5502#issuecomment-1136693309 ## CI report: * 0e41e6e83512a3a8bc57a0f7b70b0c4fc0fbe2f9 Azure:

[GitHub] [hudi] danny0405 opened a new pull request, #5676: [HUDI-4145] Archives the metadata file in HoodieInstant.State sequenc…

2022-05-24 Thread GitBox
danny0405 opened a new pull request, #5676: URL: https://github.com/apache/hudi/pull/5676 …e (part2) ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*

[jira] [Assigned] (HUDI-4153) StreamReadOperator supports asynchronous opening of format

2022-05-24 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HUDI-4153: Assignee: Bo Cui > StreamReadOperator supports asynchronous opening of format >

[jira] [Created] (HUDI-4153) StreamReadOperator supports asynchronous opening of format

2022-05-24 Thread Bo Cui (Jira)
Bo Cui created HUDI-4153: Summary: StreamReadOperator supports asynchronous opening of format Key: HUDI-4153 URL: https://issues.apache.org/jira/browse/HUDI-4153 Project: Apache Hudi Issue Type: New

[GitHub] [hudi] boneanxs commented on a diff in pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
boneanxs commented on code in PR #5502: URL: https://github.com/apache/hudi/pull/5502#discussion_r881177283 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/CustomColumnsSortPartitionerWithRows.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the

[GitHub] [hudi] boneanxs commented on a diff in pull request #5502: [HUDI-4040]Bulk insert: Add customColumnsSortpartitionerWithRows

2022-05-24 Thread GitBox
boneanxs commented on code in PR #5502: URL: https://github.com/apache/hudi/pull/5502#discussion_r881177283 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/CustomColumnsSortPartitionerWithRows.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136672546 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136670895 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure:

[GitHub] [hudi] liujinhui1994 commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136670572 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136667885 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f Azure:

[GitHub] [hudi] cuibo01 commented on a diff in pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
cuibo01 commented on code in PR #5554: URL: https://github.com/apache/hudi/pull/5554#discussion_r881169914 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -315,14 +315,14 @@ public static DataStream

[GitHub] [hudi] hudi-bot commented on pull request #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
hudi-bot commented on PR #5675: URL: https://github.com/apache/hudi/pull/5675#issuecomment-1136665826 ## CI report: * e8a278ecd4bbd33e2cc5fb3582a41c3edd90377f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136665763 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure:

[jira] [Commented] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541796#comment-17541796 ] lanyuanxiaoyao commented on HUDI-4152: -- I will push a pr to solve this later > Flink offline

[GitHub] [hudi] hudi-bot commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
hudi-bot commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136663889 ## CI report: * a885fa78a4c1420ddcf46ad2c634e4f7b6c42b12 Azure:

[jira] [Updated] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lanyuanxiaoyao updated HUDI-4152: - Status: In Progress (was: Open) > Flink offline compaction allow compact multi compaction plan

[jira] [Created] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

2022-05-24 Thread lanyuanxiaoyao (Jira)
lanyuanxiaoyao created HUDI-4152: Summary: Flink offline compaction allow compact multi compaction plan at once Key: HUDI-4152 URL: https://issues.apache.org/jira/browse/HUDI-4152 Project: Apache

[GitHub] [hudi] wzx140 commented on a diff in pull request #5522: [HUDI-3378][HUDI-3379][HUDI-3381] Rebasing usages of HoodieRecordPayload and raw Avro payload to rely on HoodieRecord instead

2022-05-24 Thread GitBox
wzx140 commented on code in PR #5522: URL: https://github.com/apache/hudi/pull/5522#discussion_r881164109 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java: ## @@ -34,13 +41,51 @@ */ public abstract class HoodieRecord implements Serializable { -

[jira] [Updated] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4151: - Labels: pull-request-available (was: ) > flink split_reader supports rocksdb >

[GitHub] [hudi] cuibo01 opened a new pull request, #5675: [HUDI-4151] flink split_reader supports rocksdb

2022-05-24 Thread GitBox
cuibo01 opened a new pull request, #5675: URL: https://github.com/apache/hudi/pull/5675 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[jira] [Assigned] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HUDI-4151: Assignee: Bo Cui > flink split_reader supports rocksdb > --- > >

[jira] [Created] (HUDI-4151) flink split_reader supports rocksdb

2022-05-24 Thread Bo Cui (Jira)
Bo Cui created HUDI-4151: Summary: flink split_reader supports rocksdb Key: HUDI-4151 URL: https://issues.apache.org/jira/browse/HUDI-4151 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] liujinhui1994 commented on pull request #5639: [HUDI-2516] Upgrade JUnit to 5.8.2

2022-05-24 Thread GitBox
liujinhui1994 commented on PR #5639: URL: https://github.com/apache/hudi/pull/5639#issuecomment-1136638397 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] liujinhui1994 closed pull request #5635: [HUDI-3555] Re-use spark config for parquet timestamp format

2022-05-24 Thread GitBox
liujinhui1994 closed pull request #5635: [HUDI-3555] Re-use spark config for parquet timestamp format URL: https://github.com/apache/hudi/pull/5635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] danny0405 commented on a diff in pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
danny0405 commented on code in PR #5554: URL: https://github.com/apache/hudi/pull/5554#discussion_r881137895 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java: ## @@ -315,14 +315,14 @@ public static DataStream

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136624766 ## CI report: * 7cee38130e461e5d072702902fd171aac4860e2a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5554: [HUDI-4082][HUDI-FLINK]enhance the readability of hudi operator names…

2022-05-24 Thread GitBox
hudi-bot commented on PR #5554: URL: https://github.com/apache/hudi/pull/5554#issuecomment-1136621810 ## CI report: * 7cee38130e461e5d072702902fd171aac4860e2a Azure:

[GitHub] [hudi] jinxing64 commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
jinxing64 commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136608977 WIP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Updated] (HUDI-4039) Make sure builtin key-generators can efficiently fetch record-key, partition-path

2022-05-24 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4039: -- Status: In Progress (was: Open) > Make sure builtin key-generators can efficiently fetch

[GitHub] [hudi] liuzhuang2017 commented on pull request #5670: [MINOR][DOCS] Update `spark.yarn.driver.memoryOverhead` and `spark.yarn.executor.memoryOverhead` in the tuning-guide.

2022-05-24 Thread GitBox
liuzhuang2017 commented on PR #5670: URL: https://github.com/apache/hudi/pull/5670#issuecomment-1136569835 @yihua,Thanks your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1136568012 @xushiyan thank you for re-opening the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Assigned] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-4143: --- Assignee: Ethan Guo > Add build for component archive of Hudi Kafka sink connector >

[jira] [Updated] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4143: Sprint: 2022/05/16 > Add build for component archive of Hudi Kafka sink connector >

[jira] [Updated] (HUDI-4143) Add build for component archive of Hudi Kafka sink connector

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4143: Status: In Progress (was: Open) > Add build for component archive of Hudi Kafka sink connector >

[jira] [Closed] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-3873. --- Resolution: Done https://www.onehouse.ai/blog/introducing-multi-modal-index-for-the-lakehouse-in-apache-hudi

[jira] [Updated] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3873: Sprint: Hudi-Sprint-Apr-12, Hudi-Sprint-Apr-19, Hudi-Sprint-Apr-25, 2022/05/02, 2022/05/16 (was:

[jira] [Updated] (HUDI-3873) Multi-modal index blog for 0.11 release

2022-05-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3873: Summary: Multi-modal index blog for 0.11 release (was: 0.11 release blog) > Multi-modal index blog for

[GitHub] [hudi] yihua closed pull request #5325: [MINOR][NOMERGE] turn drop partition col on

2022-05-24 Thread GitBox
yihua closed pull request #5325: [MINOR][NOMERGE] turn drop partition col on URL: https://github.com/apache/hudi/pull/5325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] yihua commented on pull request #5325: [MINOR][NOMERGE] turn drop partition col on

2022-05-24 Thread GitBox
yihua commented on PR #5325: URL: https://github.com/apache/hudi/pull/5325#issuecomment-1136561102 Closing this as it is meant for testing only before 0.11.0 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[hudi] branch asf-site updated: [MINOR][DOCS] Update spark.yarn.driver.memoryOverhead and spark.yarn.executor.memoryOverhead in the tuning-guide. (#5670)

2022-05-24 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 9f2ea8563f [MINOR][DOCS] Update

[GitHub] [hudi] yihua merged pull request #5670: [MINOR][DOCS] Update `spark.yarn.driver.memoryOverhead` and `spark.yarn.executor.memoryOverhead` in the tuning-guide.

2022-05-24 Thread GitBox
yihua merged PR #5670: URL: https://github.com/apache/hudi/pull/5670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] jasondavindev opened a new issue, #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
jasondavindev opened a new issue, #5484: URL: https://github.com/apache/hudi/issues/5484 **Describe the problem you faced** After some issues reported [here](https://github.com/apache/hudi/issues/5469), I upgraded my workload version from Hudi 0.10.0 to 0.11.0. In my applications

[GitHub] [hudi] Gatsby-Lee commented on issue #4839: Hudi upsert doesnt trigger compaction for MOR

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #4839: URL: https://github.com/apache/hudi/issues/4839#issuecomment-1136495929 For anyone who gets here because Async Compaction doesn't work - AWS Glue + Hudi + MoR Table, If you use Spark Write directly, Async compaction won't work. ```

[GitHub] [hudi] Gatsby-Lee commented on issue #5484: [SUPPORT] Hive Sync + AWS Data Catalog failling with Hudi 0.11.0

2022-05-24 Thread GitBox
Gatsby-Lee commented on issue #5484: URL: https://github.com/apache/hudi/issues/5484#issuecomment-1136490508 BTW, this issue still exists. I tested with Glue2 and Glue3 ( with the right bundle ), but it still doesn't work. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] vinothchandar commented on a diff in pull request #5522: [HUDI-3378][HUDI-3379][HUDI-3381] Rebasing usages of HoodieRecordPayload and raw Avro payload to rely on HoodieRecord instead

2022-05-24 Thread GitBox
vinothchandar commented on code in PR #5522: URL: https://github.com/apache/hudi/pull/5522#discussion_r880994838 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecord.java: ## @@ -34,13 +41,51 @@ */ public abstract class HoodieRecord implements Serializable

[jira] [Commented] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541683#comment-17541683 ] sivabalan narayanan commented on HUDI-4150: --- Add new cleaner policy(based on hours) to

[jira] [Updated] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4150: -- Description: Add comments for missing documentation on our website. It could be faq or

[jira] [Commented] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541681#comment-17541681 ] sivabalan narayanan commented on HUDI-4150: --- Insert_overwrite_table and delete_partition in

[jira] [Assigned] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4150: - Assignee: Bhavani Sudha Saktheeswaran > Doc updates for 0.11.1 >

[jira] [Updated] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4150: -- Component/s: docs (was: dos) > Doc updates for 0.11.1 >

[jira] [Created] (HUDI-4150) Doc updates for 0.11.1

2022-05-24 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4150: - Summary: Doc updates for 0.11.1 Key: HUDI-4150 URL: https://issues.apache.org/jira/browse/HUDI-4150 Project: Apache Hudi Issue Type: Improvement

[hudi] branch master updated: [MINOR] Fix a potential NPE and some finer points of hudi cli (#5656)

2022-05-24 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f30b3aef3e [MINOR] Fix a potential NPE and some

[GitHub] [hudi] yihua merged pull request #5656: [MINOR] Fix a potential NPE and some finer points of hudi cli

2022-05-24 Thread GitBox
yihua merged PR #5656: URL: https://github.com/apache/hudi/pull/5656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua commented on issue #5422: [SUPPORT] Enabling metadata on MOR table causes FileNotFound exception

2022-05-24 Thread GitBox
yihua commented on issue #5422: URL: https://github.com/apache/hudi/issues/5422#issuecomment-1136269395 @rahil-c ^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136268679 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure:

[GitHub] [hudi] yihua commented on issue #5280: [SUPPORT] Docker Demo: Failed to Connect to namenode

2022-05-24 Thread GitBox
yihua commented on issue #5280: URL: https://github.com/apache/hudi/issues/5280#issuecomment-1136184428 @xushiyan We've built the images for arm64. I remember some native libraries from Hadoop or Hive are not compatible on arm64 arch so at runtime the errors are thrown. It depends on

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136182475 ## CI report: * c10517f194c5eeeafa9e7bf60873290c56f114d8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1136170317 ## CI report: * f9a7a610284b8cefe52fcec55e52f0ea10a01b3e Azure:

[hudi] branch master updated: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread yuzhaojing
This is an automated email from the ASF dual-hosted git repository. yuzhaojing pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c20db99a7b [HUDI-2207] Support independent

[GitHub] [hudi] yuzhaojing merged pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
yuzhaojing merged PR #3599: URL: https://github.com/apache/hudi/pull/3599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-24 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1136162136 ## CI report: * c3405300e9bc97445637c7251536ec0f0d6fbbd1 UNKNOWN * e1689c454142ed2daa6d548e272cdbc5b353731d UNKNOWN * eb878d43a82656bdaeb2946fc9c950fd591ce95f UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #5672: [HUDI-4149] Drop-Table fails when underlying table directory is broken

2022-05-24 Thread GitBox
hudi-bot commented on PR #5672: URL: https://github.com/apache/hudi/pull/5672#issuecomment-1136118694 ## CI report: * 9232ab6a7ab32abf110c6b3835a22eee171b173d Azure:

[jira] [Updated] (HUDI-4023) de-couple spark from utilities bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4023: -- Story Points: 2 > de-couple spark from utilities bundle > - > >

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136102890 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 Azure:

[jira] [Closed] (HUDI-4052) Move AWS Cloudwatch metric configs to hudi-client-common instead of hudi-aws

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-4052. - Resolution: Fixed > Move AWS Cloudwatch metric configs to hudi-client-common instead of hudi-aws >

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Status: Patch Available (was: In Progress) > Add a Hudi AWS bundle > - > >

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Status: In Progress (was: Open) > Add a Hudi AWS bundle > - > >

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4011: -- Story Points: 3 > Add a Hudi AWS bundle > - > > Key: HUDI-4011 >

[GitHub] [hudi] hudi-bot commented on pull request #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
hudi-bot commented on PR #5674: URL: https://github.com/apache/hudi/pull/5674#issuecomment-1136099029 ## CI report: * e3f4cf8d8cf92156c0da8b9ea0bc22bac91af6d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136094876 ## CI report: * 1999f03275ed0c9da3db0ceb17b39e9943d78017 Azure:

[jira] [Updated] (HUDI-4011) Add a Hudi AWS bundle

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4011: - Labels: pull-request-available (was: ) > Add a Hudi AWS bundle > - > >

[GitHub] [hudi] codope opened a new pull request, #5674: [HUDI-4011] Add hudi-aws-bundle

2022-05-24 Thread GitBox
codope opened a new pull request, #5674: URL: https://github.com/apache/hudi/pull/5674 ## What is the purpose of the pull request Stacked on top of #5666 Adds a new bundle where aws-specific dependencies are shaded. It is meant to be used along with other bundles e.g.

[GitHub] [hudi] danny0405 commented on issue #5671: [SUPPORT] Archive can't be triggered,when parameter of the metadata table was use in the program

2022-05-24 Thread GitBox
danny0405 commented on issue #5671: URL: https://github.com/apache/hudi/issues/5671#issuecomment-1136083838 If you can connect to the HDFS on your local env, you can use the HoodieTimelineArchiver to test it locally and you can quickly find the reason -- This is an automated message from

[GitHub] [hudi] nsivabalan commented on a diff in pull request #5667: [HUDI-4142] RFC for new table APIs and config changes

2022-05-24 Thread GitBox
nsivabalan commented on code in PR #5667: URL: https://github.com/apache/hudi/pull/5667#discussion_r880650541 ## rfc/rfc-54/rfc-54.md: ## @@ -0,0 +1,175 @@ + + +# RFC-54: New Table APIs and Streamline Hudi Configs + +## Proposers + +- @codope + +## Approvers + +- @xushiyan +-

[GitHub] [hudi] fengjian428 commented on pull request #5549: [Hudi 4065] Implement a self-releasing FileLock for multiple writers scenario

2022-05-24 Thread GitBox
fengjian428 commented on PR #5549: URL: https://github.com/apache/hudi/pull/5549#issuecomment-1136053079 > can you please fill in the details in the description of the patch. would be easy for reviewers done, what's more, add description in

[GitHub] [hudi] hudi-bot commented on pull request #5668: [DO_NOT_MERGE] [WIP] Fixing partitioned dataset for hudi

2022-05-24 Thread GitBox
hudi-bot commented on PR #5668: URL: https://github.com/apache/hudi/pull/5668#issuecomment-1136052812 ## CI report: * 1999f03275ed0c9da3db0ceb17b39e9943d78017 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135966032 ## CI report: * d679d17404397e420fea70dbb5449e299ab66a76 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5666: [HUDI-3193] Decouple hudi-aws from hudi-client-common

2022-05-24 Thread GitBox
hudi-bot commented on PR #5666: URL: https://github.com/apache/hudi/pull/5666#issuecomment-1135961449 ## CI report: * d679d17404397e420fea70dbb5449e299ab66a76 Azure:

[GitHub] [hudi] sunke38 opened a new issue, #5673: [SUPPORT]

2022-05-24 Thread GitBox
sunke38 opened a new issue, #5673: URL: https://github.com/apache/hudi/issues/5673 I am try to rename a colum by spark-sql.but it show scala.MatchError spark.sql("ALTER TABLE hudi_cow_pt_tbl RENAME COLUMN ts TO tts") Environment Description Hudi version : 0.11

  1   2   >