[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1122003783 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121995359 ## CI report: * e68d1192395b62f26864d8009ed01cf4f0c18806 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8539

[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
hudi-bot commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121993995 ## CI report: * cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537

[jira] [Created] (HUDI-4079) Supports showing table comment for hudi with spark3

2022-05-09 Thread Jin Xing (Jira)
Jin Xing created HUDI-4079: -- Summary: Supports showing table comment for hudi with spark3 Key: HUDI-4079 URL: https://issues.apache.org/jira/browse/HUDI-4079 Project: Apache Hudi Issue Type: Improve

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121993031 ## CI report: * Unknown: [CANCELED](TBD) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build

[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
hudi-bot commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121996361 ## CI report: * cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537

[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
cuibo01 commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121990938 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
cuibo01 commented on code in PR #5545: URL: https://github.com/apache/hudi/pull/5545#discussion_r868868607 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -644,14 +644,17 @@ public final Stream getLatestUnCompactedFileSlice

[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121988394 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] yuzhaojing commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
yuzhaojing commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121988113 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
cuibo01 commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121987331 @danny0405 hi now i don't have a hudi 0.10.1 and flink cluster -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [hudi] JoshuaZhuCN commented on issue #5515: [SUPPORT]FileNotFoundException for hadoop-common-3.1.3.jar

2022-05-09 Thread GitBox
JoshuaZhuCN commented on issue #5515: URL: https://github.com/apache/hudi/issues/5515#issuecomment-1121985462 > I have the same problem。 > > spark:3.1.3 hudi:0.10.1 hadoop:3.1.0 I solved it. After I deleted some removed machines in the cluster from the hosts, there was no error

[GitHub] [hudi] cuibo01 commented on a diff in pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
cuibo01 commented on code in PR #5528: URL: https://github.com/apache/hudi/pull/5528#discussion_r868857643 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -746,11 +760,11 @@ public static Configuration flatOptions(Configur

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121957364 ## CI report: * e68d1192395b62f26864d8009ed01cf4f0c18806 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8539

[GitHub] [hudi] YuweiXiao commented on issue #5541: [SUPPORT] Upgrade from 0.8.0 to 0.9.0 decreases Upsert performance

2022-05-09 Thread GitBox
YuweiXiao commented on issue #5541: URL: https://github.com/apache/hudi/issues/5541#issuecomment-1121939354 Maybe related to this PR https://github.com/apache/hudi/pull/4012 Could you try adding this patch and test again? The issue is introduced in 0.9.0 and fixed in 0.11.0. -- Thi

[GitHub] [hudi] codope commented on issue #5021: [FEATURE REQUEST] Support for Presto/Athena to make updates to Hudi Tables

2022-05-09 Thread GitBox
codope commented on issue #5021: URL: https://github.com/apache/hudi/issues/5021#issuecomment-1121925490 @kazdy @maddy2u Glad to see the interest in write support for hudi tables via Presto. Currently, there is no write support either through the hive connector or the [in-progress native c

[jira] [Assigned] (HUDI-1575) Early detection by periodically checking last written commit & active markers

2022-05-09 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-1575: Assignee: Yue Zhang (was: Ethan Guo) > Early detection by periodically checking last written commi

[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.

2022-05-09 Thread GitBox
jinxing64 commented on PR #5534: URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121902158 @xushiyan @XuQianJin-Stars Please take a look if it benefits hudi sql. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
hudi-bot commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121884853 ## CI report: * cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537

[GitHub] [hudi] danny0405 commented on a diff in pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
danny0405 commented on code in PR #5528: URL: https://github.com/apache/hudi/pull/5528#discussion_r868791833 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -746,11 +760,11 @@ public static Configuration flatOptions(Config

[GitHub] [hudi] danny0405 commented on a diff in pull request #5445: [HUDI-3953]Flink Hudi module should support low-level source and sink…

2022-05-09 Thread GitBox
danny0405 commented on code in PR #5445: URL: https://github.com/apache/hudi/pull/5445#discussion_r868790677 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/ITTestDataStreamWrite.java: ## @@ -248,8 +221,129 @@ private void testWriteToHoodie( // wait

[GitHub] [hudi] hudi-bot commented on pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
hudi-bot commented on PR #5528: URL: https://github.com/apache/hudi/pull/5528#issuecomment-1121862240 ## CI report: * ac044d9c0b307b493597211c756dc8592d0a7ab6 UNKNOWN * 40a2d34ea001115562bbe5b87253e4d268ffb715 UNKNOWN * 7b32c9bd29153aa34a6aff1f7541c824f9cc4df8 Azure: [SUCCES

[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
cuibo01 commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121861555 10001 records: Before the repair ![image](https://user-images.githubusercontent.com/36881649/167538428-e5d404b4-46af-47d1-89a6-88efefbd2f7a.png) After the repair ![image](htt

[jira] [Assigned] (HUDI-3734) Add savepoint restore nodes and yamls to integ test suite

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3734: - Assignee: Raymond Xu > Add savepoint restore nodes and yamls to integ test suite

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121859694 ## CI report: * 59407592cd51b23da1cfc8f31852734e3f57a7db Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7362

[jira] [Assigned] (HUDI-4029) test out different lock providers using our integ test infra

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4029: - Assignee: sivabalan narayanan > test out different lock providers using our integ

[jira] [Assigned] (HUDI-3952) Presto HadoopExtendedFileSystem now supports getScheme. Update presto version to 0.273 is docker setup and IT test

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3952: - Assignee: Sagar Sumit > Presto HadoopExtendedFileSystem now supports getScheme. U

[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.

2022-05-09 Thread GitBox
hudi-bot commented on PR #5534: URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121856086 ## CI report: * 697a8c68d64bd9144cb81c4c918e85d15b81a15b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536

[jira] [Assigned] (HUDI-4025) Add support to validate presto, trino and hive queries in integ test framework

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4025: - Assignee: Sagar Sumit > Add support to validate presto, trino and hive queries in

[GitHub] [hudi] danny0405 commented on issue #5020: [SUPPORT] The cleaning strategy breaks the reader view completeness

2022-05-09 Thread GitBox
danny0405 commented on issue #5020: URL: https://github.com/apache/hudi/issues/5020#issuecomment-1121853297 > @danny0405 : going back to your original example in the description. If a file slice was written in C1, and never updated in any of the future commits, at C10 or C11, even if cleane

[jira] [Assigned] (HUDI-987) Need integration tests for MOR table of decimal type

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-987: Assignee: sivabalan narayanan > Need integration tests for MOR table of decimal type

[jira] [Updated] (HUDI-3668) Fix failing unit tests in hudi-integ-test

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3668: -- Sprint: Hudi-Sprint-Apr-19, Hudi-Sprint-Apr-25 (was: Hudi-Sprint-Apr-19, Hudi-Sprint-Ap

[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
danny0405 commented on code in PR #5545: URL: https://github.com/apache/hudi/pull/5545#discussion_r868778594 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -644,14 +644,17 @@ public final Stream getLatestUnCompactedFileSli

[jira] [Assigned] (HUDI-1590) Support async clustering w/ test suite job

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-1590: - Assignee: Sagar Sumit (was: sivabalan narayanan) > Support async clustering w/ t

[jira] [Assigned] (HUDI-3991) Provide bundle jar options in each e2e test pipeline

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3991: - Assignee: Sagar Sumit > Provide bundle jar options in each e2e test pipeline > --

[jira] [Assigned] (HUDI-4016) Prepare a document to list all tests to be done as part of release certification

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4016: - Assignee: Raymond Xu (was: sivabalan narayanan) > Prepare a document to list all

[jira] [Updated] (HUDI-2673) Add integration/e2e test for kafka-connect functionality

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2673: -- Reviewers: Sagar Sumit > Add integration/e2e test for kafka-connect functionality >

[jira] [Updated] (HUDI-4027) add support to test non-core write operations (insert overwrite, delete partitions) to integ test framework

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4027: -- Reviewers: Sagar Sumit > add support to test non-core write operations (insert overwrite

[jira] [Updated] (HUDI-4020) Add support to multi-writer tests to integ test framework (4 concurrent writers)

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4020: -- Reviewers: Sagar Sumit > Add support to multi-writer tests to integ test framework (4 co

[jira] [Updated] (HUDI-4018) Prepare minimal set of yamls to be tested against any write mode and against any query engine

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4018: -- Reviewers: Sagar Sumit (was: Raymond Xu) > Prepare minimal set of yamls to be tested ag

[GitHub] [hudi] hudi-bot commented on pull request #3599: [HUDI-2207] Support independent flink hudi clustering function

2022-05-09 Thread GitBox
hudi-bot commented on PR #3599: URL: https://github.com/apache/hudi/pull/3599#issuecomment-1121838855 ## CI report: * 59407592cd51b23da1cfc8f31852734e3f57a7db Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7362

[GitHub] [hudi] vinothchandar commented on pull request #5544: [DOCS] Pull up Blog into top navigation items

2022-05-09 Thread GitBox
vinothchandar commented on PR #5544: URL: https://github.com/apache/hudi/pull/5544#issuecomment-1121830498 Move it after community? Instead of after docs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
hudi-bot commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121827852 ## CI report: * cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537

[jira] [Assigned] (HUDI-4078) BootstrapOperator cannot load all index data

2022-05-09 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HUDI-4078: Assignee: Bo Cui > BootstrapOperator cannot load all index data > -

[GitHub] [hudi] danny0405 commented on a diff in pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

2022-05-09 Thread GitBox
danny0405 commented on code in PR #4739: URL: https://github.com/apache/hudi/pull/4739#discussion_r868755460 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -1301,4 +1359,33 @@ public void close() { this.heartbeatClie

[jira] [Updated] (HUDI-4044) When reading data from flink-hudi to external storage, the result is incorrect

2022-05-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4044: - Fix Version/s: 0.11.1 0.12.0 > When reading data from flink-hudi to external storage, t

[jira] [Commented] (HUDI-4044) When reading data from flink-hudi to external storage, the result is incorrect

2022-05-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534085#comment-17534085 ] Danny Chen commented on HUDI-4044: -- Fixed via master branch: 6fd21d0f1043d0a06b93332d86e6

[jira] [Resolved] (HUDI-4044) When reading data from flink-hudi to external storage, the result is incorrect

2022-05-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4044. -- > When reading data from flink-hudi to external storage, the result is incorrect > -

[hudi] branch master updated: [HUDI-4044] When reading data from flink-hudi to external storage, the … (#5516)

2022-05-09 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 6fd21d0f10 [HUDI-4044] When reading data from f

[GitHub] [hudi] danny0405 merged pull request #5516: HUDI-4044 When reading data from flink-hudi to external storage, the …

2022-05-09 Thread GitBox
danny0405 merged PR #5516: URL: https://github.com/apache/hudi/pull/5516 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
hudi-bot commented on PR #5545: URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121792331 ## CI report: * cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.

2022-05-09 Thread GitBox
hudi-bot commented on PR #5534: URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121792309 ## CI report: * 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512

[GitHub] [hudi] hudi-bot commented on pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
hudi-bot commented on PR #5528: URL: https://github.com/apache/hudi/pull/5528#issuecomment-1121792286 ## CI report: * ac044d9c0b307b493597211c756dc8592d0a7ab6 UNKNOWN * 5dd23dc4d3d8fe082a24b0e18685d9158c0623c3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-4078) BootstrapOperator cannot load all index data

2022-05-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4078: - Labels: pull-request-available (was: ) > BootstrapOperator cannot load all index data > -

[GitHub] [hudi] cuibo01 opened a new pull request, #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…

2022-05-09 Thread GitBox
cuibo01 opened a new pull request, #5545: URL: https://github.com/apache/hudi/pull/5545 …ion files ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## W

[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.

2022-05-09 Thread GitBox
hudi-bot commented on PR #5534: URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121787903 ## CI report: * 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512

[GitHub] [hudi] hudi-bot commented on pull request #5402: [WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default

2022-05-09 Thread GitBox
hudi-bot commented on PR #5402: URL: https://github.com/apache/hudi/pull/5402#issuecomment-1121782698 ## CI report: * 8c6f6e19940ce7ac04dfcfce52da3ccdaf3a8b0f UNKNOWN * c4799803cff8adffef56e889a5cd4d52599fcf73 UNKNOWN * e948d879f2bc68b252a7642f3a14333a83e293d7 Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
hudi-bot commented on PR #5528: URL: https://github.com/apache/hudi/pull/5528#issuecomment-1121778532 ## CI report: * ac044d9c0b307b493597211c756dc8592d0a7ab6 UNKNOWN * 5dd23dc4d3d8fe082a24b0e18685d9158c0623c3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #5402: [WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default

2022-05-09 Thread GitBox
hudi-bot commented on PR #5402: URL: https://github.com/apache/hudi/pull/5402#issuecomment-1121778436 ## CI report: * 8c6f6e19940ce7ac04dfcfce52da3ccdaf3a8b0f UNKNOWN * c4799803cff8adffef56e889a5cd4d52599fcf73 UNKNOWN * bc5c5aa75b8f36ad0cd1144eac028a10de52bf00 Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
hudi-bot commented on PR #5528: URL: https://github.com/apache/hudi/pull/5528#issuecomment-1121777279 ## CI report: * ac044d9c0b307b493597211c756dc8592d0a7ab6 UNKNOWN * 5dd23dc4d3d8fe082a24b0e18685d9158c0623c3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-4078) BootstrapOperator cannot load all index data

2022-05-09 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-4078: - Affects Version/s: 0.9.0 0.11.0 > BootstrapOperator cannot load all index data > --

[jira] [Created] (HUDI-4078) BootstrapOperator cannot load all index data

2022-05-09 Thread Bo Cui (Jira)
Bo Cui created HUDI-4078: Summary: BootstrapOperator cannot load all index data Key: HUDI-4078 URL: https://issues.apache.org/jira/browse/HUDI-4078 Project: Apache Hudi Issue Type: Bug Re

[jira] [Created] (HUDI-4077) Revisit log record scanner for concurrent reading from metadata table

2022-05-09 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4077: --- Summary: Revisit log record scanner for concurrent reading from metadata table Key: HUDI-4077 URL: https://issues.apache.org/jira/browse/HUDI-4077 Project: Apache Hudi

[jira] [Updated] (HUDI-4076) Optimize metadata payload conversion before writing to metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4076: Sprint: 2022/05/02 > Optimize metadata payload conversion before writing to metadata table > ---

[jira] [Assigned] (HUDI-4076) Optimize metadata payload conversion before writing to metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-4076: --- Assignee: Ethan Guo > Optimize metadata payload conversion before writing to metadata table > ---

[jira] [Updated] (HUDI-4076) Optimize metadata payload conversion before writing to metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4076: Fix Version/s: 0.12.0 > Optimize metadata payload conversion before writing to metadata table >

[jira] [Updated] (HUDI-4076) Optimize metadata payload conversion before writing to metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4076: Priority: Blocker (was: Major) > Optimize metadata payload conversion before writing to metadata table > --

[jira] [Created] (HUDI-4076) Optimize metadata payload conversion before writing to metadata table

2022-05-09 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4076: --- Summary: Optimize metadata payload conversion before writing to metadata table Key: HUDI-4076 URL: https://issues.apache.org/jira/browse/HUDI-4076 Project: Apache Hudi

[GitHub] [hudi] cuibo01 commented on a diff in pull request #5528: [HUDI-3336][HUDI-FLINK]Support custom hadoop config for flink

2022-05-09 Thread GitBox
cuibo01 commented on code in PR #5528: URL: https://github.com/apache/hudi/pull/5528#discussion_r868699640 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -746,16 +762,18 @@ public static Configuration flatOptions(Configur

[jira] [Updated] (HUDI-4075) Investigate spillable map usage in metadata table log scanning

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4075: Sprint: 2022/05/02 > Investigate spillable map usage in metadata table log scanning > --

[jira] [Created] (HUDI-4075) Investigate spillable map usage in metadata table log scanning

2022-05-09 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4075: --- Summary: Investigate spillable map usage in metadata table log scanning Key: HUDI-4075 URL: https://issues.apache.org/jira/browse/HUDI-4075 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-4074) Evaluate performance of different (de)compression algo for HFile

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4074: Fix Version/s: 0.12.0 > Evaluate performance of different (de)compression algo for HFile > -

[jira] [Updated] (HUDI-4074) Evaluate performance of different (de)compression algo for HFile

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4074: Priority: Blocker (was: Major) > Evaluate performance of different (de)compression algo for HFile > ---

[jira] [Created] (HUDI-4074) Evaluate performance of different (de)compression algo for HFile

2022-05-09 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4074: --- Summary: Evaluate performance of different (de)compression algo for HFile Key: HUDI-4074 URL: https://issues.apache.org/jira/browse/HUDI-4074 Project: Apache Hudi Iss

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121740985 ## CI report: * 68ae936690f617d70ebf821076cc3a4b28b52a00 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8533

[jira] [Updated] (HUDI-4073) Optimize FileStatus construction in file listing from metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4073: Description: In BaseTableMetadata::getAllFilesInPartitions, we construct  FileStatus instances for the later

[jira] [Updated] (HUDI-4073) Optimize FileStatus construction in file listing from metadata table

2022-05-09 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4073: Sprint: 2022/05/02 > Optimize FileStatus construction in file listing from metadata table >

[jira] [Created] (HUDI-4073) Optimize FileStatus construction in file listing from metadata table

2022-05-09 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-4073: --- Summary: Optimize FileStatus construction in file listing from metadata table Key: HUDI-4073 URL: https://issues.apache.org/jira/browse/HUDI-4073 Project: Apache Hudi

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121699558 ## CI report: * 454553a2dcbdbcd4152386572c89f5382c09e5bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8531

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121697767 ## CI report: * 454553a2dcbdbcd4152386572c89f5382c09e5bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8531

[GitHub] [hudi] hudi-bot commented on pull request #5041: [HUDI-3580] [UBER] Create RFC for LogCompaction action

2022-05-09 Thread GitBox
hudi-bot commented on PR #5041: URL: https://github.com/apache/hudi/pull/5041#issuecomment-1121691926 ## CI report: * 111d09feaaf94c2400c76f2dd199f2ea89f56e69 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8530

[GitHub] [hudi] hudi-bot commented on pull request #5402: [WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default

2022-05-09 Thread GitBox
hudi-bot commented on PR #5402: URL: https://github.com/apache/hudi/pull/5402#issuecomment-1121670768 ## CI report: * 8c6f6e19940ce7ac04dfcfce52da3ccdaf3a8b0f UNKNOWN * c4799803cff8adffef56e889a5cd4d52599fcf73 UNKNOWN * bc5c5aa75b8f36ad0cd1144eac028a10de52bf00 Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121664155 ## CI report: * 454553a2dcbdbcd4152386572c89f5382c09e5bd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8531

[GitHub] [hudi] hudi-bot commented on pull request #5402: [WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default

2022-05-09 Thread GitBox
hudi-bot commented on PR #5402: URL: https://github.com/apache/hudi/pull/5402#issuecomment-1121662188 ## CI report: * 8c6f6e19940ce7ac04dfcfce52da3ccdaf3a8b0f UNKNOWN * c4799803cff8adffef56e889a5cd4d52599fcf73 UNKNOWN * 92a7e39a15d685c1f5d8747d440b8767043b5dcb Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5402: [WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default

2022-05-09 Thread GitBox
hudi-bot commented on PR #5402: URL: https://github.com/apache/hudi/pull/5402#issuecomment-1121658365 ## CI report: * 8c6f6e19940ce7ac04dfcfce52da3ccdaf3a8b0f UNKNOWN * c4799803cff8adffef56e889a5cd4d52599fcf73 UNKNOWN * 92a7e39a15d685c1f5d8747d440b8767043b5dcb Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121656405 ## CI report: * 454553a2dcbdbcd4152386572c89f5382c09e5bd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8531

[GitHub] [hudi] bhasudha commented on pull request #5544: [DOCS] Pull up Blog into top navigation items

2022-05-09 Thread GitBox
bhasudha commented on PR #5544: URL: https://github.com/apache/hudi/pull/5544#issuecomment-1121652254 ![Screen Shot 2022-05-09 at 3 40 48 PM](https://user-images.githubusercontent.com/2179254/167510265-3cc1436c-295f-471d-8afe-0cf6100a621e.png) -- This is an automated message from the

[GitHub] [hudi] bhasudha opened a new pull request, #5544: [DOCS] Pull up Blog into top navigation items

2022-05-09 Thread GitBox
bhasudha opened a new pull request, #5544: URL: https://github.com/apache/hudi/pull/5544 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpos

[GitHub] [hudi] hudi-bot commented on pull request #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
hudi-bot commented on PR #5543: URL: https://github.com/apache/hudi/pull/5543#issuecomment-1121632645 ## CI report: * 454553a2dcbdbcd4152386572c89f5382c09e5bd UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4072: -- Priority: Critical (was: Blocker) > Clustering fails when there is an empty SCHEMA entr

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4072: - Labels: pull-request-available (was: ) > Clustering fails when there is an empty SCHEMA entry in

[GitHub] [hudi] nsivabalan opened a new pull request, #5543: [HUDI-4072] Fixing NULL schema for empty batches in deltastreamer

2022-05-09 Thread GitBox
nsivabalan opened a new pull request, #5543: URL: https://github.com/apache/hudi/pull/5543 ## What is the purpose of the pull request - Fixing schema to set with writeConfig in deltastreamer when empty batch is consumed. ## Brief change log - Fixing schema to set with w

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4072: -- Priority: Blocker (was: Major) > Clustering fails when there is an empty SCHEMA entry i

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4072: -- Fix Version/s: 0.12.0 > Clustering fails when there is an empty SCHEMA entry in commit m

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4072: -- Status: In Progress (was: Open) > Clustering fails when there is an empty SCHEMA entry

[jira] [Assigned] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-4072: - Assignee: sivabalan narayanan > Clustering fails when there is an empty SCHEMA en

[jira] [Created] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-4072: - Summary: Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer Key: HUDI-4072 URL: https://issues.apache.org/jira/browse/HUDI-4072

[jira] [Updated] (HUDI-4072) Clustering fails when there is an empty SCHEMA entry in commit metadata with deltastreamer

2022-05-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4072: -- Sprint: 2022/05/02 > Clustering fails when there is an empty SCHEMA entry in commit meta

[GitHub] [hudi] hudi-bot commented on pull request #5041: [HUDI-3580] [UBER] Create RFC for LogCompaction action

2022-05-09 Thread GitBox
hudi-bot commented on PR #5041: URL: https://github.com/apache/hudi/pull/5041#issuecomment-1121621768 ## CI report: * 0e8c88f4d90e08bf947fcef81c26fb2377411e7c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=734

[GitHub] [hudi] gunjdesai commented on issue #5542: [SUPPORT] Error Querying HUDI through Trino when syncing HMS via Spark

2022-05-09 Thread GitBox
gunjdesai commented on issue #5542: URL: https://github.com/apache/hudi/issues/5542#issuecomment-1121618067 I am sorry about this one. I got confused between `hudi-hadoop-mr-0.11.0.jar` and `hudi-hadoop-mr-bundle-0.11.0.jar`. Closing this one -- This is an automated message from the Apac

  1   2   3   4   >