[GitHub] [hudi] hudi-bot commented on pull request #5168: [HUDI-3729][SPARK] fixed the per regression by enable vectorizeReader for parquet file

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5168: URL: https://github.com/apache/hudi/pull/5168#issuecomment-1082527212 ## CI report: * 7b62ff1b2204b0845fcc9b7bb271781a25dbc030 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5168: [HUDI-3729][SPARK] fixed the per regression by enable vectorizeReader for parquet file

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5168: URL: https://github.com/apache/hudi/pull/5168#issuecomment-1081822947 ## CI report: * 7b62ff1b2204b0845fcc9b7bb271781a25dbc030 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[hudi] branch master updated (5c1b482 -> 4fed8dd)

2022-03-29 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 5c1b482 [HUDI-3741] Fix flink bucket index bulk insert generates too many small files (#5164) add 4fed8dd [H

[GitHub] [hudi] nsivabalan merged pull request #5043: [HUDI-3485] Adding scheduler pool configs for async clustering

2022-03-29 Thread GitBox
nsivabalan merged pull request #5043: URL: https://github.com/apache/hudi/pull/5043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan commented on a change in pull request #5043: [HUDI-3485] Adding scheduler pool configs for async clustering

2022-03-29 Thread GitBox
nsivabalan commented on a change in pull request #5043: URL: https://github.com/apache/hudi/pull/5043#discussion_r838034087 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -388,6 +388,14 @@ private boolean onDel

[jira] [Commented] (HUDI-3688) Double check MT init behavior for MT rollout

2022-03-29 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514389#comment-17514389 ] Yue Zhang commented on HUDI-3688: - Step1: do several normal ingestion using 0.10.0. Step2:

[GitHub] [hudi] hudi-bot commented on pull request #5173: [HUDI-3721]Delete MDT if necessary when trigger rollback to savepoint through cli

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5173: URL: https://github.com/apache/hudi/pull/5173#issuecomment-1082521015 ## CI report: * 5d56ed5abd6767e23a551e385956ee892352cf48 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5173: [HUDI-3721]Delete MDT if necessary when trigger rollback to savepoint through cli

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5173: URL: https://github.com/apache/hudi/pull/5173#issuecomment-1082519220 ## CI report: * 5d56ed5abd6767e23a551e385956ee892352cf48 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #5168: [HUDI-3729][SPARK] fixed the per regression by enable vectorizeReader for parquet file

2022-03-29 Thread GitBox
xiarixiaoyao commented on a change in pull request #5168: URL: https://github.com/apache/hudi/pull/5168#discussion_r838030298 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala ## @@ -290,11 +290,8 @@ abstract class Ho

[GitHub] [hudi] xiarixiaoyao commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

2022-03-29 Thread GitBox
xiarixiaoyao commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1082519316 @bvaradar Thank you very much for your patient review for so many days let me rebase the code -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] hudi-bot commented on pull request #5173: [HUDI-3721]Delete MDT if necessary when trigger rollback to savepoint through cli

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5173: URL: https://github.com/apache/hudi/pull/5173#issuecomment-1082519220 ## CI report: * 5d56ed5abd6767e23a551e385956ee892352cf48 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] zhangyue19921010 opened a new pull request #5173: [HUDI-3721]Delete MDT if necessary when trigger rollback to savepoint through cli

2022-03-29 Thread GitBox
zhangyue19921010 opened a new pull request #5173: URL: https://github.com/apache/hudi/pull/5173 please refer https://issues.apache.org/jira/browse/HUDI-3721 for details. ## What is the purpose of the pull request *(For example: This pull request adds quick-start document.)*

[GitHub] [hudi] hudi-bot commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082513487 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba56871fe Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082512057 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba5687

[GitHub] [hudi] hudi-bot commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082512057 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba56871fe Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082510619 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba5687

[GitHub] [hudi] hudi-bot commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082510619 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba56871fe Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1080214434 ## CI report: * 7ee24be4d11864af37bf300250d571e15d5f9ae9 UNKNOWN * 9a9e544ba48a52c7b54134fc9533c3e5a51ccfff UNKNOWN * f69bbf06e9cb669dfe0785b5eee8501ba5687

[GitHub] [hudi] peanut-chenzhong commented on pull request #5042: Three bulk_insert files are concurrently submitted and executed with a difference of 2s, the insert fails occasionally.

2022-03-29 Thread GitBox
peanut-chenzhong commented on pull request #5042: URL: https://github.com/apache/hudi/pull/5042#issuecomment-1082509968 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[hudi] branch master updated (941c254 -> 5c1b482)

2022-03-29 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 941c254 [HUDI-2520] Fix CTAS statment issue when sync to hive (#5145) add 5c1b482 [HUDI-3741] Fix flink bucke

[GitHub] [hudi] danny0405 merged pull request #5164: [HUDI-3741] Fix flink bucket index bulk insert generates too many sma…

2022-03-29 Thread GitBox
danny0405 merged pull request #5164: URL: https://github.com/apache/hudi/pull/5164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] hudi-bot commented on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot commented on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082481851 ## CI report: * b1b66d9e789e16ab90b6cd20d31befd643e45edc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082443989 ## CI report: * b1b66d9e789e16ab90b6cd20d31befd643e45edc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5171: URL: https://github.com/apache/hudi/pull/5171#issuecomment-1082465643 ## CI report: * 80d7377bf9846f5f1da01684b246c8a2be5013c0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5171: URL: https://github.com/apache/hudi/pull/5171#issuecomment-1082421108 ## CI report: * 80d7377bf9846f5f1da01684b246c8a2be5013c0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] bhasudha opened a new pull request #5172: Update roadmap page

2022-03-29 Thread GitBox
bhasudha opened a new pull request #5172: URL: https://github.com/apache/hudi/pull/5172 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpos

[GitHub] [hudi] hudi-bot commented on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot commented on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082443989 ## CI report: * b1b66d9e789e16ab90b6cd20d31befd643e45edc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082292813 ## CI report: * b1b66d9e789e16ab90b6cd20d31befd643e45edc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] guanziyue commented on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
guanziyue commented on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082442330 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082440923 ## CI report: * 5721b6086f439a09eb9d94ffbd09d53f083e1316 UNKNOWN * 9267c4b23e411349a4be9556a44e37f31dedd5ac Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082361287 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[jira] [Closed] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3062. - Resolution: Not A Problem > savepoint rollback of last but one savepoint fails > -

[jira] [Updated] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3062: -- Status: Open (was: In Progress) > savepoint rollback of last but one savepoint fails >

[jira] [Commented] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514353#comment-17514353 ] sivabalan narayanan commented on HUDI-3062: --- This is expected behavior  Code sn

[jira] [Updated] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3062: -- Status: In Progress (was: Open) > savepoint rollback of last but one savepoint fails >

[GitHub] [hudi] hudi-bot commented on pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5171: URL: https://github.com/apache/hudi/pull/5171#issuecomment-1082421108 ## CI report: * 80d7377bf9846f5f1da01684b246c8a2be5013c0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5171: URL: https://github.com/apache/hudi/pull/5171#issuecomment-1082418814 ## CI report: * 80d7377bf9846f5f1da01684b246c8a2be5013c0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5171: URL: https://github.com/apache/hudi/pull/5171#issuecomment-1082418814 ## CI report: * 80d7377bf9846f5f1da01684b246c8a2be5013c0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[jira] [Updated] (HUDI-3681) Provision additional bundles aliased to Spark minor version

2022-03-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3681: - Labels: pull-request-available (was: ) > Provision additional bundles aliased to Spark minor vers

[jira] [Assigned] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3062: - Assignee: sivabalan narayanan > savepoint rollback of last but one savepoint fail

[GitHub] [hudi] yihua opened a new pull request #5171: [WIP][HUDI-3681] Provision additional hudi-spark-bundle with different versions

2022-03-29 Thread GitBox
yihua opened a new pull request #5171: URL: https://github.com/apache/hudi/pull/5171 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose o

[jira] [Assigned] (HUDI-3724) Too many open files w/ COW spark long running tests

2022-03-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-3724: - Assignee: sivabalan narayanan (was: Alexey Kudinkin) > Too many open files w/ COW spark

[jira] [Updated] (HUDI-3748) Hudi fails to insert into a partitioned table when the partition column is dropped from the parquet schema

2022-03-29 Thread Vinoth Govindarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Govindarajan updated HUDI-3748: -- Description: When you add this config to drop the partition column from the parquet sche

[jira] [Created] (HUDI-3748) Hudi fails to insert into a partitioned table when the partition column is dropped from the parquet schema

2022-03-29 Thread Vinoth Govindarajan (Jira)
Vinoth Govindarajan created HUDI-3748: - Summary: Hudi fails to insert into a partitioned table when the partition column is dropped from the parquet schema Key: HUDI-3748 URL: https://issues.apache.org/jira/br

[GitHub] [hudi] bvaradar commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

2022-03-29 Thread GitBox
bvaradar commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1082370943 @xiarixiaoyao : Can you rebase and fix the conflicts. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] bvaradar commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

2022-03-29 Thread GitBox
bvaradar commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1082370418 Overall looks good. @xushiyan : There is a dependency with https://github.com/apache/hudi/pull/5168 also need to go with this together. cc @vinothchandar: For a final pass

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082361287 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082353688 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082353688 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082336759 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[jira] [Updated] (HUDI-3745) Add support for spark data-source reader options in S3EventsHoodieIncrSource

2022-03-29 Thread Harshal Patil (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patil updated HUDI-3745: Description: S3EventsHoodieIncrSource reader supports different file formats .  For each of these so

[GitHub] [hudi] hudi-bot commented on pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

2022-03-29 Thread GitBox
hudi-bot commented on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1082347853 ## CI report: * 010de76ddd6c0201db746a13a5b04fc5e94125d4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4693: [HUDI-2488][HUDI-3175] Implement async metadata indexing

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #4693: URL: https://github.com/apache/hudi/pull/4693#issuecomment-1082169589 ## CI report: * be08ba499bb88d8a00f20695b360336853be708e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
nsivabalan commented on a change in pull request #5170: URL: https://github.com/apache/hudi/pull/5170#discussion_r837883740 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/S3EventsHoodieIncrSource.java ## @@ -71,6 +75,12 @@ static final String

[GitHub] [hudi] harsh1231 commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
harsh1231 commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082341621 > Have you tested the patch ? @nsivabalan Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] harsh1231 commented on a change in pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
harsh1231 commented on a change in pull request #5170: URL: https://github.com/apache/hudi/pull/5170#discussion_r837880142 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/HoodieIncrSource.java ## @@ -88,6 +88,13 @@ * {@value #SOURCE_FILE_FORM

[GitHub] [hudi] harsh1231 removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
harsh1231 removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082339739 > Have you tested the patch ? Yes > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hudi] harsh1231 commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
harsh1231 commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082339739 > Have you tested the patch ? Yes > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082333952 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082336759 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082333952 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082310550 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #5169: [HUDI-3743] Support DELETE_PARTITION for metadata table

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5169: URL: https://github.com/apache/hudi/pull/5169#issuecomment-1082322067 ## CI report: * 63fdb815e3c1a5ca12e9ea6ae54b002a8af85222 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5169: [HUDI-3743] Support DELETE_PARTITION for metadata table

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5169: URL: https://github.com/apache/hudi/pull/5169#issuecomment-1082167045 ## CI report: * 25b886df4fb025c526a93e81bafeb5a0dc3ef89b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
nsivabalan commented on a change in pull request #5170: URL: https://github.com/apache/hudi/pull/5170#discussion_r837861114 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/S3EventsHoodieIncrSource.java ## @@ -174,7 +180,19 @@ public S3EventsHoodieI

[jira] [Updated] (HUDI-3726) Switching from non-partitioned to partitioned key gen does not throw any exception

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3726: -- Description: in commit C1, if non-partitioned key gen is used and for commit C2, if key

[jira] [Assigned] (HUDI-3726) Switching from non-partitioned to partitioned key gen does not throw any exception

2022-03-29 Thread Rajesh (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh reassigned HUDI-3726: Assignee: Rajesh (was: sivabalan narayanan) > Switching from non-partitioned to partitioned key gen does n

[GitHub] [hudi] hudi-bot removed a comment on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082307869 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082310550 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Updated] (HUDI-1508) Partition update with global index in MOR tables resulting in duplicate values during read optimized queries

2022-03-29 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated HUDI-1508: Labels: blocker release-blocker (was: ) > Partition update with global index in MOR tables resultin

[GitHub] [hudi] hudi-bot commented on pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5170: URL: https://github.com/apache/hudi/pull/5170#issuecomment-1082307869 ## CI report: * 02d5b4b9dc2eeb87fd413f40398b064904df7fff UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[jira] [Updated] (HUDI-3745) Add support for spark data-source reader options in S3EventsHoodieIncrSource

2022-03-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3745: - Labels: pull-request-available (was: ) > Add support for spark data-source reader options in S3Ev

[GitHub] [hudi] harsh1231 opened a new pull request #5170: [HUDI-3745] Support for spark datasource options in S3EventsHoodieInc…

2022-03-29 Thread GitBox
harsh1231 opened a new pull request #5170: URL: https://github.com/apache/hudi/pull/5170 Support for spark datasource options in S3EventsHoodieIncrSource While reading from spark file datasource, reader options could be provided in deltastreamer like ```- --hoodie-conf hoodie.deltas

[jira] [Updated] (HUDI-3724) Too many open files w/ COW spark long running tests

2022-03-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3724: -- Status: In Progress (was: Open) > Too many open files w/ COW spark long running tests > ---

[jira] [Assigned] (HUDI-3724) Too many open files w/ COW spark long running tests

2022-03-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin reassigned HUDI-3724: - Assignee: Alexey Kudinkin (was: sivabalan narayanan) > Too many open files w/ COW spark

[jira] [Updated] (HUDI-3653) Clean up Column Stats Index introduced along with Spatial Curves Clustering

2022-03-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3653: -- Status: Patch Available (was: In Progress) > Clean up Column Stats Index introduced along with

[GitHub] [hudi] hudi-bot commented on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot commented on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082292813 ## CI report: * b1b66d9e789e16ab90b6cd20d31befd643e45edc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4264: [HUDI-2875] Make HoodieParquetWriter Thread safe and memory executor …

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #4264: URL: https://github.com/apache/hudi/pull/4264#issuecomment-1082109014 ## CI report: * 6c01b6b8b105384aa260ea0a3cbe7d98c207760c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] alexeykudinkin commented on pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

2022-03-29 Thread GitBox
alexeykudinkin commented on pull request #4957: URL: https://github.com/apache/hudi/pull/4957#issuecomment-1082292435 Great work @XuQianJin-Stars! Great to see this flow cleaned up and greatly simplified. -- This is an automated message from the Apache Git Service. To respond to the mess

[jira] [Updated] (HUDI-3729) Enable Spark vectorized read for non-incremental read paths

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3729: - Status: Patch Available (was: In Progress) > Enable Spark vectorized read for non-incremental read paths

[jira] [Updated] (HUDI-3729) Enable Spark vectorized read for non-incremental read paths

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3729: - Status: In Progress (was: Open) > Enable Spark vectorized read for non-incremental read paths > -

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4957: [HUDI-3406] Rollback incorrectly relying on FS listing instead of Com…

2022-03-29 Thread GitBox
alexeykudinkin commented on a change in pull request #4957: URL: https://github.com/apache/hudi/pull/4957#discussion_r837817036 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/rollback/ListingBasedRollbackStrategy.java ## @@ -57,20 +81,2

[jira] [Updated] (HUDI-3743) Support DELETE_PARTITION for metadata table

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3743: - Sprint: Hudi-Sprint-Mar-22 > Support DELETE_PARTITION for metadata table > ---

[jira] [Updated] (HUDI-3724) Too many open files w/ COW spark long running tests

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3724: - Sprint: Hudi-Sprint-Mar-22 > Too many open files w/ COW spark long running tests > ---

[jira] [Updated] (HUDI-3062) savepoint rollback of last but one savepoint fails

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3062: - Sprint: Hudi-Sprint-Mar-22 > savepoint rollback of last but one savepoint fails >

[jira] [Updated] (HUDI-2473) Fix compaction action type in commit metadata

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2473: - Sprint: Hudi-Sprint-Mar-22 > Fix compaction action type in commit metadata > -

[jira] [Closed] (HUDI-2520) Certify sync with Hive 3

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2520. Reviewers: Raymond Xu, rex xiong (was: Raymond Xu) Resolution: Fixed > Certify sync with Hive 3 >

[hudi] branch master updated (e5a2bae -> 941c254)

2022-03-29 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from e5a2bae [HUDI-3549] Removing dependency on "spark-avro" (#4955) add 941c254 [HUDI-2520] Fix CTAS statment iss

[GitHub] [hudi] xushiyan merged pull request #5145: [HUDI-2520] Fix CTAS statment issue when sync to hive

2022-03-29 Thread GitBox
xushiyan merged pull request #5145: URL: https://github.com/apache/hudi/pull/5145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[jira] [Updated] (HUDI-3747) Automate pyspark quick start guide runbook

2022-03-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3747: -- Fix Version/s: 0.12.0 > Automate pyspark quick start guide runbook > ---

[jira] [Created] (HUDI-3747) Automate pyspark quick start guide runbook

2022-03-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-3747: - Summary: Automate pyspark quick start guide runbook Key: HUDI-3747 URL: https://issues.apache.org/jira/browse/HUDI-3747 Project: Apache Hudi Issue

[jira] [Commented] (HUDI-3681) Provision additional bundles aliased to Spark minor version

2022-03-29 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514290#comment-17514290 ] Ethan Guo commented on HUDI-3681: - Per discussion, we'll keep the current hudi-utilities-b

[GitHub] [hudi] nsivabalan commented on issue #4604: [SUPPORT] Archive functionality fails

2022-03-29 Thread GitBox
nsivabalan commented on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1082283943 @andykrk : do you have any updates for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] nsivabalan commented on issue #4635: [SUPPORT] Bulk write failing due to hudi timeline archive exception

2022-03-29 Thread GitBox
nsivabalan commented on issue #4635: URL: https://github.com/apache/hudi/issues/4635#issuecomment-1082277719 We have identified an issue w/ multi-writer wrt archival resulting in NPE and the fix is [here](https://github.com/apache/hudi/pull/5138). If you can give it a try and let us know,

[jira] [Updated] (HUDI-3746) CI ignored test failure in TestDataSkippingUtils

2022-03-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3746: - Attachment: ci.log.zip > CI ignored test failure in TestDataSkippingUtils > --

[jira] [Created] (HUDI-3746) CI ignored test failure in TestDataSkippingUtils

2022-03-29 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3746: Summary: CI ignored test failure in TestDataSkippingUtils Key: HUDI-3746 URL: https://issues.apache.org/jira/browse/HUDI-3746 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #5145: [HUDI-2520] Fix CTAS statment issue when sync to hive

2022-03-29 Thread GitBox
hudi-bot commented on pull request #5145: URL: https://github.com/apache/hudi/pull/5145#issuecomment-1082264681 ## CI report: * 89c1abb8bf59cd72baffef0c3732271a74b6a52d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #5145: [HUDI-2520] Fix CTAS statment issue when sync to hive

2022-03-29 Thread GitBox
hudi-bot removed a comment on pull request #5145: URL: https://github.com/apache/hudi/pull/5145#issuecomment-1082062512 ## CI report: * 05246d21de4705aca24d3c984b24025f0ddb62d8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[jira] [Closed] (HUDI-3549) Investigate spark3 read issues w/ hudi spark bundle 3.2 with S3 dataset

2022-03-29 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3549. - Resolution: Fixed > Investigate spark3 read issues w/ hudi spark bundle 3.2 with S3 dataset >

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #5166: [MINOR] Fix dates as per UTC in TestDataSkippingUtils

2022-03-29 Thread GitBox
alexeykudinkin commented on a change in pull request #5166: URL: https://github.com/apache/hudi/pull/5166#discussion_r837803667 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestDataSkippingUtils.scala ## @@ -384,186 +384,187 @@ object TestDataS

<    1   2   3   4   5   6   7   >