[GitHub] [hudi] hudi-bot removed a comment on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-990699643 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-990705653 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-990699643 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4236: [HUDI-2936] Add data count checks in async clustering tests

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4236: URL: https://github.com/apache/hudi/pull/4236#issuecomment-987617704 ## CI report: * e4908379cb7faee6bdc554b0937b9a4557797eea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-990655541 ## CI report: * 1e718c4bcfe432a4ac03f807c889c67ee8d962ae Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-990683626 ## CI report: * 1e718c4bcfe432a4ac03f807c889c67ee8d962ae Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] suribabu-un commented on issue #4151: [SUPPORT] ClassNotFoundException: org.apache.hudi.hadoop.HoodieParquetInputFormat while running hive queries in EMR

2021-12-09 Thread GitBox
suribabu-un commented on issue #4151: URL: https://github.com/apache/hudi/issues/4151#issuecomment-990681320 Issue is unrelated to hudi, it has to do with the llap is running in the emr cluster. As mentioned above if llap is disabled then queries are running as expected. Issue can be re

[GitHub] [hudi] suribabu-un closed issue #4151: [SUPPORT] ClassNotFoundException: org.apache.hudi.hadoop.HoodieParquetInputFormat while running hive queries in EMR

2021-12-09 Thread GitBox
suribabu-un closed issue #4151: URL: https://github.com/apache/hudi/issues/4151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990661766 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990681037 ## CI report: * bcc62e5eeea6a2929e4144c00f2d0b29bcc786cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] YannByron commented on a change in pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron commented on a change in pull request #4269: URL: https://github.com/apache/hudi/pull/4269#discussion_r766402385 ## File path: website/docs/quick-start-guide.md ## @@ -175,18 +175,163 @@ values={[ +Spark-sql needs an explicit create table command. + +- Table typ

[GitHub] [hudi] JoshuaZhuCN opened a new issue #4275: [SUPPORT] How can I control the number of archive files

2021-12-09 Thread GitBox
JoshuaZhuCN opened a new issue #4275: URL: https://github.com/apache/hudi/issues/4275 When I use clustering async, I generate a lot of archive files, similar to commits. archive. xx_ 1-0-1 how to control or clean up the number of these files **Environment Description**

[GitHub] [hudi] hudi-bot commented on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990672722 ## CI report: * dd0773b261cd2d6d503eaa3e02c93edddcb31093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990651217 ## CI report: * dd0773b261cd2d6d503eaa3e02c93edddcb31093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] YannByron commented on a change in pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron commented on a change in pull request #4269: URL: https://github.com/apache/hudi/pull/4269#discussion_r766402385 ## File path: website/docs/quick-start-guide.md ## @@ -175,18 +175,163 @@ values={[ +Spark-sql needs an explicit create table command. + +- Table typ

[GitHub] [hudi] YannByron commented on a change in pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron commented on a change in pull request #4269: URL: https://github.com/apache/hudi/pull/4269#discussion_r766387860 ## File path: website/docs/quick-start-guide.md ## @@ -175,18 +175,163 @@ values={[ +Spark-sql needs an explicit create table command. + +- Table typ

[hudi] branch asf-site updated (34e151d -> d003ae0)

2021-12-09 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git. from 34e151d [MINOR] Fix asf-site build error (#4273) add d003ae0 Travis CI build asf-site No new revisions were a

[GitHub] [hudi] YannByron commented on a change in pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron commented on a change in pull request #4269: URL: https://github.com/apache/hudi/pull/4269#discussion_r766383550 ## File path: website/docs/quick-start-guide.md ## @@ -175,18 +175,163 @@ values={[ +Spark-sql needs an explicit create table command. + +- Table typ

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990660728 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990661766 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990660728 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990611154 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xushiyan commented on a change in pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
xushiyan commented on a change in pull request #4269: URL: https://github.com/apache/hudi/pull/4269#discussion_r766367063 ## File path: website/docs/quick-start-guide.md ## @@ -175,18 +175,163 @@ values={[ +Spark-sql needs an explicit create table command. + +- Table type

[hudi] branch master updated (ea154bc -> 456d74c)

2021-12-09 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from ea154bc Revert "Claiming RFC for data skipping index for updated version (#4271)" (#4272) add 456d74c [HUDI-2901

[GitHub] [hudi] yihua merged pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
yihua merged pull request #4178: URL: https://github.com/apache/hudi/pull/4178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot commented on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-990655541 ## CI report: * 1e718c4bcfe432a4ac03f807c889c67ee8d962ae Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-990654497 ## CI report: * 1e718c4bcfe432a4ac03f807c889c67ee8d962ae UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] Carl-Zhou-CN commented on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

2021-12-09 Thread GitBox
Carl-Zhou-CN commented on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-990654733 Because of your hudi version, you may need to manually update the partition after writing ALTER TABLE table_name RECOVER PARTITIONS; -- This is an automated message from the A

[GitHub] [hudi] hudi-bot commented on pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4274: URL: https://github.com/apache/hudi/pull/4274#issuecomment-990654497 ## CI report: * 1e718c4bcfe432a4ac03f807c889c67ee8d962ae UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[jira] [Updated] (HUDI-2974) Make the prefix for metrics name configurable

2021-12-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2974: - Labels: pull-request-available (was: ) > Make the prefix for metrics name configurable >

[GitHub] [hudi] rmahindra123 opened a new pull request #4274: [HUDI-2974] Make the prefix for metrics name configurable

2021-12-09 Thread GitBox
rmahindra123 opened a new pull request #4274: URL: https://github.com/apache/hudi/pull/4274 Currently metrics names always start with table name. This makes it less flexible to create grafana dashboards with prometheus query. since its easier to have consistent metrics names across all spa

[jira] [Created] (HUDI-2974) Make the prefix for metrics name configurable

2021-12-09 Thread Rajesh Mahindra (Jira)
Rajesh Mahindra created HUDI-2974: - Summary: Make the prefix for metrics name configurable Key: HUDI-2974 URL: https://issues.apache.org/jira/browse/HUDI-2974 Project: Apache Hudi Issue Type:

[GitHub] [hudi] codope merged pull request #4273: [MINOR] Fix asf-site build error

2021-12-09 Thread GitBox
codope merged pull request #4273: URL: https://github.com/apache/hudi/pull/4273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[hudi] branch asf-site updated: [MINOR] Fix asf-site build error (#4273)

2021-12-09 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 34e151d [MINOR] Fix asf-site build error (#42

[GitHub] [hudi] hudi-bot removed a comment on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990556714 ## CI report: * dd0773b261cd2d6d503eaa3e02c93edddcb31093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990651217 ## CI report: * dd0773b261cd2d6d503eaa3e02c93edddcb31093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] YuweiXiao commented on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
YuweiXiao commented on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990650172 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [hudi] xiarixiaoyao commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
xiarixiaoyao commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990647589 @vinothchandar @alexeykudinkin @leesf already update the code and address all comments. pls help me review again, thanks -- This is an automated message from the Apache G

[GitHub] [hudi] leesf commented on pull request #3964: [HUDI-2732][RFC-38] Spark Datasource V2 Integration

2021-12-09 Thread GitBox
leesf commented on pull request #3964: URL: https://github.com/apache/hudi/pull/3964#issuecomment-990645306 > > And In the first phase, we would fallback to V1 write path > > Can this be done? Love to see some code for this. yes, will open a PR in recent days. -- This is an

[GitHub] [hudi] Carl-Zhou-CN commented on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

2021-12-09 Thread GitBox
Carl-Zhou-CN commented on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-990644240 "hoodie.datasource.hive_sync.enable": "true", "hoodie.datasource.hive_sync.table": "my_hudi_table", "hoodie.datasource.hive_sync.partition_fields": "creation_date",

[hudi] branch master updated (8321d20 -> ea154bc)

2021-12-09 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 8321d20 Claiming RFC for data skipping index for updated version (#4271) add ea154bc Revert "Claiming RFC fo

[GitHub] [hudi] nsivabalan merged pull request #4272: [MINOR] Revert "Claiming RFC for data skipping index for updated version (#42…

2021-12-09 Thread GitBox
nsivabalan merged pull request #4272: URL: https://github.com/apache/hudi/pull/4272 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan opened a new pull request #4272: [MINOR] Revert "Claiming RFC for data skipping index for updated version (#42…

2021-12-09 Thread GitBox
nsivabalan opened a new pull request #4272: URL: https://github.com/apache/hudi/pull/4272 …71)" This reverts commit 8321d20c2cced15150621c9ad828f5ba9d79399a. ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contri

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990610366 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990626291 ## CI report: * 43c1e05bea47d18730eec37c24d94755d291c2f1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] Arun-kc commented on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

2021-12-09 Thread GitBox
Arun-kc commented on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-990615939 @Carl-Zhou-CN The following is the hudi options I'm using as of now. ```python hudiOptions = { "hoodie.table.name": "my_hudi_table", "hoodie.datasource.write.recor

[hudi] branch master updated: Claiming RFC for data skipping index for updated version (#4271)

2021-12-09 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8321d20 Claiming RFC for data skipping index for

[GitHub] [hudi] codope merged pull request #4271: [HUDI-2973] Claiming RFC number for data skipping index (updated version)

2021-12-09 Thread GitBox
codope merged pull request #4271: URL: https://github.com/apache/hudi/pull/4271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990597314 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990611154 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990610366 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990609511 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990609511 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990597210 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot removed a comment on pull request #4271: [HUDI-2973] Claiming RFC number for data skipping index (updated version)

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4271: URL: https://github.com/apache/hudi/pull/4271#issuecomment-990597335 ## CI report: * b089271cd1db1ee41ed34018a9056450194cb900 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4271: [HUDI-2973] Claiming RFC number for data skipping index (updated version)

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4271: URL: https://github.com/apache/hudi/pull/4271#issuecomment-990598889 ## CI report: * b089271cd1db1ee41ed34018a9056450194cb900 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot commented on pull request #4271: [HUDI-2973] Claiming RFC number for data skipping index (updated version)

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4271: URL: https://github.com/apache/hudi/pull/4271#issuecomment-990597335 ## CI report: * b089271cd1db1ee41ed34018a9056450194cb900 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990595447 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990597314 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990569685 ## CI report: * 2589cfb570762c4dca5968fae72f9b7948a69f31 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990597210 ## CI report: * c4cffc9908f2a8e79f4c24dc566942f2c6d8b752 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[jira] [Updated] (HUDI-2973) Rewrite/re-publish RFC for Data skipping index

2021-12-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2973: - Labels: pull-request-available (was: ) > Rewrite/re-publish RFC for Data skipping index > ---

[GitHub] [hudi] hudi-bot commented on pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4270: URL: https://github.com/apache/hudi/pull/4270#issuecomment-990595447 ## CI report: * 7095ede3d5fa162df3804d05c3a1ff009e6f4ef4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` r

[GitHub] [hudi] nsivabalan opened a new pull request #4271: [HUDI-2973] Claiming RFC number for data skipping index (updated version)

2021-12-09 Thread GitBox
nsivabalan opened a new pull request #4271: URL: https://github.com/apache/hudi/pull/4271 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purp

[jira] [Created] (HUDI-2973) Rewrite/re-publish RFC for Data skipping index

2021-12-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2973: - Summary: Rewrite/re-publish RFC for Data skipping index Key: HUDI-2973 URL: https://issues.apache.org/jira/browse/HUDI-2973 Project: Apache Hudi Is

[jira] [Assigned] (HUDI-2973) Rewrite/re-publish RFC for Data skipping index

2021-12-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2973: - Assignee: sivabalan narayanan > Rewrite/re-publish RFC for Data skipping index >

[jira] [Updated] (HUDI-2973) Rewrite/re-publish RFC for Data skipping index

2021-12-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2973: -- Parent: HUDI-1822 Issue Type: Sub-task (was: Improvement) > Rewrite/re-publish

[jira] [Updated] (HUDI-2811) Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2811: - Labels: pull-request-available sev:critical (was: sev:critical) > Support Spark 3.2 and Parquet 1

[GitHub] [hudi] YannByron opened a new pull request #4270: [HUDI-2811] Support Spark 3.2 and Parquet 1.12.x

2021-12-09 Thread GitBox
YannByron opened a new pull request #4270: URL: https://github.com/apache/hudi/pull/4270 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] YannByron commented on issue #4208: [SUPPORT] On Hudi 0.9.0 - Alter table throws java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.alterTable(java.lang.String, o

2021-12-09 Thread GitBox
YannByron commented on issue #4208: URL: https://github.com/apache/hudi/issues/4208#issuecomment-990591132 Hi, @BenjMaq i can't reproduce this issue. Can you check your environment? Based on the error above, i guess maybe the conflicts between jar cause this. -- This is an automated m

[GitHub] [hudi] nsivabalan commented on a change in pull request #3887: [HUDI-2648] Retry FileSystem action instead of failed directly.

2021-12-09 Thread GitBox
nsivabalan commented on a change in pull request #3887: URL: https://github.com/apache/hudi/pull/3887#discussion_r766321890 ## File path: hudi-common/src/main/java/org/apache/hudi/common/fs/FileSystemGuardConfig.java ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] nsivabalan commented on pull request #3887: [HUDI-2648] Retry FileSystem action instead of failed directly.

2021-12-09 Thread GitBox
nsivabalan commented on pull request #3887: URL: https://github.com/apache/hudi/pull/3887#issuecomment-990589947 sure, makes sense if there are other cloud stores that needs this retry. Can you please address the feedback given already. -- This is an automated message from the Apache Gi

[GitHub] [hudi] YannByron edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-09 Thread GitBox
YannByron edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-990561751 Hey, @BenjMaq I can't reproduce this issue using your sql in both hudi 0.9 and 0.10. I use spark-2.4.4 in [here](https://archive.apache.org/dist/spark/spark-2.4.4/) and hu

[GitHub] [hudi] YannByron edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-09 Thread GitBox
YannByron edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-990561751 Hey, @BenjMaq In both hudi 0.9 and 0.10, `insert overwrite` can work well. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] Rap70r commented on issue #4242: [SUPPORT] Split Data into Multiple Parquet files under Partitions

2021-12-09 Thread GitBox
Rap70r commented on issue #4242: URL: https://github.com/apache/hudi/issues/4242#issuecomment-990582235 Got it, thank you. You mentioned that we can employ clustering to batch lot of small files together. Is there a specific configuration we need to set to achieve that? We are running Hudi

[GitHub] [hudi] YannByron edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-09 Thread GitBox
YannByron edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-990561751 Hey, @BenjMaq In both hudi 0.9 and 0.10, `insert overwrite` can work well. My spark version is 2.4.7, but i think it's ok. -- This is an automated message from the Apache

[GitHub] [hudi] Carl-Zhou-CN commented on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

2021-12-09 Thread GitBox
Carl-Zhou-CN commented on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-990576501 @Arun-kc It feels like a connection problem, please check hoodie.datasource.hive_sync.jdbcurl, it seems to be a default value now -- This is an automated message from the Apache

[jira] [Assigned] (HUDI-2903) get table schema from the last commit with data written

2021-12-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-2903: Assignee: Yann Byron > get table schema from the last commit with data written > --

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990569685 ## CI report: * 2589cfb570762c4dca5968fae72f9b7948a69f31 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] nsivabalan commented on issue #4242: [SUPPORT] Split Data into Multiple Parquet files under Partitions

2021-12-09 Thread GitBox
nsivabalan commented on issue #4242: URL: https://github.com/apache/hudi/issues/4242#issuecomment-990569759 yes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990552126 ## CI report: * 2589cfb570762c4dca5968fae72f9b7948a69f31 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] YannByron commented on pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron commented on pull request #4269: URL: https://github.com/apache/hudi/pull/4269#issuecomment-990569190 @nsivabalan @xushiyan please help to review this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HUDI-2878) Enhance hudi-quick start guide for spark-sql

2021-12-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2878: - Labels: pull-request-available (was: ) > Enhance hudi-quick start guide for spark-sql > -

[GitHub] [hudi] zztttt commented on issue #4072: [SUPPORT]Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/scala/table6

2021-12-09 Thread GitBox
zz commented on issue #4072: URL: https://github.com/apache/hudi/issues/4072#issuecomment-990568201 > ``` > Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/zzt/code/spark-debug/metastore_db. > ``` > > guess you already have anoth

[GitHub] [hudi] YannByron opened a new pull request #4269: [HUDI-2878] enhance hudi-quick-start guide for spark-sql

2021-12-09 Thread GitBox
YannByron opened a new pull request #4269: URL: https://github.com/apache/hudi/pull/4269 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpo

[GitHub] [hudi] YannByron commented on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-09 Thread GitBox
YannByron commented on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-990561751 Hey, @BenjMaq I test that it works in version 0.10. Can you use hudi 0.10? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990527370 ## CI report: * a085e101422d1df36b94127e75e5d60716986e69 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990556714 ## CI report: * dd0773b261cd2d6d503eaa3e02c93edddcb31093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990550517 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990552126 ## CI report: * 2589cfb570762c4dca5968fae72f9b7948a69f31 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990550517 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990534719 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
xiarixiaoyao commented on a change in pull request #4178: URL: https://github.com/apache/hudi/pull/4178#discussion_r766290661 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java ## @@ -

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
alexeykudinkin commented on a change in pull request #4178: URL: https://github.com/apache/hudi/pull/4178#discussion_r766290137 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java ## @@

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990534719 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990532933 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
alexeykudinkin commented on a change in pull request #4178: URL: https://github.com/apache/hudi/pull/4178#discussion_r766289221 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java ## @@

[GitHub] [hudi] hudi-bot commented on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-990532933 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

[GitHub] [hudi] hudi-bot removed a comment on pull request #4178: [HUDI-2901] Fixed the bug clustering jobs cannot running in parallel

2021-12-09 Thread GitBox
hudi-bot removed a comment on pull request #4178: URL: https://github.com/apache/hudi/pull/4178#issuecomment-984239555 ## CI report: * c454677b96fab062cf31634426646d741ac9dbe5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[GitHub] [hudi] hudi-bot commented on pull request #4222: [HUDI-2849] improve SparkUI job description for write path

2021-12-09 Thread GitBox
hudi-bot commented on pull request #4222: URL: https://github.com/apache/hudi/pull/4222#issuecomment-990527370 ## CI report: * a085e101422d1df36b94127e75e5d60716986e69 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?bu

  1   2   3   4   >