[GitHub] [hudi] 7c00 commented on a change in pull request #4563: [HUDI-3211][RFC-44] Add RFC for Hudi Connector for Presto

2022-01-16 Thread GitBox
7c00 commented on a change in pull request #4563: URL: https://github.com/apache/hudi/pull/4563#discussion_r785712959 ## File path: rfc/rfc-44/rfc-44.md ## @@ -0,0 +1,156 @@ + + +# RFC-44: Hudi Connector for Presto + +## Proposers + +- @7c00 + +## Approvers + +- @ + +## Status

[GitHub] [hudi] 7c00 commented on a change in pull request #4563: [HUDI-3211][RFC-44] Add RFC for Hudi Connector for Presto

2022-01-16 Thread GitBox
7c00 commented on a change in pull request #4563: URL: https://github.com/apache/hudi/pull/4563#discussion_r785712959 ## File path: rfc/rfc-44/rfc-44.md ## @@ -0,0 +1,156 @@ + + +# RFC-44: Hudi Connector for Presto + +## Proposers + +- @7c00 + +## Approvers + +- @ + +## Status

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r785709523 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java ## @@ -248,11 +253,32 @@ private HoodieInstant

[GitHub] [hudi] 7c00 commented on a change in pull request #4563: [HUDI-3211][RFC-44] Add RFC for Hudi Connector for Presto

2022-01-16 Thread GitBox
7c00 commented on a change in pull request #4563: URL: https://github.com/apache/hudi/pull/4563#discussion_r785708133 ## File path: rfc/rfc-44/rfc-44.md ## @@ -0,0 +1,156 @@ + + +# RFC-44: Hudi Connector for Presto + +## Proposers + +- @7c00 + +## Approvers + +- @ + +## Status

[GitHub] [hudi] XuQianJin-Stars edited a comment on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
XuQianJin-Stars edited a comment on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1014225694 > > hi @danny0405 Have time to help me review this PR? > > Thanks for the contribution, why we change `HoodieTableSource` into `HoodieTableSourceContext` ?

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
XuQianJin-Stars commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1014225694 > > hi @danny0405 Have time to help me review this PR? > > Thanks for the contribution, why we change `HoodieTableSource` into `HoodieTableSourceContext` ?

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r785684621 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java ## @@ -160,4 +160,33 @@ public static void closeQuietly(Closeable

[GitHub] [hudi] danny0405 commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
danny0405 commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1014218148 > hi @danny0405 Have time to help me review this PR? Thanks for the contribution, why we change `HoodieTableSource` into `HoodieTableSourceContext` ? -- This is an

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r785684248 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java ## @@ -160,4 +160,33 @@ public static void closeQuietly(Closeable

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r785684248 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java ## @@ -160,4 +160,33 @@ public static void closeQuietly(Closeable

[GitHub] [hudi] yihua commented on a change in pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4078: URL: https://github.com/apache/hudi/pull/4078#discussion_r785684248 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java ## @@ -160,4 +160,33 @@ public static void closeQuietly(Closeable

[GitHub] [hudi] hudi-bot removed a comment on pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4616: URL: https://github.com/apache/hudi/pull/4616#issuecomment-1014195679 ## CI report: * a4173a03fceab5ba874ebd9e86aba63e005d640d UNKNOWN * 79ce599b8833c1606bdcb463ab74f21bbd01c521 UNKNOWN Bot commands @hudi-bot

[GitHub] [hudi] hudi-bot commented on pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4616: URL: https://github.com/apache/hudi/pull/4616#issuecomment-1014197006 ## CI report: * a4173a03fceab5ba874ebd9e86aba63e005d640d UNKNOWN * 79ce599b8833c1606bdcb463ab74f21bbd01c521 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4616: URL: https://github.com/apache/hudi/pull/4616#issuecomment-1014194369 ## CI report: * a4173a03fceab5ba874ebd9e86aba63e005d640d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot

[GitHub] [hudi] hudi-bot commented on pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4616: URL: https://github.com/apache/hudi/pull/4616#issuecomment-1014195679 ## CI report: * a4173a03fceab5ba874ebd9e86aba63e005d640d UNKNOWN * 79ce599b8833c1606bdcb463ab74f21bbd01c521 UNKNOWN Bot commands @hudi-bot supports

[GitHub] [hudi] hudi-bot commented on pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4616: URL: https://github.com/apache/hudi/pull/4616#issuecomment-1014194369 ## CI report: * a4173a03fceab5ba874ebd9e86aba63e005d640d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] yihua commented on a change in pull request #4531: [HUDI-3191][Stacked on 4520] Rebasing Hive's FileInputFormat onto `AbstractHoodieTableFileIndex`

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4531: URL: https://github.com/apache/hudi/pull/4531#discussion_r785677265 ## File path: hudi-common/src/main/scala/org/apache/hudi/AbstractHoodieTableFileIndex.scala ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (HUDI-3257) Excluding clustering instants from pending rollback info

2022-01-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3257: - Labels: pull-request-available sev:normal (was: sev:normal) > Excluding clustering instants from

[GitHub] [hudi] danny0405 opened a new pull request #4616: [HUDI-3257] Excluding clustering instants from pending rollback info

2022-01-16 Thread GitBox
danny0405 opened a new pull request #4616: URL: https://github.com/apache/hudi/pull/4616 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] yihua commented on a change in pull request #4531: [HUDI-3191][Stacked on 4520] Rebasing Hive's FileInputFormat onto `AbstractHoodieTableFileIndex`

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4531: URL: https://github.com/apache/hudi/pull/4531#discussion_r785676860 ## File path: hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java ## @@ -202,7 +202,13 @@ public static String getFileId(String fullFileName) {

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1014190454 ## CI report: * 6c1c19492e82d894d20095e1b5038d29fd2d3322 Azure:

[hudi] branch master updated (ed92c21 -> 75caa7d)

2022-01-16 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from ed92c21 [MINOR] Delete unused parameter in TablePathUtils (#4595) add 75caa7d [HUDI-3179] Extracted common

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1014161030 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure:

[GitHub] [hudi] yihua commented on a change in pull request #4520: [HUDI-3179] Extracted common `AbstractHoodieTableFileIndex` to be shared across engines

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4520: URL: https://github.com/apache/hudi/pull/4520#discussion_r785675507 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/AbstractHoodieTableFileIndex.scala ## @@ -0,0 +1,309 @@ +/* + * Licensed to the

[jira] [Assigned] (HUDI-2941) Show _hoodie_operation in spark sql results

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-2941: Assignee: Forward Xu (was: dalongliu) > Show _hoodie_operation in spark sql results >

[GitHub] [hudi] yihua merged pull request #4520: [HUDI-3179] Extracted common `AbstractHoodieTableFileIndex` to be shared across engines

2022-01-16 Thread GitBox
yihua merged pull request #4520: URL: https://github.com/apache/hudi/pull/4520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (HUDI-3151) Docs for Spark SQL type support

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3151: Assignee: Forward Xu (was: Yann Byron) > Docs for Spark SQL type support >

[GitHub] [hudi] yihua commented on a change in pull request #4520: [HUDI-3179] Extracted common `AbstractHoodieTableFileIndex` to be shared across engines

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4520: URL: https://github.com/apache/hudi/pull/4520#discussion_r785674613 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/AbstractHoodieTableFileIndex.scala ## @@ -0,0 +1,309 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot removed a comment on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1014182792 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 Azure:

[GitHub] [hudi] yihua commented on a change in pull request #4520: [HUDI-3179] Extracted common `AbstractHoodieTableFileIndex` to be shared across engines

2022-01-16 Thread GitBox
yihua commented on a change in pull request #4520: URL: https://github.com/apache/hudi/pull/4520#discussion_r785674002 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/AbstractHoodieTableFileIndex.scala ## @@ -0,0 +1,309 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1014189252 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 Azure:

[jira] [Updated] (HUDI-3193) Move hudi-aws dependency out of hudi-client-common

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3193: - Component/s: (was: writer-core) > Move hudi-aws dependency out of hudi-client-common >

[jira] [Updated] (HUDI-3193) Move hudi-aws dependency out of hudi-client-common

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3193: - Component/s: dependencies > Move hudi-aws dependency out of hudi-client-common >

[GitHub] [hudi] hudi-bot commented on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1014182792 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1014181219 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot

[jira] [Updated] (HUDI-3161) Add Call Produce Command for spark sql

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3161: - Component/s: spark-sql (was: spark) > Add Call Produce Command for spark sql >

[hudi] branch master updated (d2dda55 -> ed92c21)

2022-01-16 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from d2dda55 [HUDI-2968] add UT for update/delete on non-pk condition (#4568) add ed92c21 [MINOR] Delete unused

[GitHub] [hudi] hudi-bot commented on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1014181219 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] yihua merged pull request #4595: [MINOR] Fix delete unused parameter in `TablePathUtils`

2022-01-16 Thread GitBox
yihua merged pull request #4595: URL: https://github.com/apache/hudi/pull/4595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4612: [MINOR] Add instructions to build and upload Docker Demo images

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4612: URL: https://github.com/apache/hudi/pull/4612#issuecomment-1013990142 ## CI report: * a3550a0e23444c04ddbe8d118d541390b5d4b190 Azure:

[GitHub] [hudi] xiarixiaoyao edited a comment on issue #4609: [SUPPORT] Got exception while using clustering with z-order

2022-01-16 Thread GitBox
xiarixiaoyao edited a comment on issue #4609: URL: https://github.com/apache/hudi/issues/4609#issuecomment-1014176723 @ravs11 sorry for that i cannot reproduce the exception, env: spark3.1.1 , hadoop 3.1.1, parquet 1.12.2 could you pls help me check me code: val path1 = new

[GitHub] [hudi] xiarixiaoyao commented on issue #4609: [SUPPORT] Got exception while using clustering with z-order

2022-01-16 Thread GitBox
xiarixiaoyao commented on issue #4609: URL: https://github.com/apache/hudi/issues/4609#issuecomment-1014176723 @ravs11 sorry for that i cannot reproduce the exception, pls help me check me code val path1 = new Path("/tmp/default/clustering/") // create parquet file

[jira] [Updated] (HUDI-3256) Add Links to Hudi Meetup Jan 2022

2022-01-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3256: - Labels: pull-request-available sev:normal (was: sev:normal) > Add Links to Hudi Meetup Jan 2022

[jira] [Created] (HUDI-3257) Excluding clustering instants from pending rollback info

2022-01-16 Thread Danny Chen (Jira)
Danny Chen created HUDI-3257: Summary: Excluding clustering instants from pending rollback info Key: HUDI-3257 URL: https://issues.apache.org/jira/browse/HUDI-3257 Project: Apache Hudi Issue

[GitHub] [hudi] n3nash opened a new pull request #4615: [HUDI-3256] Adding Video of Hudi Meetup on Jan11, 2022

2022-01-16 Thread GitBox
n3nash opened a new pull request #4615: URL: https://github.com/apache/hudi/pull/4615 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose

[jira] [Created] (HUDI-3256) Add Links to Hudi Meetup Jan 2022

2022-01-16 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-3256: - Summary: Add Links to Hudi Meetup Jan 2022 Key: HUDI-3256 URL: https://issues.apache.org/jira/browse/HUDI-3256 Project: Apache Hudi Issue Type: Task

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1014161030 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1014160029 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1014160029 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4523: [WIP][HUDI-3173] Add INDEX action type and corresponding commit metadata

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4523: URL: https://github.com/apache/hudi/pull/4523#issuecomment-1011089899 ## CI report: * b2d14ae38d1da89caa89129d28d2d3032aa13610 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4613: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4613: URL: https://github.com/apache/hudi/pull/4613#issuecomment-1014157034 ## CI report: * 9911aa35306964222fe4da383a6fcf47920aa40e Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4613: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4613: URL: https://github.com/apache/hudi/pull/4613#issuecomment-1014130867 ## CI report: * 9911aa35306964222fe4da383a6fcf47920aa40e Azure:

[GitHub] [hudi] xiaotianzhang01 commented on a change in pull request #4589: [MINOR] Fix the check condition in the `readFromVector` method to alway true

2022-01-16 Thread GitBox
xiaotianzhang01 commented on a change in pull request #4589: URL: https://github.com/apache/hudi/pull/4589#discussion_r785643080 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/AvroOrcUtils.java ## @@ -511,8 +511,8 @@ public static Object

[GitHub] [hudi] xiaotianzhang01 commented on a change in pull request #4589: [MINOR] Fix the check condition in the `readFromVector` method to alway true

2022-01-16 Thread GitBox
xiaotianzhang01 commented on a change in pull request #4589: URL: https://github.com/apache/hudi/pull/4589#discussion_r785643080 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/AvroOrcUtils.java ## @@ -511,8 +511,8 @@ public static Object

[GitHub] [hudi] esaeki commented on issue #4614: Encountered "Could not load Hoodie properties" error

2022-01-16 Thread GitBox
esaeki commented on issue #4614: URL: https://github.com/apache/hudi/issues/4614#issuecomment-1014142130 Same as https://github.com/apache/hudi/issues/2849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] esaeki closed issue #4614: Encountered "Could not load Hoodie properties" error

2022-01-16 Thread GitBox
esaeki closed issue #4614: URL: https://github.com/apache/hudi/issues/4614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] esaeki opened a new issue #4614: Encountered "Could not load Hoodie properties" error

2022-01-16 Thread GitBox
esaeki opened a new issue #4614: URL: https://github.com/apache/hudi/issues/4614 I use Glue and Apache Hudi to replicate data in AWS (S3 bucket→ another S3 bucket) I created a Glue job with jar files of Hudi, and successfully replicate the data. However, when I execute the job

[GitHub] [hudi] hudi-bot removed a comment on pull request #4613: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4613: URL: https://github.com/apache/hudi/pull/4613#issuecomment-1014129466 ## CI report: * 9911aa35306964222fe4da383a6fcf47920aa40e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot

[GitHub] [hudi] hudi-bot commented on pull request #4613: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4613: URL: https://github.com/apache/hudi/pull/4613#issuecomment-1014130867 ## CI report: * 9911aa35306964222fe4da383a6fcf47920aa40e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4613: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4613: URL: https://github.com/apache/hudi/pull/4613#issuecomment-1014129466 ## CI report: * 9911aa35306964222fe4da383a6fcf47920aa40e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] YannByron commented on pull request #3420: [HUDI-2283] Support Clustering Command For Spark Sql

2022-01-16 Thread GitBox
YannByron commented on pull request #3420: URL: https://github.com/apache/hudi/pull/3420#issuecomment-1014128455 @nsivabalan @yihua I cherry-pick @pengzhiwei2018 codes, and submit another pr: #4613 . -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] YannByron opened a new pull request #4613: Master 2283

2022-01-16 Thread GitBox
YannByron opened a new pull request #4613: URL: https://github.com/apache/hudi/pull/4613 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] danny0405 commented on a change in pull request #4589: [MINOR] Fix the check condition in the `readFromVector` method to alway true

2022-01-16 Thread GitBox
danny0405 commented on a change in pull request #4589: URL: https://github.com/apache/hudi/pull/4589#discussion_r785612106 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/AvroOrcUtils.java ## @@ -511,8 +511,8 @@ public static Object

[GitHub] [hudi] YannByron commented on a change in pull request #4565: [HUDI-3215] Solve UT for Spark 3.2

2022-01-16 Thread GitBox
YannByron commented on a change in pull request #4565: URL: https://github.com/apache/hudi/pull/4565#discussion_r785606065 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@ -729,6 +733,12 @@ object

[GitHub] [hudi] YannByron commented on a change in pull request #4565: [HUDI-3215] Solve UT for Spark 3.2

2022-01-16 Thread GitBox
YannByron commented on a change in pull request #4565: URL: https://github.com/apache/hudi/pull/4565#discussion_r785606065 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala ## @@ -729,6 +733,12 @@ object

[GitHub] [hudi] YannByron commented on a change in pull request #4565: [HUDI-3215] Solve UT for Spark 3.2

2022-01-16 Thread GitBox
YannByron commented on a change in pull request #4565: URL: https://github.com/apache/hudi/pull/4565#discussion_r785605100 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala ## @@ -118,6 +123,18 @@ class

[GitHub] [hudi] harishraju-govindaraju closed issue #4597: [SUPPORT] - Hudi Upserts Not working

2022-01-16 Thread GitBox
harishraju-govindaraju closed issue #4597: URL: https://github.com/apache/hudi/issues/4597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] harishraju-govindaraju commented on issue #4597: [SUPPORT] - Hudi Upserts Not working

2022-01-16 Thread GitBox
harishraju-govindaraju commented on issue #4597: URL: https://github.com/apache/hudi/issues/4597#issuecomment-1014088395 Folks, please mark this as resolved. Much appreciate your timely responses.. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] YannByron commented on a change in pull request #4565: [HUDI-3215] Solve UT for Spark 3.2

2022-01-16 Thread GitBox
YannByron commented on a change in pull request #4565: URL: https://github.com/apache/hudi/pull/4565#discussion_r785604585 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala ## @@ -54,15 +54,20 @@ class

[jira] [Comment Edited] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476923#comment-17476923 ] Harsha Teja Kanna edited comment on HUDI-3242 at 1/17/22, 2:30 AM: --- I am

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4610: [HUDI-3251] Add HoodieFlinkSource for flink datastream api

2022-01-16 Thread GitBox
XuQianJin-Stars commented on pull request #4610: URL: https://github.com/apache/hudi/pull/4610#issuecomment-1014082593 hi @danny0405 Have time to help me review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] YannByron commented on pull request #4180: [HUDI-2903] get table schema from the last commit with data written

2022-01-16 Thread GitBox
YannByron commented on pull request #4180: URL: https://github.com/apache/hudi/pull/4180#issuecomment-1014082336 @nsivabalan add some UT for this. have time to review this again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1014051032 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9743a6f98e62888c5e9cb575dd8cf0d38ade0319 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1014076298 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 519a5b123a7315948cf83dfd2e13ee275abcbd39 Azure:

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476923#comment-17476923 ] Harsha Teja Kanna commented on HUDI-3242: - I am now seeing this for all the tables.. after running

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Priority: Blocker (was: Critical) > Checkpoint 0 is ignored -Partial parquet file

[jira] [Commented] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-01-16 Thread taisenki (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476920#comment-17476920 ] taisenki commented on HUDI-3204: [~shivnarayan] using query with _hoodie_partition_path : {code:java} //

[jira] [Assigned] (HUDI-3245) Convert uppercase letters to lowercase in storage configs

2022-01-16 Thread cdmikechen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cdmikechen reassigned HUDI-3245: Assignee: cdmikechen > Convert uppercase letters to lowercase in storage configs >

[GitHub] [hudi] cdmikechen commented on pull request #4602: [HUDI-3245] Convert uppercase letters to lowercase in storage configs

2022-01-16 Thread GitBox
cdmikechen commented on pull request #4602: URL: https://github.com/apache/hudi/pull/4602#issuecomment-1014055807 @nsivabalan Hi~ This PR may be a required patch for versions 0.10.1 and later. Can you contact the relevant committer/PMC for help, please? In addition, there is a

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-16 Thread GitBox
hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1014051032 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9743a6f98e62888c5e9cb575dd8cf0d38ade0319 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

2022-01-16 Thread GitBox
hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1013988204 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * 9743a6f98e62888c5e9cb575dd8cf0d38ade0319 Azure:

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): hudi-bot edited a comment on pull request #3071: URL: https://github.com/apache/hudi/pull/3071#issuecomment-869222972

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): vinothchandar commented on pull request #3071: URL: https://github.com/apache/hudi/pull/3071#issuecomment-896337587 Have

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): n3nash commented on a change in pull request #3071: URL: https://github.com/apache/hudi/pull/3071#discussion_r662042993

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): veenaypatil commented on a change in pull request #3071: URL: https://github.com/apache/hudi/pull/3071#discussion_r662055532

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): hudi-bot edited a comment on pull request #3071: URL: https://github.com/apache/hudi/pull/3071#issuecomment-869222972

[jira] (HUDI-1976) Upgrade hive, jackson, log4j, hadoop to remove vulnerability

2022-01-16 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1976 ] Raymond Xu deleted comment on HUDI-1976: -- was (Author: githubbot): veenaypatil commented on a change in pull request #3071: URL: https://github.com/apache/hudi/pull/3071#discussion_r679915797

[jira] [Updated] (HUDI-2546) Validate Full Schema Evolution in Hudi including Rename

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2546: -- Component/s: reader-core writer-core > Validate Full Schema Evolution

[jira] [Updated] (HUDI-2546) Validate Full Schema Evolution in Hudi including Rename

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2546: -- Priority: Major (was: Critical) > Validate Full Schema Evolution in Hudi including

[jira] [Commented] (HUDI-2546) Validate Full Schema Evolution in Hudi including Rename

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476914#comment-17476914 ] sivabalan narayanan commented on HUDI-2546: --- [~codope] :Can we close out this task?  > Validate

[jira] [Updated] (HUDI-2768) Enable async timeline server by default

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2768: -- Component/s: timeline-server > Enable async timeline server by default >

[jira] [Updated] (HUDI-2768) Enable async timeline server by default

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2768: -- Component/s: writer-core > Enable async timeline server by default >

[jira] [Updated] (HUDI-2893) Metadata table not found warn logs repeated

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2893: -- Component/s: metadata > Metadata table not found warn logs repeated >

[jira] [Updated] (HUDI-2893) Metadata table not found warn logs repeated

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2893: -- Priority: Major (was: Critical) > Metadata table not found warn logs repeated >

[jira] [Updated] (HUDI-2768) Enable async timeline server by default

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2768: -- Priority: Major (was: Critical) > Enable async timeline server by default >

[jira] [Updated] (HUDI-2867) Make HoodiePartitionPath optional

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2867: -- Component/s: writer-core > Make HoodiePartitionPath optional >

[jira] [Updated] (HUDI-2836) Added kerberos environment to test Hudi integration kerberos

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2836: -- Priority: Major (was: Critical) > Added kerberos environment to test Hudi integration

[jira] [Updated] (HUDI-2867) Make HoodiePartitionPath optional

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2867: -- Priority: Major (was: Critical) > Make HoodiePartitionPath optional >

[jira] [Updated] (HUDI-2296) flink support ConsistencyGuard plugin

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2296: -- Priority: Major (was: Critical) > flink support ConsistencyGuard plugin >

[jira] [Updated] (HUDI-2940) Support sync database and table created by Flink catalog to hive

2022-01-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2940: -- Priority: Major (was: Critical) > Support sync database and table created by Flink

  1   2   3   4   >