[jira] [Updated] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3107: - Component/s: Hive Integration > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Assigned] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3107: Assignee: Yue Zhang > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Updated] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3107: - Fix Version/s: 0.11.0 0.10.1 > Fix HiveSyncTool drop partitions using JDBC >

[GitHub] [hudi] dongkelun commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
dongkelun commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002442373 > @dongkelun @xushiyan I offer another solution to discuss. > > Query incrementally in hive need to set `hoodie.%s.consume.start.timestamp` which is used in

[GitHub] [hudi] RocMarshal commented on pull request #3813: [HUDI-2563][hudi-client] Refactor CompactionTriggerStrategy.

2021-12-28 Thread GitBox
RocMarshal commented on pull request #3813: URL: https://github.com/apache/hudi/pull/3813#issuecomment-1002441676 > I'm not in favor of fat Enum either. But would like to understand the main benefit of this change: is it meant for portability of these logic? @RocMarshal Thanks

[jira] [Closed] (HUDI-3093) spark-sql query error when use TimestampBasedKeyGenerator

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3093. Reviewers: Raymond Xu Resolution: Fixed > spark-sql query error when use TimestampBasedKeyGenerator >

[jira] [Updated] (HUDI-3093) spark-sql query error when use TimestampBasedKeyGenerator

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3093: - Reporter: Raymond Xu (was: Yann Byron) > spark-sql query error when use TimestampBasedKeyGenerator >

[jira] [Updated] (HUDI-3093) spark-sql query error when use TimestampBasedKeyGenerator

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3093: - Priority: Critical (was: Major) > spark-sql query error when use TimestampBasedKeyGenerator >

[jira] [Updated] (HUDI-2990) Sync to HMS when deleting partitions

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2990: - Component/s: Hive Integration > Sync to HMS when deleting partitions >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Priority: Critical (was: Major) > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Reporter: Raymond Xu (was: Forward Xu) > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2837) The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2837: - Fix Version/s: 0.11.0 Reviewers: Raymond Xu > The original hoodie.table.name should be maintained

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002428182 ## CI report: * 2e5ad082fa641bd060c7b8b25a23ef042c240460 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002409677 ## CI report: * 0c1a86fb69261aef8f6bd7f017a04ce087b2fc98 Azure:

[jira] [Closed] (HUDI-2986) Deltastreamer continuous mode run into Too many open files exception

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2986. Fix Version/s: (was: 0.11.0) (was: 0.10.1) Resolution: Won't Fix not

[jira] [Closed] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2989. Fix Version/s: (was: 0.11.0) (was: 0.10.1) Resolution: Won't Fix > Hive

[jira] [Updated] (HUDI-2987) event time not recorded in commit metadata when insert or bulk insert

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2987: - Priority: Critical (was: Blocker) > event time not recorded in commit metadata when insert or bulk

[jira] [Updated] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2989: - Status: Resolved (was: Patch Available) > Hive sync to Glue tables not updating S3 location >

[jira] [Reopened] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reopened HUDI-2989: -- > Hive sync to Glue tables not updating S3 location > - > >

[jira] [Updated] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2989: - Priority: Critical (was: Blocker) > Hive sync to Glue tables not updating S3 location >

[GitHub] [hudi] hudi-bot removed a comment on pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4467: URL: https://github.com/apache/hudi/pull/4467#issuecomment-1002402985 ## CI report: * 200dc06debc347edec4496d5b09fb7942cdec1a3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4467: URL: https://github.com/apache/hudi/pull/4467#issuecomment-1002422707 ## CI report: * 200dc06debc347edec4496d5b09fb7942cdec1a3 Azure:

[GitHub] [hudi] bhasudha commented on issue #2529: [SUPPORT] - Hudi Jar update in EMR

2021-12-28 Thread GitBox
bhasudha commented on issue #2529: URL: https://github.com/apache/hudi/issues/2529#issuecomment-1002415806 Should be in FAQ already - https://hudi.apache.org/learn/faq/#how-to-override-hudi-jars-in-emr -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] bhasudha closed issue #2529: [SUPPORT] - Hudi Jar update in EMR

2021-12-28 Thread GitBox
bhasudha closed issue #2529: URL: https://github.com/apache/hudi/issues/2529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002415715 ## CI report: * 3904a789ff694a3b4ef0bc015e73f840e150a797 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002399306 ## CI report: * 0ec7317ac54ffcfe925206deeb0f4866dff1f298 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1002412658 ## CI report: * d9182c1661e37f29622caafd9eaa23de73b26331 UNKNOWN * 270eee7ef88fc59339675b1443b8918e63015fed Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1002401796 ## CI report: * d9182c1661e37f29622caafd9eaa23de73b26331 UNKNOWN * 270eee7ef88fc59339675b1443b8918e63015fed Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002409677 ## CI report: * 0c1a86fb69261aef8f6bd7f017a04ce087b2fc98 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002409018 ## CI report: * 0c1a86fb69261aef8f6bd7f017a04ce087b2fc98 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002409018 ## CI report: * 0c1a86fb69261aef8f6bd7f017a04ce087b2fc98 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002165902 ## CI report: * 0c1a86fb69261aef8f6bd7f017a04ce087b2fc98 Azure:

[GitHub] [hudi] cdmikechen commented on a change in pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
cdmikechen commented on a change in pull request #3391: URL: https://github.com/apache/hudi/pull/3391#discussion_r776160563 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/avro/HudiAvroParquetReader.java ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] cdmikechen commented on a change in pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
cdmikechen commented on a change in pull request #3391: URL: https://github.com/apache/hudi/pull/3391#discussion_r776160563 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/avro/HudiAvroParquetReader.java ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] hudi-bot commented on pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4467: URL: https://github.com/apache/hudi/pull/4467#issuecomment-1002402985 ## CI report: * 200dc06debc347edec4496d5b09fb7942cdec1a3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4467: URL: https://github.com/apache/hudi/pull/4467#issuecomment-1002402429 ## CI report: * 200dc06debc347edec4496d5b09fb7942cdec1a3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot

[GitHub] [hudi] hudi-bot commented on pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4467: URL: https://github.com/apache/hudi/pull/4467#issuecomment-1002402429 ## CI report: * 200dc06debc347edec4496d5b09fb7942cdec1a3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[jira] [Updated] (HUDI-3124) Bootstrap when timeline have completed instant

2021-12-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3124: - Labels: pull-request-available (was: ) > Bootstrap when timeline have completed instant >

[GitHub] [hudi] hudi-bot commented on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1002401796 ## CI report: * d9182c1661e37f29622caafd9eaa23de73b26331 UNKNOWN * 270eee7ef88fc59339675b1443b8918e63015fed Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1002058975 ## CI report: * d9182c1661e37f29622caafd9eaa23de73b26331 UNKNOWN * 270eee7ef88fc59339675b1443b8918e63015fed Azure:

[GitHub] [hudi] zhangyue19921010 commented on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
zhangyue19921010 commented on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1002401764 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] zhangyue19921010 removed a comment on pull request #4459: [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job.

2021-12-28 Thread GitBox
zhangyue19921010 removed a comment on pull request #4459: URL: https://github.com/apache/hudi/pull/4459#issuecomment-1001991875 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] yuzhaojing opened a new pull request #4467: [HUDI-3124] Bootstrap when timeline have completed instant

2021-12-28 Thread GitBox
yuzhaojing opened a new pull request #4467: URL: https://github.com/apache/hudi/pull/4467 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] hudi-bot commented on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot commented on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002400317 ## CI report: * ecb72b89015831cfbfa99ebcb027f660729b3195 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002388626 ## CI report: * e19068fd9ef591062e9ae920f3e2fe74f1eabfe3 Azure:

[jira] [Created] (HUDI-3124) Bootstrap when timeline have completed instant

2021-12-28 Thread yuzhaojing (Jira)
yuzhaojing created HUDI-3124: Summary: Bootstrap when timeline have completed instant Key: HUDI-3124 URL: https://issues.apache.org/jira/browse/HUDI-3124 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] hudi-bot commented on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002399306 ## CI report: * 0ec7317ac54ffcfe925206deeb0f4866dff1f298 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002398601 ## CI report: * 0ec7317ac54ffcfe925206deeb0f4866dff1f298 Azure:

[GitHub] [hudi] cdmikechen commented on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
cdmikechen commented on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002398757 > I have a concern around performance overhead and also wondering if we can just do it as a part of the existing inputformat with a flag, instead of switching over entirely to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002101189 ## CI report: * 0ec7317ac54ffcfe925206deeb0f4866dff1f298 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4463: [HUDI-3120] Cache compactionPlan in buffer

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4463: URL: https://github.com/apache/hudi/pull/4463#issuecomment-1002398601 ## CI report: * 0ec7317ac54ffcfe925206deeb0f4866dff1f298 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002391692 ## CI report: * 15a6d4ea2eaae3e5b8fe5e174127016ea72b0e05 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002379085 ## CI report: * 1f8244a3e0db6f82af5e8d45c8045c8b759309ba Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot commented on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002388626 ## CI report: * e19068fd9ef591062e9ae920f3e2fe74f1eabfe3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002387959 ## CI report: * e19068fd9ef591062e9ae920f3e2fe74f1eabfe3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot commented on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-1002387959 ## CI report: * e19068fd9ef591062e9ae920f3e2fe74f1eabfe3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3391: [HUDI-83] Fix Timestamp type read by Hive

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #3391: URL: https://github.com/apache/hudi/pull/3391#issuecomment-961588069 ## CI report: * e19068fd9ef591062e9ae920f3e2fe74f1eabfe3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002371458 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002385852 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN * 32ec46f289e4ecf4c1e66241ea954a9f3b34e9a6

[GitHub] [hudi] stym06 commented on issue #4318: [SUPPORT] Duplicate records in COW table within same partition path

2021-12-28 Thread GitBox
stym06 commented on issue #4318: URL: https://github.com/apache/hudi/issues/4318#issuecomment-1002380917 yes, to access s3 data from the local -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] YannByron commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread GitBox
YannByron commented on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1002380413 @dongkelun @xushiyan I offer another solution to discuss. Query incrementally in hive need to set `hoodie.%s.consume.start.timestamp` which is used in

[jira] [Updated] (HUDI-2590) Validate Diff key gen w/ and w/o glob path with and w/o metadata enabled

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2590: -- Status: Open (was: In Progress) > Validate Diff key gen w/ and w/o glob path with and

[jira] [Updated] (HUDI-2947) HoodieDeltaStreamer/DeltaSync can improperly pick up the checkpoint config from CLI in continuous mode

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2947: -- Labels: sev:high (was: ) > HoodieDeltaStreamer/DeltaSync can improperly pick up the

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3066: -- Status: Open (was: In Progress) > Very slow file listing after enabling metadata for

[jira] [Updated] (HUDI-3057) Instants should be generated strictly under locks

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3057: -- Labels: sev:high (was: ) > Instants should be generated strictly under locks >

[jira] [Updated] (HUDI-2947) HoodieDeltaStreamer/DeltaSync can improperly pick up the checkpoint config from CLI in continuous mode

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2947: -- Priority: Critical (was: Blocker) > HoodieDeltaStreamer/DeltaSync can improperly pick

[jira] [Updated] (HUDI-3057) Instants should be generated strictly under locks

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3057: -- Priority: Critical (was: Blocker) > Instants should be generated strictly under locks

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002368726 ## CI report: * 1f8244a3e0db6f82af5e8d45c8045c8b759309ba Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002379085 ## CI report: * 1f8244a3e0db6f82af5e8d45c8045c8b759309ba Azure:

[GitHub] [hudi] putaozhi123 opened a new issue #4466: [SUPPORT]ERROR table.HoodieTimelineArchiveLog: Failed to archive commits,Not an Avro data file

2021-12-28 Thread GitBox
putaozhi123 opened a new issue #4466: URL: https://github.com/apache/hudi/issues/4466 **Environment Description** Hudi version : 0.8.0 Spark version : 2.4.7 Storage (HDFS/S3/GCS..) : HDFS Running on Docker? (yes/no) : no **Additional context** the hudi

[GitHub] [hudi] nikenfls commented on issue #4461: [SUPPORT]Hudi(0.10.0) write to Aliyun oss using metadata table warning

2021-12-28 Thread GitBox
nikenfls commented on issue #4461: URL: https://github.com/apache/hudi/issues/4461#issuecomment-1002372820 > Thank you very much for your time. I will try these sdk. XD -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (HUDI-3110) parquet max file size not honored

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466293#comment-17466293 ] sivabalan narayanan commented on HUDI-3110: --- setting parquet block size fixed the issue.  >

[jira] [Closed] (HUDI-3110) parquet max file size not honored

2021-12-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3110. - Resolution: Invalid > parquet max file size not honored >

[GitHub] [hudi] hudi-bot commented on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002371458 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN * 32ec46f289e4ecf4c1e66241ea954a9f3b34e9a6

[GitHub] [hudi] hudi-bot removed a comment on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002365199 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN *

[GitHub] [hudi] vingov commented on issue #4429: [SUPPORT] Spark SQL CTAS command doesn't work with 0.10.0 version and Spark 3.1.1

2021-12-28 Thread GitBox
vingov commented on issue #4429: URL: https://github.com/apache/hudi/issues/4429#issuecomment-1002370883 Thanks, @xushiyan, I've updated the title to reflect the CTAS issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] YannByron edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-28 Thread GitBox
YannByron edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1002370436 @nsivabalan @BenjMaq I use the basically same commands, in Hudi0.9 + Spark2.4.4. ``` CREATE TABLE IF NOT EXISTS test_overwrite ( id bigint, name

[GitHub] [hudi] YannByron edited a comment on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-28 Thread GitBox
YannByron edited a comment on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1002370436 @nsivabalan @BenjMaq I use the basically same commands, in Hudi0.9 + Spark2.4.4. ```CREATE TABLE IF NOT EXISTS test_overwrite ( id bigint, name string,

[GitHub] [hudi] YannByron commented on issue #4154: [SUPPORT] INSERT OVERWRITE operation does not work when using Spark SQL

2021-12-28 Thread GitBox
YannByron commented on issue #4154: URL: https://github.com/apache/hudi/issues/4154#issuecomment-1002370436 @nsivabalan @BenjMaq I use the basically same commands, in Hudi0.9 + Spark2.4.4. `CREATE TABLE IF NOT EXISTS test_overwrite ( id bigint, name string,

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002368196 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002368726 ## CI report: * 1f8244a3e0db6f82af5e8d45c8045c8b759309ba Azure:

[GitHub] [hudi] dongkelun commented on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
dongkelun commented on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002368631 > @dongkelun : can you rebase with latest master. @nsivabalan Hello,I have rebased with latest master. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002365326 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002368196 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] YuweiXiao commented on issue #4461: [SUPPORT]Hudi(0.10.0) write to Aliyun oss using metadata table warning

2021-12-28 Thread GitBox
YuweiXiao commented on issue #4461: URL: https://github.com/apache/hudi/issues/4461#issuecomment-1002367507 I cannot reproduce the warning. I am using master branch and run directly inside the IDE (with spark local mode and core-site.xml setup) by the way, I am using the following

[GitHub] [hudi] lamberken commented on a change in pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
lamberken commented on a change in pull request #4455: URL: https://github.com/apache/hudi/pull/4455#discussion_r776130859 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/DropHoodieTableCommand.scala ## @@ -85,25 +91,42 @@ case

[jira] [Updated] (HUDI-3123) Consistent hashing index for upsert/insert write path

2021-12-28 Thread Yuwei Xiao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuwei Xiao updated HUDI-3123: - Parent: HUDI-3000 Issue Type: Sub-task (was: Improvement) > Consistent hashing index for

[jira] [Created] (HUDI-3123) Consistent hashing index for upsert/insert write path

2021-12-28 Thread Yuwei Xiao (Jira)
Yuwei Xiao created HUDI-3123: Summary: Consistent hashing index for upsert/insert write path Key: HUDI-3123 URL: https://issues.apache.org/jira/browse/HUDI-3123 Project: Apache Hudi Issue Type:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002364571 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002365326 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1002365199 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN * 32ec46f289e4ecf4c1e66241ea954a9f3b34e9a6

[GitHub] [hudi] hudi-bot removed a comment on pull request #4016: [HUDI-2675] Fix the exception 'Not an Avro data file' when archive and clean

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4016: URL: https://github.com/apache/hudi/pull/4016#issuecomment-1001876380 ## CI report: * aec4dde1fb90319de9cf0c6f34771c0f193ccfd9 UNKNOWN * f426cb3cc3513d1baf26a70fdcb18114ffe5ddc5 UNKNOWN *

[GitHub] [hudi] hudi-bot removed a comment on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot removed a comment on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002005192 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
hudi-bot commented on pull request #4455: URL: https://github.com/apache/hudi/pull/4455#issuecomment-1002364571 ## CI report: * 5cd1675199d0dd65733982cca2132c03b5bf9d6c Azure:

[jira] [Updated] (HUDI-3108) Fix Purge Drop MOR Table Cause error

2021-12-28 Thread Forward Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated HUDI-3108: - Attachment: image-2021-12-29-10-04-31-025.png image-2021-12-29-09-52-30-999.png

[GitHub] [hudi] minihippo commented on pull request #3173: [HUDI-1951] Add bucket hash index, compatible with the hive bucket

2021-12-28 Thread GitBox
minihippo commented on pull request #3173: URL: https://github.com/apache/hudi/pull/3173#issuecomment-1002355744 @vinothchandar I addressed all comments and the failure ut is not related with this pr. Can we land this? -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] lamberken commented on a change in pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
lamberken commented on a change in pull request #4455: URL: https://github.com/apache/hudi/pull/4455#discussion_r776120977 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/DropHoodieTableCommand.scala ## @@ -38,6 +38,9 @@ case

[GitHub] [hudi] lamberken commented on a change in pull request #4455: [HUDI-3108] Fix Purge Drop MOR Table Cause error

2021-12-28 Thread GitBox
lamberken commented on a change in pull request #4455: URL: https://github.com/apache/hudi/pull/4455#discussion_r776120977 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/DropHoodieTableCommand.scala ## @@ -38,6 +38,9 @@ case

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Wenning Ding (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466279#comment-17466279 ] Wenning Ding commented on HUDI-3122: Thanks I will give a shot > presto query failed for bootstrap

[jira] [Commented] (HUDI-3122) presto query failed for bootstrap tables

2021-12-28 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466280#comment-17466280 ] Yue Zhang commented on HUDI-3122: - I see it in hudi-presto-bundle.jar but i am not sure if it solve your

  1   2   3   4   >