[jira] [Updated] (HUDI-3053) [UMBRELLA] Tracker for all CI flaky test failures dec2021

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3053: - Labels: hudi-umbrellas (was: ) > [UMBRELLA] Tracker for all CI flaky test failures dec2021 >

[jira] [Updated] (HUDI-3052) Flaky TestJsonKafkaSource in CI runs

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3052: - Epic Link: HUDI-3053 > Flaky TestJsonKafkaSource in CI runs > - > >

[jira] [Updated] (HUDI-2261) [UMBRELLA] Dev Hygiene Issues

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2261: - Issue Type: Epic (was: Improvement) > [UMBRELLA] Dev Hygiene Issues > - > >

[jira] [Updated] (HUDI-3053) [UMBRELLA] Tracker for all CI flaky test failures dec2021

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3053: - Epic Name: CI Flakiness 2021-12 > [UMBRELLA] Tracker for all CI flaky test failures dec2021 >

[jira] [Updated] (HUDI-3053) [UMBRELLA] Tracker for all CI flaky test failures dec2021

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3053: - Fix Version/s: 0.10.1 > [UMBRELLA] Tracker for all CI flaky test failures dec2021 >

[jira] [Updated] (HUDI-3043) Test failure with TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWriters

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3043: - Epic Link: HUDI-3053 > Test failure with >

[jira] [Reopened] (HUDI-3043) Test failure with TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWriters

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reopened HUDI-3043: -- > Test failure with > TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWriters

[jira] [Closed] (HUDI-3043) Test failure with TestHoodieDeltaStreamerWithMultiWriter.testUpsertsContinuousModeWithMultipleWriters

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3043. Resolution: Fixed > Test failure with >

[jira] [Reopened] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reopened HUDI-3054: -- > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter >

[jira] [Closed] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3054. Resolution: Fixed > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter >

[jira] [Updated] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3054: - Epic Link: HUDI-3053 > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter >

[jira] [Updated] (HUDI-3052) Flaky TestJsonKafkaSource in CI runs

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3052: - Labels: hudi-subtask pull-request-available sev:critical (was: pull-request-available sev:critical) >

[jira] [Updated] (HUDI-2324) [UMBRELLA] Implement Hudi Transaction writes for Kafka Connect platform

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2324: - Issue Type: Epic (was: New Feature) > [UMBRELLA] Implement Hudi Transaction writes for Kafka Connect

[jira] [Updated] (HUDI-1292) [Umbrella] RFC-15 : File Listing and Query Planning Optimizations

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1292: - Fix Version/s: 0.11.0 > [Umbrella] RFC-15 : File Listing and Query Planning Optimizations >

[jira] [Updated] (HUDI-3053) [UMBRELLA] Tracker for all CI flaky test failures dec2021

2022-01-02 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3053: - Issue Type: Epic (was: Bug) > [UMBRELLA] Tracker for all CI flaky test failures dec2021 >

[jira] [Assigned] (HUDI-3162) Shade AWS dependencies for bundled packages

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3162: Assignee: Hui An > Shade AWS dependencies for bundled packages >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Status: In Progress (was: Open) > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Status: In Progress (was: Open) > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Status: Patch Available (was: In Progress) > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Status: Open (was: Patch Available) > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3157) Shade aws-dependencies to avoid class conflicts

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3157: - Sprint: Hudi 0.10.1 - 2021/01/03 > Shade aws-dependencies to avoid class conflicts >

[jira] [Updated] (HUDI-3158) Reduce warn logs in Spark SQL INSERT OVERWRITE

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3158: - Sprint: Hudi 0.10.1 - 2021/01/03 > Reduce warn logs in Spark SQL INSERT OVERWRITE >

[jira] [Assigned] (HUDI-3163) Validate/certify hudi against diff spark 3 versions

2022-01-04 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3163: Assignee: Raymond Xu (was: Raymond Xu) > Validate/certify hudi against diff spark 3 versions >

[jira] [Updated] (HUDI-2516) Upgrade to Junit 5.8.1

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2516: - Parent: (was: HUDI-1248) Issue Type: Improvement (was: Sub-task) > Upgrade to Junit 5.8.1 >

[jira] [Commented] (HUDI-1387) [UMBRELLA] Support Apache Calcite for writing/querying Hudi datasets

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463398#comment-17463398 ] Raymond Xu commented on HUDI-1387: -- [~x1q1j1] can you share some thoughts on this pls?  > [UMBRELLA]

[jira] [Updated] (HUDI-3060) DROP TABLE for spark sql

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3060: - Fix Version/s: 0.11.0 > DROP TABLE for spark sql > > > Key:

[jira] [Updated] (HUDI-2991) Add rename partition for spark sql

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2991: - Fix Version/s: 0.11.0 > Add rename partition for spark sql > -- > >

[jira] [Updated] (HUDI-2992) Add add partition for spark sql

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2992: - Fix Version/s: 0.11.0 > Add add partition for spark sql > --- > >

[jira] [Created] (HUDI-3088) Make Spark 3 the default profile for build and test

2021-12-21 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-3088: Summary: Make Spark 3 the default profile for build and test Key: HUDI-3088 URL: https://issues.apache.org/jira/browse/HUDI-3088 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-2990) Sync to HMS when deleting partitions

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2990: - Fix Version/s: 0.11.0 0.10.1 > Sync to HMS when deleting partitions >

[jira] [Updated] (HUDI-2939) [DOC] Add description of write commit callback by pulsar to document

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2939: - Fix Version/s: 0.11.0 > [DOC] Add description of write commit callback by pulsar to document >

[jira] [Updated] (HUDI-2937) Introduce a pulsar implementation of hoodie write commit callback

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2937: - Fix Version/s: 0.11.0 > Introduce a pulsar implementation of hoodie write commit callback >

[jira] [Updated] (HUDI-3022) Fix npe for isDropPartition method

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3022: - Fix Version/s: 0.11.0 0.10.1 > Fix npe for isDropPartition method >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-21 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Fix Version/s: 0.11.0 0.10.1 > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2021-12-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1850: - Component/s: Spark Integration > Read on table fails if the first write to table failed >

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2021-12-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1850: - Fix Version/s: 0.10.1 > Read on table fails if the first write to table failed >

[jira] [Updated] (HUDI-2465) Fix merge, update for spark sql dml support to test suite infra

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2465: - Fix Version/s: 0.10.1 > Fix merge, update for spark sql dml support to test suite infra >

[jira] [Updated] (HUDI-2661) java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.catalog.CatalogTable.copy

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2661: - Sprint: Hudi-Sprint-0.10.1 > java.lang.NoSuchMethodError: >

[jira] [Updated] (HUDI-281) HiveSync failure through Spark when useJdbc is set to false

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-281: Fix Version/s: 0.10.1 > HiveSync failure through Spark when useJdbc is set to false >

[jira] [Updated] (HUDI-281) HiveSync failure through Spark when useJdbc is set to false

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-281: Sprint: Hudi-Sprint-0.10.1 > HiveSync failure through Spark when useJdbc is set to false >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Sprint: Hudi-Sprint-0.10.1 > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3100) Hive Conditional sync cannot be set from deltastreamer

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3100: - Sprint: Hudi-Sprint-0.10.1 > Hive Conditional sync cannot be set from deltastreamer >

[jira] [Updated] (HUDI-2966) Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2966: - Sprint: Hudi-Sprint-0.10.1 > Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner >

[jira] [Updated] (HUDI-2837) The original hoodie.table.name should be maintained in Spark SQL

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2837: - Sprint: (was: Hudi-Sprint-0.10.1) > The original hoodie.table.name should be maintained in Spark SQL >

[jira] [Updated] (HUDI-2837) The original hoodie.table.name should be maintained in Spark SQL

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2837: - Sprint: Hudi-Sprint-0.10.1 > The original hoodie.table.name should be maintained in Spark SQL >

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1850: - Sprint: Hudi-Sprint-0.10.1 > Read on table fails if the first write to table failed >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Sprint: Hudi-Sprint-0.10.1 > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2611) `create table if not exists` should print message instead of throwing error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2611: - Sprint: Hudi-Sprint-0.10.1 > `create table if not exists` should print message instead of throwing error

[jira] [Updated] (HUDI-2426) spark sql extensions breaks read.table from metastore

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2426: - Sprint: Hudi-Sprint-0.10.1 > spark sql extensions breaks read.table from metastore >

[jira] [Updated] (HUDI-2837) The original hoodie.table.name should be maintained in Spark SQL

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2837: - Fix Version/s: (was: 0.10.1) > The original hoodie.table.name should be maintained in Spark SQL >

[jira] [Updated] (HUDI-2990) Sync to HMS when deleting partitions

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2990: - Component/s: Hive Integration > Sync to HMS when deleting partitions >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Reporter: Raymond Xu (was: Forward Xu) > Fix field not found in record error for spark-sql >

[jira] [Closed] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2989. Fix Version/s: (was: 0.11.0) (was: 0.10.1) Resolution: Won't Fix > Hive

[jira] [Closed] (HUDI-2986) Deltastreamer continuous mode run into Too many open files exception

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2986. Fix Version/s: (was: 0.11.0) (was: 0.10.1) Resolution: Won't Fix not

[jira] [Updated] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2989: - Priority: Critical (was: Blocker) > Hive sync to Glue tables not updating S3 location >

[jira] [Updated] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2989: - Status: Resolved (was: Patch Available) > Hive sync to Glue tables not updating S3 location >

[jira] [Reopened] (HUDI-2989) Hive sync to Glue tables not updating S3 location

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reopened HUDI-2989: -- > Hive sync to Glue tables not updating S3 location > - > >

[jira] [Updated] (HUDI-2987) event time not recorded in commit metadata when insert or bulk insert

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2987: - Priority: Critical (was: Blocker) > event time not recorded in commit metadata when insert or bulk

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Priority: Critical (was: Major) > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2837) The original hoodie.table.name should be maintained in Spark SQL

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2837: - Fix Version/s: 0.11.0 Reviewers: Raymond Xu > The original hoodie.table.name should be maintained

[jira] [Closed] (HUDI-3093) spark-sql query error when use TimestampBasedKeyGenerator

2021-12-28 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3093. Reviewers: Raymond Xu Resolution: Fixed > spark-sql query error when use TimestampBasedKeyGenerator >

[jira] [Commented] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466686#comment-17466686 ] Raymond Xu commented on HUDI-2323: -- This should be resolved by having parquet 1.12, which will be the

[jira] [Commented] (HUDI-1079) Cannot upsert on schema with Array of Record with single field

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466687#comment-17466687 ] Raymond Xu commented on HUDI-1079: -- This should be resolved by having parquet 1.12, which will be the

[jira] [Updated] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3107: - Reviewers: Raymond Xu > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Updated] (HUDI-2426) spark sql extensions breaks read.table from metastore

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2426: - Reviewers: Raymond Xu > spark sql extensions breaks read.table from metastore >

[jira] [Updated] (HUDI-2611) `create table if not exists` should print message instead of throwing error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2611: - Reviewers: Raymond Xu > `create table if not exists` should print message instead of throwing error >

[jira] [Updated] (HUDI-2990) Sync to HMS when deleting partitions

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2990: - Reviewers: Raymond Xu > Sync to HMS when deleting partitions > > >

[jira] [Updated] (HUDI-3106) Fix HiveSyncTool not sync schema

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3106: - Reviewers: Raymond Xu > Fix HiveSyncTool not sync schema > > >

[jira] [Assigned] (HUDI-3104) Hudi-kafka-connect can not scan hadoop config files by HADOOP_CONF_DIR

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3104: Assignee: cdmikechen > Hudi-kafka-connect can not scan hadoop config files by HADOOP_CONF_DIR >

[jira] [Updated] (HUDI-3104) Hudi-kafka-connect can not scan hadoop config files by HADOOP_CONF_DIR

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3104: - Status: In Progress (was: Open) > Hudi-kafka-connect can not scan hadoop config files by HADOOP_CONF_DIR

[jira] [Updated] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3107: - Story Points: 1 > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Updated] (HUDI-281) HiveSync failure through Spark when useJdbc is set to false

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-281: Story Points: 1 > HiveSync failure through Spark when useJdbc is set to false >

[jira] [Updated] (HUDI-2938) Code Refactor: Metadata util to get latest file slices for readers and writers

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2938: - Issue Type: Improvement (was: Task) > Code Refactor: Metadata util to get latest file slices for readers

[jira] [Closed] (HUDI-2611) `create table if not exists` should print message instead of throwing error

2021-12-30 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-2611. Resolution: Not A Bug > `create table if not exists` should print message instead of throwing error >

[jira] [Assigned] (HUDI-3131) Spark3.1.1 CTAS error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3131: Assignee: Yann Byron > Spark3.1.1 CTAS error > - > > Key:

[jira] [Updated] (HUDI-2987) event time not recorded in commit metadata when insert or bulk insert

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2987: - Fix Version/s: (was: 0.10.1) > event time not recorded in commit metadata when insert or bulk insert

[jira] [Updated] (HUDI-3100) Hive Conditional sync cannot be set from deltastreamer

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3100: - Status: In Progress (was: Open) > Hive Conditional sync cannot be set from deltastreamer >

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1850: - Story Points: 1 > Read on table fails if the first write to table failed >

[jira] [Updated] (HUDI-3100) Hive Conditional sync cannot be set from deltastreamer

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3100: - Story Points: 1 > Hive Conditional sync cannot be set from deltastreamer >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Story Points: 2 > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2966) Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2966: - Story Points: 1 > Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner >

[jira] [Assigned] (HUDI-281) HiveSync failure through Spark when useJdbc is set to false

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-281: --- Assignee: (was: Raymond Xu) > HiveSync failure through Spark when useJdbc is set to false >

[jira] [Updated] (HUDI-3131) Spark3.1.1 CTAS error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3131: - Sprint: Hudi-Sprint-0.10.1 > Spark3.1.1 CTAS error > - > > Key:

[jira] [Updated] (HUDI-2661) java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.catalog.CatalogTable.copy

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2661: - Reviewers: Raymond Xu > java.lang.NoSuchMethodError: >

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Reviewers: Raymond Xu > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-1850) Read on table fails if the first write to table failed

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1850: - Reviewers: Raymond Xu > Read on table fails if the first write to table failed >

[jira] [Updated] (HUDI-2946) Upgrade maven plugin to make Hudi be compatible with higher Java versions

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2946: - Fix Version/s: (was: 0.10.1) > Upgrade maven plugin to make Hudi be compatible with higher Java

[jira] [Comment Edited] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466690#comment-17466690 ] Raymond Xu edited comment on HUDI-3107 at 12/30/21, 5:41 AM: - [~danielzhang]

[jira] [Assigned] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-3107: Assignee: Yue Zhang (was: Yue Zhang) > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Commented] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466690#comment-17466690 ] Raymond Xu commented on HUDI-3107: -- [~danielzhang]  > Fix HiveSyncTool drop partitions using JDBC >

[jira] [Updated] (HUDI-3131) Spark3.1.1 CTAS error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3131: - Story Points: 1 > Spark3.1.1 CTAS error > - > > Key: HUDI-3131 >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Story Points: 1 > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-3125) Spark SQL writing timestamp type don't need to disable `spark.sql.datetime.java8API.enabled` manually

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3125: - Issue Type: Bug (was: Improvement) > Spark SQL writing timestamp type don't need to disable >

[jira] [Updated] (HUDI-2426) spark sql extensions breaks read.table from metastore

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2426: - Story Points: 1 > spark sql extensions breaks read.table from metastore >

[jira] [Updated] (HUDI-2611) `create table if not exists` should print message instead of throwing error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2611: - Story Points: 1 > `create table if not exists` should print message instead of throwing error >

[jira] [Updated] (HUDI-2661) java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.catalog.CatalogTable.copy

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2661: - Story Points: 1 > java.lang.NoSuchMethodError: > org.apache.spark.sql.catalyst.catalog.CatalogTable.copy

[jira] [Updated] (HUDI-2915) Fix field not found in record error for spark-sql

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2915: - Story Points: 1 (was: 2) > Fix field not found in record error for spark-sql >

[jira] [Updated] (HUDI-2901) Fixed the bug clustering jobs are not running in parallel

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2901: - Story Points: 1 > Fixed the bug clustering jobs are not running in parallel >

[jira] [Updated] (HUDI-2901) Fixed the bug clustering jobs are not running in parallel

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2901: - Sprint: Hudi-Sprint-0.10.1 > Fixed the bug clustering jobs are not running in parallel >

[jira] [Updated] (HUDI-3131) Spark3.1.1 CTAS error

2021-12-29 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3131: - Reviewers: Raymond Xu > Spark3.1.1 CTAS error > - > > Key: HUDI-3131

<    4   5   6   7   8   9   10   11   12   13   >