[GitHub] [hudi] xushiyan commented on issue #3672: [SUPPORT] Spark-SQL version 3.1.2 get exception while insert record into table

2021-09-19 Thread GitBox
xushiyan commented on issue #3672: URL: https://github.com/apache/hudi/issues/3672#issuecomment-922665870 also mentioned in #3609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[jira] [Updated] (HUDI-1689) Support Multipath query for HoodieFileIndex

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1689: - Parent: HUDI-1658 Issue Type: Sub-task (was: Improvement) > Support Multipath query for HoodieFil

[jira] [Updated] (HUDI-1292) [Umbrella] RFC-15 : File Listing and Query Planning Optimizations

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1292: - Priority: Critical (was: Major) > [Umbrella] RFC-15 : File Listing and Query Planning Optimizations > --

[jira] [Updated] (HUDI-1297) [Umbrella] Revamp Spark Datasource support using Spark 3 APIs

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1297: - Priority: Critical (was: Major) > [Umbrella] Revamp Spark Datasource support using Spark 3 APIs > ---

[jira] [Closed] (HUDI-30) Explore support for Spark Datasource V2

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-30. -- Resolution: Duplicate > Explore support for Spark Datasource V2 > --- > >

[jira] [Updated] (HUDI-893) Add spark datasource V2 reader support for Hudi tables

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-893: Parent Issue: HUDI-1297 (was: HUDI-30) > Add spark datasource V2 reader support for Hudi tables > --

[jira] [Updated] (HUDI-1307) spark datasource load path format is confused for snapshot and increment read mode

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1307: - Labels: sev:high user-support-issues (was: sev:normal user-support-issues) > spark datasource load path f

[jira] [Updated] (HUDI-1307) spark datasource load path format is confused for snapshot and increment read mode

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1307: - Priority: Critical (was: Major) > spark datasource load path format is confused for snapshot and incremen

[jira] [Updated] (HUDI-1608) MOR fetches all records for read optimized query w/ spark sql

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1608: - Priority: Critical (was: Major) > MOR fetches all records for read optimized query w/ spark sql > ---

[jira] [Updated] (HUDI-1689) Support Multipath query for HoodieFileIndex

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1689: - Priority: Critical (was: Major) > Support Multipath query for HoodieFileIndex > -

[jira] [Updated] (HUDI-1894) NULL values in timestamp column defaulted

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1894: - Priority: Critical (was: Major) > NULL values in timestamp column defaulted > --

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: schema sev:critical (was: schema sev:high) > Upsert of Case Class with single field causes Schema

[jira] [Updated] (HUDI-2071) Support Reading Bootstrap MOR RT Table In Spark DataSource Table

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2071: - Priority: Critical (was: Major) > Support Reading Bootstrap MOR RT Table In Spark DataSource Table > ---

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: schema sev sev:high (was: schema sev:high) > Upsert of Case Class with single field causes Schema

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Priority: Critical (was: Major) > Upsert of Case Class with single field causes SchemaParseException > --

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: schema sev:high (was: schema sev sev:high) > Upsert of Case Class with single field causes Schema

[jira] [Updated] (HUDI-1079) Cannot upsert on schema with Array of Record with single field

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1079: - Labels: schema sev:critical user-support-issues (was: sev:critical user-support-issues) > Cannot upsert

[jira] [Updated] (HUDI-1079) Cannot upsert on schema with Array of Record with single field

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1079: - Labels: sev:critical user-support-issues (was: sev:high user-support-issues) > Cannot upsert on schema w

[jira] [Updated] (HUDI-1079) Cannot upsert on schema with Array of Record with single field

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1079: - Priority: Critical (was: Blocker) > Cannot upsert on schema with Array of Record with single field >

[jira] [Updated] (HUDI-1079) Cannot upsert on schema with Array of Record with single field

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1079: - Priority: Blocker (was: Major) > Cannot upsert on schema with Array of Record with single field > ---

[jira] [Updated] (HUDI-1117) Add tdunning json library to spark and utilities bundle

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1117: - Component/s: (was: Spark Integration) Hive Integration > Add tdunning json library to

[jira] [Updated] (HUDI-1278) Need a generic payload class which can skip late arriving data based on specific fields

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1278: - Component/s: (was: Spark Integration) > Need a generic payload class which can skip late arriving data

[jira] [Updated] (HUDI-1491) Support partition pruning for MOR snapshot query

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1491: - Parent: HUDI-1297 Issue Type: Sub-task (was: Improvement) > Support partition pruning for MOR sna

[jira] [Updated] (HUDI-1272) Add utility scripts to manage Savepoints

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1272: - Parent: HUDI-1297 Issue Type: Sub-task (was: New Feature) > Add utility scripts to manage Savepoi

[jira] [Updated] (HUDI-1271) Add utility scripts to perform Restores

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1271: - Parent: HUDI-1297 Issue Type: Sub-task (was: New Feature) > Add utility scripts to perform Restor

[jira] [Updated] (HUDI-1341) hudi cli command such as rollback 、bootstrap support spark sql implement

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1341: - Parent: HUDI-1297 Issue Type: Sub-task (was: New Feature) > hudi cli command such as rollback 、bo

[jira] [Updated] (HUDI-1368) Merge On Read Snapshot Reader not working for Databricks on ADLS Gen2

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1368: - Issue Type: Bug (was: New Feature) > Merge On Read Snapshot Reader not working for Databricks on ADLS Gen

[jira] [Updated] (HUDI-1608) MOR fetches all records for read optimized query w/ spark sql

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1608: - Labels: pull-request-available sev:critical (was: pull-request-available) > MOR fetches all records for r

[jira] [Commented] (HUDI-1689) Support Multipath query for HoodieFileIndex

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417453#comment-17417453 ] Raymond Xu commented on HUDI-1689: -- [~pzw2018] to confirm, I think this is a feature enha

[jira] [Closed] (HUDI-1680) Add getPreferredLocations for RDD

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-1680. Fix Version/s: (was: 0.10.0) Resolution: Won't Do Closing due to inactive > Add getPreferredLoca

[jira] [Updated] (HUDI-1689) Support Multipath query for HoodieFileIndex

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1689: - Labels: (was: sev:high) > Support Multipath query for HoodieFileIndex >

[jira] [Updated] (HUDI-1894) NULL values in timestamp column defaulted

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1894: - Labels: schema sev:high triaged (was: sev:high triaged) > NULL values in timestamp column defaulted > --

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: schema sev:high (was: AvroSchema sev:high) > Upsert of Case Class with single field causes Schema

[jira] [Updated] (HUDI-1926) Concurrent write conflict when generating same instanttime

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1926: - Labels: sev:triage (was: ) > Concurrent write conflict when generating same instanttime > --

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: AvroSchema sev:high (was: AvroSchema) > Upsert of Case Class with single field causes SchemaParse

[jira] [Updated] (HUDI-17) Better documentation for paths passed to incr and ro views from Spark datasource

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-17: --- Priority: Minor (was: Major) > Better documentation for paths passed to incr and ro views from Spark > datasou

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Labels: AvroSchema (was: ) > Upsert of Case Class with single field causes SchemaParseException > ---

[jira] [Updated] (HUDI-2323) Upsert of Case Class with single field causes SchemaParseException

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2323: - Priority: Major (was: Minor) > Upsert of Case Class with single field causes SchemaParseException > -

[jira] [Updated] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Summary: KeyGenerator discrepancy between DataFrame writer and SQL (was: KeyGenerator discrepancy between

[jira] [Commented] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and Spark SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417449#comment-17417449 ] Raymond Xu commented on HUDI-2390: -- Pivoting this ticket to an improvement: to address di

[jira] [Updated] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and Spark SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Labels: (was: features) > KeyGenerator discrepancy between DataFrame writer and Spark SQL >

[jira] [Updated] (HUDI-2390) KeyGenerator discrepancy between DataFrame writer and Spark SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Summary: KeyGenerator discrepancy between DataFrame writer and Spark SQL (was: Create table by hudisql,wr

[jira] [Updated] (HUDI-2390) Create table by hudisql,write data into table by datasource,hudi delete cmd can not delete data

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Issue Type: Improvement (was: Bug) > Create table by hudisql,write data into table by datasource,hudi del

[jira] [Updated] (HUDI-2390) Create table by hudisql,write data into table by datasource,hudi delete cmd can not delete data

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Priority: Minor (was: Critical) > Create table by hudisql,write data into table by datasource,hudi delete

[jira] [Updated] (HUDI-2390) Create table by hudisql,write data into table by datasource,hudi delete cmd can not delete data

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2390: - Priority: Critical (was: Major) > Create table by hudisql,write data into table by datasource,hudi delete

[jira] [Updated] (HUDI-2452) spark on hudi metadata key length < 0

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2452: - Labels: sev:critical (was: ) > spark on hudi metadata key length < 0 > --

[jira] [Resolved] (HUDI-2387) Too many HEAD requests from Hudi to S3

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu resolved HUDI-2387. -- Resolution: Information Provided > Too many HEAD requests from Hudi to S3 > ---

[jira] [Commented] (HUDI-2387) Too many HEAD requests from Hudi to S3

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417438#comment-17417438 ] Raymond Xu commented on HUDI-2387: -- [~souravtri] Have you contacted AWS support to get so

[jira] [Updated] (HUDI-2387) Too many HEAD requests from Hudi to S3

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2387: - Component/s: (was: Spark Integration) (was: Common Core) Usabilit

[jira] [Updated] (HUDI-2387) Too many HEAD requests from Hudi to S3

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2387: - Issue Type: Improvement (was: Bug) > Too many HEAD requests from Hudi to S3 > --

[jira] [Updated] (HUDI-2452) spark on hudi metadata key length < 0

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2452: - Labels: (was: pull-request-available) > spark on hudi metadata key length < 0 >

[jira] [Updated] (HUDI-2452) spark on hudi metadata key length < 0

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2452: - Description: spark on hudi metadata key length <= 0, but data primary key is not "" or null,error message

[GitHub] [hudi] xushiyan commented on pull request #3691: [HUDI-2455] Adding spark_avro dependency to hudi-integ-test

2021-09-19 Thread GitBox
xushiyan commented on pull request #3691: URL: https://github.com/apache/hudi/pull/3691#issuecomment-922617121 @YannByron Don't see this is a problem. The package command ran ok on master. Can you confirm if this is a local issue in your environment? -- This is an automated message from

[jira] [Assigned] (HUDI-2455) Fail to build Spark caused by spark_avro dependency

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-2455: Assignee: Yann Byron > Fail to build Spark caused by spark_avro dependency > --

[jira] [Updated] (HUDI-2455) Fail to build Spark caused by spark_avro dependency

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2455: - Component/s: (was: Spark Integration) > Fail to build Spark caused by spark_avro dependency >

[jira] [Assigned] (HUDI-2456) Support show partitions SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-2456: Assignee: Yann Byron > Support show partitions SQL > --- > >

[jira] [Updated] (HUDI-2456) Support show partitions SQL

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2456: - Status: In Progress (was: Open) > Support show partitions SQL > --- > >

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-19 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 6f1ded9808d42381fd2ce644d441f8830a5a55cb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] t822876884 commented on issue #3689: [SUPPORT] flink sql write to hudi fail

2021-09-19 Thread GitBox
t822876884 commented on issue #3689: URL: https://github.com/apache/hudi/issues/3689#issuecomment-922604125 > Hi, you may need to use the 0.9.0 with flink 1.12.2 or master with flink 1.13.1. You also need to open the checkpoint. tks,i will try again,I think the official website sugge

[GitHub] [hudi] hudi-bot edited a comment on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-19 Thread GitBox
hudi-bot edited a comment on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 6f1ded9808d42381fd2ce644d441f8830a5a55cb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] nsivabalan commented on a change in pull request #3413: [HUDI-2277] HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-09-19 Thread GitBox
nsivabalan commented on a change in pull request #3413: URL: https://github.com/apache/hudi/pull/3413#discussion_r711845020 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java ## @@ -1398,6 +1399,34 @@ private void testP

[jira] [Updated] (HUDI-2456) Support show partitions SQL

2021-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2456: - Labels: features pull-request-available (was: features) > Support show partitions SQL > -

[GitHub] [hudi] hudi-bot commented on pull request #3693: [HUDI-2456] support 'show partitions' sql

2021-09-19 Thread GitBox
hudi-bot commented on pull request #3693: URL: https://github.com/apache/hudi/pull/3693#issuecomment-922600059 ## CI report: * 6f1ded9808d42381fd2ce644d441f8830a5a55cb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-1071) Upgrade dropwizard metrics to make use of SettableGauge

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1071: - Labels: new-to-hudi (was: newbie) > Upgrade dropwizard metrics to make use of SettableGauge > ---

[jira] [Updated] (HUDI-1355) Allowing multipleSourceOrdering fields for doing the preCombine on payload

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1355: - Labels: new-to-hudi patch starter (was: patch starter) > Allowing multipleSourceOrdering fields for doing

[jira] [Updated] (HUDI-904) Segregate metrics configs by reporter type

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-904: Labels: new-to-hudi (was: newbie) > Segregate metrics configs by reporter type > ---

[jira] [Assigned] (HUDI-2454) Remove travis config from asf-site branch

2021-09-19 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-2454: Assignee: Raymond Xu > Remove travis config from asf-site branch >

[GitHub] [hudi] danny0405 commented on pull request #3203: [HUDI-2086] Redo the logical of mor_incremental_view for hive

2021-09-19 Thread GitBox
danny0405 commented on pull request #3203: URL: https://github.com/apache/hudi/pull/3203#issuecomment-922561725 > @danny0405 Thank you for your attention to this pr。 This pr has not been updated for a long time. Do we still need this now? We need it definitely, for a complete Hive re

[GitHub] [hudi] codejoyan closed issue #3581: [SUPPORT] Slow snapshot query performance

2021-09-19 Thread GitBox
codejoyan closed issue #3581: URL: https://github.com/apache/hudi/issues/3581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[GitHub] [hudi] codejoyan commented on issue #3581: [SUPPORT] Slow snapshot query performance

2021-09-19 Thread GitBox
codejoyan commented on issue #3581: URL: https://github.com/apache/hudi/issues/3581#issuecomment-922536416 It works without the file listing step and is reading from metadata table !!! @xushiyan Thanks a lot for the solution. -- This is an automated message from the Apache Git Servi

[hudi] branch asf-site updated: [HUDI-2457] Fixing minor issues in docs (#3692)

2021-09-19 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 9c71829 [HUDI-2457] Fixing minor issues in

[GitHub] [hudi] nsivabalan merged pull request #3692: [HUDI-2457] Fixing minor issues in docs

2021-09-19 Thread GitBox
nsivabalan merged pull request #3692: URL: https://github.com/apache/hudi/pull/3692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[jira] [Updated] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2457: - Labels: pull-request-available (was: ) > Fix hudi website for current and 090 for spark bundle ve

[GitHub] [hudi] nsivabalan opened a new pull request #3692: [HUDI-2457] Fixing minor issues in docs

2021-09-19 Thread GitBox
nsivabalan opened a new pull request #3692: URL: https://github.com/apache/hudi/pull/3692 ## What is the purpose of the pull request *(For example: This pull request adds quick-start document.)* ## Brief change log *(for example:)* - *Modify AnnotationLocation check

[jira] [Created] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-19 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2457: - Summary: Fix hudi website for current and 090 for spark bundle versions. Key: HUDI-2457 URL: https://issues.apache.org/jira/browse/HUDI-2457 Project: Apach

[jira] [Assigned] (HUDI-2457) Fix hudi website for current and 090 for spark bundle versions.

2021-09-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-2457: - Assignee: sivabalan narayanan > Fix hudi website for current and 090 for spark bu

[GitHub] [hudi] nsivabalan commented on issue #3676: MOR table rolls out new parquet files at 10MB for new inserts - even though max file size set as 128MB

2021-09-19 Thread GitBox
nsivabalan commented on issue #3676: URL: https://github.com/apache/hudi/issues/3676#issuecomment-922508543 I found the rootcause. Looks like in MOR, when an index is used which cannot index log files (which is the case for all out of box indexes in hudi), we just choose the smallest parqu

[GitHub] [hudi] YannByron commented on issue #3555: [SUPPORT] support show/drop partitions tablename sql

2021-09-19 Thread GitBox
YannByron commented on issue #3555: URL: https://github.com/apache/hudi/issues/3555#issuecomment-922441246 jira to show partitions: https://issues.apache.org/jira/browse/HUDI-2455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[jira] [Created] (HUDI-2456) Support show partitions SQL

2021-09-19 Thread Yann Byron (Jira)
Yann Byron created HUDI-2456: Summary: Support show partitions SQL Key: HUDI-2456 URL: https://issues.apache.org/jira/browse/HUDI-2456 Project: Apache Hudi Issue Type: Improvement Compo