[jira] [Updated] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3425: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Clean up spill path created by Hudi during

[jira] [Updated] (HUDI-4241) Revisit Disabled IT testRunHoodieJavaAppOnMultiPartitionKeysMORTable

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4241: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled IT

[jira] [Updated] (HUDI-3384) Implement Spark-specific FileWriters

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3384: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Implement Spark-specific FileWriters >

[jira] [Updated] (HUDI-3529) Improve dependency management and bundling

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3529: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Improve dependency management and bundling

[jira] [Updated] (HUDI-3523) Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3523: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Introduce AddColumnSchemaPostProcessor to

[jira] [Updated] (HUDI-3596) Understand why NULLSchemaProvider is used for empty batch in InputBatch instead of empty schema

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3596: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Understand why NULLSchemaProvider is used

[jira] [Updated] (HUDI-3532) Refactor FileSystemBackedTableMetadata and related classes to support getColumnStats directly

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3532: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Refactor FileSystemBackedTableMetadata and

[jira] [Updated] (HUDI-3818) hudi doesn't support bytes column as primary key

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3818: -- Fix Version/s: 0.12.1 (was: 0.12.0) > hudi doesn't support bytes column as

[jira] [Updated] (HUDI-3632) ensure Deltastreamer writes succeed if a target base path exists, but w/ no contents

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3632: -- Fix Version/s: 0.12.1 (was: 0.12.0) > ensure Deltastreamer writes succeed if a

[jira] [Updated] (HUDI-3648) Failed to execute rollback due to HoodieIOException: Could not delete instant

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3648: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Failed to execute rollback due to

[jira] [Updated] (HUDI-4230) Revisit Disabled Tests in TestHiveIncrementalPuller.testPuller

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4230: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled Tests in

[jira] [Updated] (HUDI-3809) Make full scan optional for metadata partitions other than FILES

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3809: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Make full scan optional for metadata

[jira] [Updated] (HUDI-3778) Rename module names in hudi-spark-datasource

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3778: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Rename module names in

[jira] [Updated] (HUDI-4248) Upgrade Apache Avro version for hudi-flink

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4248: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Upgrade Apache Avro version for hudi-flink

[jira] [Updated] (HUDI-3874) Improve Hudi Quickstart Docs

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3874: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Improve Hudi Quickstart Docs >

[jira] [Updated] (HUDI-4228) Clean up literal usage in Hudi CLI argument check

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4228: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Clean up literal usage in Hudi CLI

[jira] [Updated] (HUDI-4269) Support multiple precombine fields

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4269: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support multiple precombine fields >

[jira] [Updated] (HUDI-3852) Enable embedded timeline server for integ tests

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3852: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Enable embedded timeline server for integ

[jira] [Updated] (HUDI-3998) getCommitsSinceLastCleaning failed when async cleaning

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3998: -- Fix Version/s: 0.12.1 (was: 0.12.0) > getCommitsSinceLastCleaning failed when

[jira] [Updated] (HUDI-4062) Only rollback the failed writes pre upgrade under optimistic concurrency

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4062: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Only rollback the failed writes pre

[jira] [Updated] (HUDI-4431) Fix log file will not roll over to a new file

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4431: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Fix log file will not roll over to a new

[jira] [Updated] (HUDI-3851) Re-populate completed instant from inflight/requested for an empty instant during archival

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3851: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Re-populate completed instant from

[jira] [Updated] (HUDI-4222) Make HoodieMetadataPayload#preCombine not return a new payload

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4222: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Make HoodieMetadataPayload#preCombine not

[jira] [Updated] (HUDI-3951) Support general parameter 'sink.parallelism' to define the parallelism of the hudi sink operator for flink.

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3951: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support general parameter

[jira] [Updated] (HUDI-3822) Fail metadata table validation early for mismatch file slice if timeline has no inflight instant

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3822: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Fail metadata table validation early for

[jira] [Updated] (HUDI-4155) Support optional Source schema config for S3EventsHoodieIncrSource

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4155: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support optional Source schema config for

[jira] [Updated] (HUDI-4088) Flink ORC base file format does not work

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4088: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Flink ORC base file format does not work >

[jira] [Updated] (HUDI-4310) Add docs around spark scheduler configs for compaction and clustering

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4310: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Add docs around spark scheduler configs

[jira] [Updated] (HUDI-4229) Revisit Disabled tests in TestOrcBootstrap

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4229: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled tests in TestOrcBootstrap

[jira] [Updated] (HUDI-3646) The Hudi update syntax should not modify the nullability attribute of a column

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3646: -- Fix Version/s: 0.12.1 (was: 0.12.0) > The Hudi update syntax should not modify

[jira] [Updated] (HUDI-4245) Support nested fields in Column Stats Index

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4245: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support nested fields in Column Stats

[jira] [Updated] (HUDI-4231) Revisit Disabled testHoodieFlinkQuickstart

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4231: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled testHoodieFlinkQuickstart

[jira] [Updated] (HUDI-3967) Automatic savepoint in Hudi

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3967: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Automatic savepoint in Hudi >

[jira] [Updated] (HUDI-3882) Make sure Hudi Spark relations implementations provide similar file-scanning metrics

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3882: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Make sure Hudi Spark relations

[jira] [Updated] (HUDI-3892) Add HoodieReadClient with java

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3892: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Add HoodieReadClient with java >

[jira] [Updated] (HUDI-4341) HoodieHFileReader is not compatible with Hadoop 3

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4341: -- Fix Version/s: 0.12.1 (was: 0.12.0) > HoodieHFileReader is not compatible with

[jira] [Updated] (HUDI-4238) Revisit TestCOWDataSourceStorage#testCopyOnWriteStorage

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4238: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit

[jira] [Updated] (HUDI-4442) Converting from json to avro does not sanitize field names

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4442: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Converting from json to avro does not

[jira] [Updated] (HUDI-4417) Update Hudi Storage docs

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4417: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Update Hudi Storage docs >

[jira] [Updated] (HUDI-4306) ComplexKeyGenerator and ComplexAvroKeyGenerator support non-partitioned table

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4306: -- Fix Version/s: 0.12.1 (was: 0.12.0) > ComplexKeyGenerator and

[jira] [Updated] (HUDI-4434) Disable EMRFS and EMR spark related properties

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4434: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Disable EMRFS and EMR spark related

[jira] [Updated] (HUDI-4236) Revisit Disabled ITTestHoodieSanity#testRunHoodieJavaAppOnMultiPartitionKeysMORTable

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4236: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled >

[jira] [Updated] (HUDI-4244) Support common Spark transformations w/in Spark SQL "partitioned by" clause

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4244: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support common Spark transformations w/in

[jira] [Updated] (HUDI-4330) NPE when trying to upsert into a dataset with no Meta Fields

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4330: -- Fix Version/s: 0.12.1 (was: 0.12.0) > NPE when trying to upsert into a dataset

[jira] [Updated] (HUDI-4603) Improve HMS Catalog function in flink

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4603: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Improve HMS Catalog function in flink >

[jira] [Updated] (HUDI-4383) Make hudi-flink-bundle module compile with the correct flink version

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4383: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Make hudi-flink-bundle module compile with

[jira] [Updated] (HUDI-4620) No expected exception is thrown when create hudi table without primaryKey

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4620: -- Fix Version/s: 0.12.1 (was: 0.12.0) > No expected exception is thrown when

[jira] [Updated] (HUDI-4462) Flink Sink cannot report metrics

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4462: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Flink Sink cannot report metrics >

[jira] [Updated] (HUDI-4529) Tweak some default config options for flink

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4529: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Tweak some default config options for

[jira] [Updated] (HUDI-4243) Revisit the usage of SerializableConfiguration in Spark

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4243: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit the usage of

[jira] [Updated] (HUDI-4429) Make Spark 3.1.3 the default profile

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4429: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Make Spark 3.1.3 the default profile >

[jira] [Updated] (HUDI-4274) Presto/Trino support hive timestamp type

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4274: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Presto/Trino support hive timestamp type >

[jira] [Updated] (HUDI-4454) Support hiveSync command based on Call Produce Command

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4454: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support hiveSync command based on Call

[jira] [Updated] (HUDI-4232) Revisit disabled tests affected by hadoop mini cluster

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4232: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit disabled tests affected by hadoop

[jira] [Updated] (HUDI-4339) Add example configuration for HoodieCleaner in docs

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4339: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Add example configuration for

[jira] [Updated] (HUDI-4266) Flink streaming reader can not work when there are multiple partition fields

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4266: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Flink streaming reader can not work when

[jira] [Updated] (HUDI-4488) Improve S3 File listing efficiency

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4488: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Improve S3 File listing efficiency >

[jira] [Updated] (HUDI-4502) Handle default partitions in upgrade as default value changed

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4502: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Handle default partitions in upgrade as

[jira] [Updated] (HUDI-4327) TestHoodieDeltaStreamer#testCleanerDeleteReplacedDataWithArchive is flaky

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4327: -- Fix Version/s: 0.12.1 (was: 0.12.0) >

[jira] [Updated] (HUDI-4342) Improve handling of 5xx in timeline server

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4342: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Improve handling of 5xx in timeline server

[jira] [Updated] (HUDI-4240) Revisit TestCOWDataSourceStorage#testCopyOnWriteStorage

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4240: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit

[jira] [Updated] (HUDI-4358) Standardize the order field(orderingVal/eventTime) of Hudi

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4358: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Standardize the order

[jira] [Updated] (HUDI-4233) Revisit Disabled testMergeOnReadSnapshotRelationWithDeltaLogsFallback

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4233: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Revisit Disabled

[jira] [Updated] (HUDI-4452) Include hudi-aws to hudi-spark-bundle to fix cloudwatch reporter issue

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4452: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Include hudi-aws to hudi-spark-bundle to

[jira] [Updated] (HUDI-4370) Support JsonConverter in Kafka Connect sink

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4370: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Support JsonConverter in Kafka Connect

[jira] [Updated] (HUDI-4522) [DOCS] Set presto session prop to use parquet column names in case of type mismatch

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4522: -- Fix Version/s: 0.12.1 (was: 0.12.0) > [DOCS] Set presto session prop to use

[jira] [Updated] (HUDI-4541) Flink job fails with column stats enabled in metadata table due to NotSerializableException

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4541: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Flink job fails with column stats enabled

[jira] [Updated] (HUDI-4573) Fix HoodieMultiTableDeltaStreamer to write all tables in continuous mode

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4573: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Fix HoodieMultiTableDeltaStreamer to write

[jira] [Updated] (HUDI-4542) Flink streaming query fails with ClassNotFoundException

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4542: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Flink streaming query fails with

[jira] [Updated] (HUDI-4563) Docs writing for 0.12.0: key gen API change and perf improvements

2022-08-16 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4563: -- Fix Version/s: 0.12.1 (was: 0.12.0) > Docs writing for 0.12.0: key gen API

[GitHub] [hudi] hudi-bot commented on pull request #6386: [HUDI-4616] Adding `PulsarSource` to `DeltaStreamer` to support ingesting from Apache Pulsar

2022-08-16 Thread GitBox
hudi-bot commented on PR #6386: URL: https://github.com/apache/hudi/pull/6386#issuecomment-1217464725 ## CI report: * 2dae584421add6a89fbf4c7b51775581d37c03c5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6358: [HUDI-4588] Fixing `HoodieParquetReader` to properly specify projected schema when reading Parquet file

2022-08-16 Thread GitBox
hudi-bot commented on PR #6358: URL: https://github.com/apache/hudi/pull/6358#issuecomment-1217464702 ## CI report: * 378a3752f4cdf975b47efeada5c26cd4ce089215 Azure:

[GitHub] [hudi] boneanxs commented on issue #6412: [SUPPORT]query between 0 and max commit time yields empty result set.

2022-08-16 Thread GitBox
boneanxs commented on issue #6412: URL: https://github.com/apache/hudi/issues/6412#issuecomment-1217463991 @bithw1 Have you tried: `spark.sql(s"select * from tbl_order_incremental where _hoodie_commit_time > "0" and _hoodie_commit_time <= $T2").show(truncate = false)` to make "0" as

[GitHub] [hudi] hudi-bot commented on pull request #6386: [HUDI-4616] Adding `PulsarSource` to `DeltaStreamer` to support ingesting from Apache Pulsar

2022-08-16 Thread GitBox
hudi-bot commented on PR #6386: URL: https://github.com/apache/hudi/pull/6386#issuecomment-1217462449 ## CI report: * 2dae584421add6a89fbf4c7b51775581d37c03c5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6287: [HUDI-4529] Tweak some default config options for flink

2022-08-16 Thread GitBox
hudi-bot commented on PR #6287: URL: https://github.com/apache/hudi/pull/6287#issuecomment-1217462341 ## CI report: * 8ee4fe78f5946a90cf2e8c011ab9ba4470d82d94 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6358: [HUDI-4588] Fixing `HoodieParquetReader` to properly specify projected schema when reading Parquet file

2022-08-16 Thread GitBox
hudi-bot commented on PR #6358: URL: https://github.com/apache/hudi/pull/6358#issuecomment-1217462420 ## CI report: * 378a3752f4cdf975b47efeada5c26cd4ce089215 Azure:

[GitHub] [hudi] alexeykudinkin opened a new pull request, #6416: [WIP] Fixing `DebeziumSource` to properly commit consumed offsets

2022-08-16 Thread GitBox
alexeykudinkin opened a new pull request, #6416: URL: https://github.com/apache/hudi/pull/6416 ### Change Logs Fixing `DebeziumSource` to properly commit consumed offsets; Tidying up ### Impact Low. ### Contributor's checklist - [ ] Read through

[GitHub] [hudi] danny0405 commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-16 Thread GitBox
danny0405 commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r946486735 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,27 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing data we

[GitHub] [hudi] hudi-bot commented on pull request #6287: [HUDI-4529] Tweak some default config options for flink

2022-08-16 Thread GitBox
hudi-bot commented on PR #6287: URL: https://github.com/apache/hudi/pull/6287#issuecomment-1217437400 ## CI report: * 8ee4fe78f5946a90cf2e8c011ab9ba4470d82d94 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6413: [MINOR] Update DOAP with 0.12.0 Release

2022-08-16 Thread GitBox
hudi-bot commented on PR #6413: URL: https://github.com/apache/hudi/pull/6413#issuecomment-1217431180 ## CI report: * 98d233b95b8653fa681b2c24aa900c7a86adddf3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6413: [MINOR] Update DOAP with 0.12.0 Release

2022-08-16 Thread GitBox
hudi-bot commented on PR #6413: URL: https://github.com/apache/hudi/pull/6413#issuecomment-1217427261 ## CI report: * 98d233b95b8653fa681b2c24aa900c7a86adddf3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6409: [HUDI-4629] Create hive table from existing hoodie Table failed when the table schema is not defined

2022-08-16 Thread GitBox
hudi-bot commented on PR #6409: URL: https://github.com/apache/hudi/pull/6409#issuecomment-1217427234 ## CI report: * 6adf14d3fa24209bfafac3d6e3180ec0d49fab10 Azure:

[GitHub] [hudi] YannByron closed pull request #5885: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-16 Thread GitBox
YannByron closed pull request #5885: [HUDI-3478] Support CDC for Spark in Hudi URL: https://github.com/apache/hudi/pull/5885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] YannByron commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-16 Thread GitBox
YannByron commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r947420470 ## rfc/rfc-51/rfc-51.md: ## @@ -64,69 +65,72 @@ We follow the debezium output format: four columns as shown below Note: the illustration here ignores all the Hudi

[GitHub] [hudi] YannByron commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-16 Thread GitBox
YannByron commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r947417129 ## rfc/rfc-51/rfc-51.md: ## @@ -148,20 +152,27 @@ hudi_cdc_table/ Under a partition directory, the `.log` file with `CDCBlock` above will keep the changing data we

[GitHub] [hudi] 1032851561 commented on issue #6167: [SUPPORT] No results are returned from incremental queries within the archived range

2022-08-16 Thread GitBox
1032851561 commented on issue #6167: URL: https://github.com/apache/hudi/issues/6167#issuecomment-1217415847 > > In this case, why not merge archived instants before return? > > @1032851561 i don't think it's expected to return incremental results for archived commits. A design

[GitHub] [hudi] vinothchandar commented on pull request #6408: [DOCS] Edits to the Hudi Tech specs

2022-08-16 Thread GitBox
vinothchandar commented on PR #6408: URL: https://github.com/apache/hudi/pull/6408#issuecomment-1217410416 @prasannarajaperumal Not sure if `tunable` is the right word either. Not married to it. Landed for now, lets keep looking and update if we find sth better. -- This is an automated

[hudi] branch asf-site updated: [DOCS] Edits to the Hudi Tech specs (#6408)

2022-08-16 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 9735364cc0 [DOCS] Edits to the Hudi Tech

[GitHub] [hudi] vinothchandar merged pull request #6408: [DOCS] Edits to the Hudi Tech specs

2022-08-16 Thread GitBox
vinothchandar merged PR #6408: URL: https://github.com/apache/hudi/pull/6408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 opened a new pull request, #6415: [HUDI-4632] Remove the force active property for flink1.14 profile

2022-08-16 Thread GitBox
danny0405 opened a new pull request, #6415: URL: https://github.com/apache/hudi/pull/6415 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[jira] [Updated] (HUDI-4632) Remove the force active property for flink1.14 profile

2022-08-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4632: - Labels: pull-request-available (was: ) > Remove the force active property for flink1.14 profile

[jira] [Created] (HUDI-4632) Remove the force active property for flink1.14 profile

2022-08-16 Thread Danny Chen (Jira)
Danny Chen created HUDI-4632: Summary: Remove the force active property for flink1.14 profile Key: HUDI-4632 URL: https://issues.apache.org/jira/browse/HUDI-4632 Project: Apache Hudi Issue Type:

[GitHub] [hudi] boneanxs opened a new issue, #6414: [SUPPORT] Spark3 with Hadoop3 using metadata could have compatible issue when reading hfile

2022-08-16 Thread GitBox
boneanxs opened a new issue, #6414: URL: https://github.com/apache/hudi/issues/6414 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

[GitHub] [hudi] danny0405 commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-16 Thread GitBox
danny0405 commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r947407085 ## rfc/rfc-51/rfc-51.md: ## @@ -64,69 +65,72 @@ We follow the debezium output format: four columns as shown below Note: the illustration here ignores all the Hudi

[GitHub] [hudi] danny0405 commented on a diff in pull request #6312: [HUDI-4551] The default value of READ_TASKS, WRITE_TASKS, CLUSTERING_TASKS is the parallelism of the execution environment

2022-08-16 Thread GitBox
danny0405 commented on code in PR #6312: URL: https://github.com/apache/hudi/pull/6312#discussion_r947403350 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java: ## @@ -492,8 +492,6 @@ void

[GitHub] [hudi] codope opened a new pull request, #6413: [MINOR] Update DOAP with 0.12.0 Release

2022-08-16 Thread GitBox
codope opened a new pull request, #6413: URL: https://github.com/apache/hudi/pull/6413 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] YannByron commented on a diff in pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-16 Thread GitBox
YannByron commented on code in PR #6256: URL: https://github.com/apache/hudi/pull/6256#discussion_r947401482 ## rfc/rfc-51/rfc-51.md: ## @@ -64,69 +65,72 @@ We follow the debezium output format: four columns as shown below Note: the illustration here ignores all the Hudi

[GitHub] [hudi] hudi-bot commented on pull request #6358: [HUDI-4588] Fixing `HoodieParquetReader` to properly specify projected schema when reading Parquet file

2022-08-16 Thread GitBox
hudi-bot commented on PR #6358: URL: https://github.com/apache/hudi/pull/6358#issuecomment-1217392517 ## CI report: * 378a3752f4cdf975b47efeada5c26cd4ce089215 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6358: [HUDI-4588] Fixing `HoodieParquetReader` to properly specify projected schema when reading Parquet file

2022-08-16 Thread GitBox
hudi-bot commented on PR #6358: URL: https://github.com/apache/hudi/pull/6358#issuecomment-1217390172 ## CI report: * 9cb5a7a62af7c2a6bf418b7556caa56348522a00 Azure:

[GitHub] [hudi] danny0405 commented on issue #6411: Hudi Record Key Data Type Must be String

2022-08-16 Thread GitBox
danny0405 commented on issue #6411: URL: https://github.com/apache/hudi/issues/6411#issuecomment-1217374162 The byte primary key type expects to be supported, what exception it throws there for your use case ? -- This is an automated message from the Apache Git Service. To respond to the

<    1   2   3   4   5   >