[jira] [Updated] (HUDI-4248) Upgrade Apache Avro version for hudi-flink

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4248: Fix Version/s: 0.14.0 (was: 0.13.1) > Upgrade Apache Avro version for hudi-flink >

[jira] [Updated] (HUDI-3519) Make sure every public Hudi Client Method invokes necessary prologue

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3519: Fix Version/s: 0.14.0 (was: 0.13.1) > Make sure every public Hudi Client Method

[jira] [Updated] (HUDI-4245) Support nested fields in Column Stats Index

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4245: Fix Version/s: 0.14.0 (was: 0.13.1) > Support nested fields in Column Stats Index >

[jira] [Updated] (HUDI-3674) Remove unnecessary HBase-related dependencies from bundles if there is any

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3674: Fix Version/s: 0.14.0 (was: 0.13.1) > Remove unnecessary HBase-related dependencies

[jira] [Updated] (HUDI-2767) Enable timeline server based marker type as default

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2767: Fix Version/s: 0.14.0 (was: 0.13.1) > Enable timeline server based marker type as

[jira] [Updated] (HUDI-3879) Suppress exceptions that are not fatal in HoodieMetadataTableValidator

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3879: Fix Version/s: 0.14.0 (was: 0.13.1) > Suppress exceptions that are not fatal in

[jira] [Updated] (HUDI-3115) Kafka Connect should not be packaged as a bundle

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3115: Fix Version/s: 0.14.0 (was: 0.13.1) > Kafka Connect should not be packaged as a

[jira] [Updated] (HUDI-3531) Review and shade transitive dependencies in hudi bundle jar

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3531: Fix Version/s: 0.14.0 (was: 0.13.1) > Review and shade transitive dependencies in

[jira] [Updated] (HUDI-3321) HFileWriter, HFileReader and HFileDataBlock should avoid hardcoded key field name

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3321: Fix Version/s: 0.14.0 (was: 0.13.1) > HFileWriter, HFileReader and HFileDataBlock

[jira] [Updated] (HUDI-3317) Partition specific pointed lookup/reading strategy for metadata table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3317: Fix Version/s: 0.14.0 (was: 0.13.1) > Partition specific pointed lookup/reading

[jira] [Updated] (HUDI-2737) Use earliest instant by default for compaction and clustering job

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2737: Fix Version/s: 0.14.0 (was: 0.13.1) > Use earliest instant by default for compaction

[jira] [Updated] (HUDI-2736) Redundant metadata table initialization by the metadata writer

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2736: Fix Version/s: 0.14.0 (was: 0.13.1) > Redundant metadata table initialization by the

[jira] [Updated] (HUDI-2458) Relax compaction in metadata being fenced based on inflight requests in data table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2458: Fix Version/s: 0.14.0 (was: 0.13.1) > Relax compaction in metadata being fenced

[jira] [Updated] (HUDI-2388) Add test nodes for Spark SQL in integration test suite

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-2388: Fix Version/s: 0.14.0 (was: 0.13.1) > Add test nodes for Spark SQL in integration

[jira] [Updated] (HUDI-1101) Decouple Hive dependencies from hudi-spark

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-1101: Fix Version/s: 0.14.0 (was: 0.13.1) > Decouple Hive dependencies from hudi-spark >

[jira] [Updated] (HUDI-6127) Flink Hudi Support Commit on empty batch

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6127: Issue Type: Improvement (was: New Feature) > Flink Hudi Support Commit on empty batch >

[jira] [Updated] (HUDI-5517) HoodieTimeline support filter instants by state transition time

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5517: Fix Version/s: (was: 0.13.1) > HoodieTimeline support filter instants by state transition time >

[jira] [Updated] (HUDI-6091) Add Java 11 and 17 to bundle validation image

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6091: Issue Type: Improvement (was: New Feature) > Add Java 11 and 17 to bundle validation image >

[jira] [Updated] (HUDI-5941) Support savepoint CALL procedure with table base path

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5941: Fix Version/s: 0.14.0 (was: 0.13.1) > Support savepoint CALL procedure with table

[jira] [Updated] (HUDI-6176) Fix flaky test testArchivalWithMultiWriters

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6176: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix flaky test testArchivalWithMultiWriters >

[jira] [Updated] (HUDI-6138) HoodieAvroRecord - Fix Option get for empty values

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6138: Fix Version/s: 0.14.0 (was: 0.13.1) > HoodieAvroRecord - Fix Option get for empty

[jira] [Updated] (HUDI-6061) NPE with nullable MapType and new hudi merger

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6061: Fix Version/s: 0.14.0 (was: 0.13.1) > NPE with nullable MapType and new hudi merger

[jira] [Updated] (HUDI-5904) support more than one update actions in merge into table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5904: Fix Version/s: 0.14.0 (was: 0.13.1) > support more than one update actions in merge

[jira] [Updated] (HUDI-5890) Fix build failure of asf-site branch

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5890: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix build failure of asf-site branch >

[jira] [Updated] (HUDI-6011) Hudi CLI show archived commits is broken for replace commit

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6011: Fix Version/s: (was: 0.13.1) > Hudi CLI show archived commits is broken for replace commit >

[jira] [Updated] (HUDI-5914) Fix for RowData class cast exception

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5914: Fix Version/s: (was: 0.13.1) > Fix for RowData class cast exception >

[jira] [Updated] (HUDI-5968) Global index update partition for MOR creating duplicates

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5968: Fix Version/s: (was: 0.13.1) > Global index update partition for MOR creating duplicates >

[jira] [Updated] (HUDI-6025) Incremental read with MOR doesn't give correct results

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-6025: Fix Version/s: 0.14.0 (was: 0.13.1) > Incremental read with MOR doesn't give correct

[jira] [Updated] (HUDI-5824) COMBINE_BEFORE_UPSERT=false option does not work for upsert

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5824: Fix Version/s: 0.14.0 (was: 0.13.1) > COMBINE_BEFORE_UPSERT=false option does not

[jira] [Updated] (HUDI-5867) Use commons.io v2.7+ for hbase-server

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5867: Fix Version/s: 0.14.0 (was: 0.13.1) > Use commons.io v2.7+ for hbase-server >

[jira] [Updated] (HUDI-5864) Update release notes regarding the HoodieMetadataFileSystemView regression

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5864: Fix Version/s: 0.14.0 (was: 0.13.1) > Update release notes regarding the

[jira] [Updated] (HUDI-5866) Fix unnecessary log messages during bulk insert in Spark

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5866: Fix Version/s: (was: 0.13.1) > Fix unnecessary log messages during bulk insert in Spark >

[GitHub] [hudi] codope commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
codope commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1201549502 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieMetaSyncOperations.java: ## @@ -186,16 +188,20 @@ default void

[jira] [Updated] (HUDI-5760) Make sure DeleteBlock doesn't use Kryo for serialization to disk

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5760: Fix Version/s: 0.14.0 (was: 0.13.1) > Make sure DeleteBlock doesn't use Kryo for

[jira] [Updated] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5807: Fix Version/s: 0.14.0 (was: 0.13.1) > HoodieSparkParquetReader is not appending

[jira] [Updated] (HUDI-5759) Hudi do not support add column on mor table with log

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5759: Fix Version/s: 0.14.0 (was: 0.13.1) > Hudi do not support add column on mor table

[jira] [Updated] (HUDI-5769) Partitions created by Async indexer could be deleted by regular writers

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5769: Fix Version/s: 0.14.0 (was: 0.13.1) > Partitions created by Async indexer could be

[jira] [Updated] (HUDI-5737) Fix Deletes issued without any prior commits

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5737: Fix Version/s: (was: 0.13.1) > Fix Deletes issued without any prior commits >

[jira] [Updated] (HUDI-5733) TestHoodieDeltaStreamer.testHoodieIndexer failure

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5733: Fix Version/s: 0.14.0 (was: 0.13.1) > TestHoodieDeltaStreamer.testHoodieIndexer

[jira] [Updated] (HUDI-5731) Fix com.google.common classes still being relocated in Hudi Spark bundle

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5731: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix com.google.common classes still being

[GitHub] [hudi] xushiyan commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
xushiyan commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1201547833 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieMetaSyncOperations.java: ## @@ -186,16 +188,20 @@ default void

[jira] [Updated] (HUDI-5670) Server-based markers creation times out

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5670: Fix Version/s: 0.14.0 (was: 0.13.1) > Server-based markers creation times out >

[GitHub] [hudi] bvaradar commented on a diff in pull request #8303: [HUDI-5998] Speed up reads from bootstrapped tables in spark

2023-05-22 Thread via GitHub
bvaradar commented on code in PR #8303: URL: https://github.com/apache/hudi/pull/8303#discussion_r1201546844 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBootstrapRelation.scala: ## @@ -188,11 +188,23 @@ case class

[jira] [Updated] (HUDI-5711) NPE occurs when enabling metadata on table which does'nt has metadata previously

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5711: Fix Version/s: 0.14.0 (was: 0.13.1) > NPE occurs when enabling metadata on table

[GitHub] [hudi] xushiyan commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
xushiyan commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1201544493 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -280,83 +282,87 @@ protected void syncHoodieTable(String tableName, boolean

[jira] [Updated] (HUDI-5697) Spark SQL re-lists Hudi table after every SQL operations

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5697: Fix Version/s: 0.14.0 (was: 0.13.1) > Spark SQL re-lists Hudi table after every SQL

[jira] [Updated] (HUDI-5688) schema field of EmptyRelation subtype of BaseRelation should not be null

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5688: Fix Version/s: 0.14.0 (was: 0.13.1) > schema field of EmptyRelation subtype of

[jira] [Updated] (HUDI-5716) Fix Partitioners to avoid assuming that parallelism is always present

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5716: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix Partitioners to avoid assuming that

[jira] [Updated] (HUDI-5609) Hudi table not queryable by SQL on Databricks Spark

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5609: Fix Version/s: 0.14.0 (was: 0.13.1) > Hudi table not queryable by SQL on Databricks

[jira] [Updated] (HUDI-5619) Fix HoodieTableFileSystemView inefficient latest base-file lookups

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5619: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix HoodieTableFileSystemView inefficient

[jira] [Updated] (HUDI-5597) Deltastreamer ingestion fails when consistent hashing index is used

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5597: Fix Version/s: 0.14.0 (was: 0.13.1) > Deltastreamer ingestion fails when consistent

[jira] [Updated] (HUDI-5641) Streamline Advanced Schema Evolution flow

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5641: Fix Version/s: 0.14.0 (was: 0.13.1) > Streamline Advanced Schema Evolution flow >

[jira] [Updated] (HUDI-5602) Troubleshoot METADATA_ONLY bootstrapped table not being able to read back partition path

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5602: Fix Version/s: 0.14.0 (was: 0.13.1) > Troubleshoot METADATA_ONLY bootstrapped table

[jira] [Updated] (HUDI-5608) Support decimals w/ precision > 30 in Column Stats

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5608: Fix Version/s: 0.14.0 (was: 0.13.1) > Support decimals w/ precision > 30 in Column

[jira] [Updated] (HUDI-5575) Support any record key generation along w/ any partition path generation for row writer

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5575: Fix Version/s: 0.14.0 (was: 0.13.1) > Support any record key generation along w/ any

[jira] [Updated] (HUDI-5574) Support auto record key generation with Spark SQL

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5574: Fix Version/s: 0.14.0 (was: 0.13.1) > Support auto record key generation with Spark

[jira] [Updated] (HUDI-5588) Fix Metadata table validator to deduce valid partitions when first commit where partition was added is failed

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5588: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix Metadata table validator to deduce valid

[jira] [Updated] (HUDI-5444) FileNotFound issue w/ metadata enabled

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5444: Fix Version/s: 0.14.0 (was: 0.13.1) > FileNotFound issue w/ metadata enabled >

[jira] [Updated] (HUDI-5507) SparkSQL can not read the latest change data without execute "refresh table xxx"

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5507: Fix Version/s: 0.14.0 (was: 0.13.1) > SparkSQL can not read the latest change data

[jira] [Updated] (HUDI-5557) Wrong candidate files found in metadata table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5557: Fix Version/s: 0.14.0 (was: 0.13.1) > Wrong candidate files found in metadata table

[jira] [Updated] (HUDI-5463) Apply rollback commits from data table as rollbacks in MDT instead of Delta commit

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5463: Fix Version/s: 0.14.0 (was: 0.13.1) > Apply rollback commits from data table as

[jira] [Updated] (HUDI-5442) Fix HiveHoodieTableFileIndex to use lazy listing

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5442: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix HiveHoodieTableFileIndex to use lazy

[jira] [Updated] (HUDI-5520) Fail MDT when list of log files grows unboundedly

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5520: Fix Version/s: 0.14.0 (was: 0.13.1) > Fail MDT when list of log files grows

[jira] [Updated] (HUDI-5436) Auto repair tool for MDT out of sync

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5436: Fix Version/s: 0.14.0 (was: 0.13.1) > Auto repair tool for MDT out of sync >

[jira] [Updated] (HUDI-5374) Use KeyGeneratorFactory class for instantiating a KeyGenerator

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5374: Fix Version/s: 0.14.0 (was: 0.13.1) > Use KeyGeneratorFactory class for

[jira] [Updated] (HUDI-5271) Inconsistent reader and writer schema in HoodieAvroDataBlock cause exception

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5271: Fix Version/s: 0.14.0 (was: 0.13.1) > Inconsistent reader and writer schema in

[jira] [Updated] (HUDI-5364) Make sure Hudi's Column Stats are wired into Spark's relation stats

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5364: Fix Version/s: 0.14.0 (was: 0.13.1) > Make sure Hudi's Column Stats are wired into

[jira] [Updated] (HUDI-5385) Make behavior of keeping File Writers open configurable

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5385: Fix Version/s: 0.14.0 (was: 0.13.1) > Make behavior of keeping File Writers open

[jira] [Updated] (HUDI-5322) Bulk-insert (row-writing) is not rewriting incoming dataset into Writer's schema

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5322: Fix Version/s: 0.14.0 (was: 0.13.1) > Bulk-insert (row-writing) is not rewriting

[jira] [Updated] (HUDI-5405) Avoid using Projections in generic Merge Into DMLs

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5405: Fix Version/s: 0.14.0 (was: 0.13.1) > Avoid using Projections in generic Merge Into

[jira] [Updated] (HUDI-5361) Propagate Hudi properties set in Spark's SQLConf to Hudi

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5361: Fix Version/s: 0.14.0 (was: 0.13.1) > Propagate Hudi properties set in Spark's

[jira] [Updated] (HUDI-5438) Benchmark calls w/ metadata enabled and ensure no calls to direct FS

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5438: Fix Version/s: 0.14.0 (was: 0.13.1) > Benchmark calls w/ metadata enabled and ensure

[jira] [Updated] (HUDI-5352) Jackson fails to serialize LocalDate when updating Delta Commit metadata

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5352: Fix Version/s: 0.14.0 (was: 0.13.1) > Jackson fails to serialize LocalDate when

[jira] [Updated] (HUDI-5319) NPE in Bloom Filter Index

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5319: Fix Version/s: 0.14.0 (was: 0.13.1) > NPE in Bloom Filter Index >

[GitHub] [hudi] hudi-bot commented on pull request #8445: [HUDI-3088] Use Spark 3.2 as default Spark version

2023-05-22 Thread via GitHub
hudi-bot commented on PR #8445: URL: https://github.com/apache/hudi/pull/8445#issuecomment-1558546855 ## CI report: * fe494c5e09f8c3a57446834c86ad82904bcda585 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xushiyan commented on a diff in pull request #8775: [HUDI-5584] Metasync update props when changed

2023-05-22 Thread via GitHub
xushiyan commented on code in PR #8775: URL: https://github.com/apache/hudi/pull/8775#discussion_r1201537869 ## hudi-aws/src/main/java/org/apache/hudi/aws/sync/AWSGlueCatalogSyncClient.java: ## @@ -477,13 +472,19 @@ private static Table getTable(AWSGlue awsGlue, String

[jira] [Updated] (HUDI-4944) The encoded slash (%2F) in partition path is not properly decoded during Spark read

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4944: Fix Version/s: 0.14.0 (was: 0.13.1) > The encoded slash (%2F) in partition path is

[jira] [Updated] (HUDI-4937) Fix HoodieTable injecting HoodieBackedTableMetadata not reusing underlying MT readers

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4937: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix HoodieTable injecting

[jira] [Updated] (HUDI-4738) [MOR] Bloom Index missing new records inserted into Log files

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4738: Fix Version/s: 0.14.0 (was: 0.13.1) > [MOR] Bloom Index missing new records inserted

[jira] [Updated] (HUDI-5080) UnpersistRdds unpersist all rdds in the spark context

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5080: Fix Version/s: (was: 0.13.1) > UnpersistRdds unpersist all rdds in the spark context >

[jira] [Updated] (HUDI-4947) Missing .hoodie/hoodie.properties in Hudi table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4947: Fix Version/s: 0.14.0 (was: 0.13.1) > Missing .hoodie/hoodie.properties in Hudi

[jira] [Updated] (HUDI-4922) Presto query of bootstrapped data returns null

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4922: Fix Version/s: 0.14.0 (was: 0.13.1) > Presto query of bootstrapped data returns

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5092: Fix Version/s: 0.14.0 (was: 0.13.1) > Querying Hudi table throws NoSuchMethodError

[jira] [Updated] (HUDI-5015) Cleaner does not work properly when metadata table is enabled

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5015: Fix Version/s: 0.14.0 (was: 0.13.1) > Cleaner does not work properly when metadata

[jira] [Updated] (HUDI-4958) Provide accurate numDeletes in commit metadata

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4958: Fix Version/s: 0.14.0 (was: 0.13.1) > Provide accurate numDeletes in commit metadata

[jira] [Updated] (HUDI-5229) Add flink avro version entry in root pom

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-5229: Fix Version/s: 0.14.0 (was: 0.13.1) > Add flink avro version entry in root pom >

[jira] [Updated] (HUDI-4777) Flink gen bucket index of mor table not consistent with spark lead to duplicate bucket issue

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4777: Fix Version/s: 0.14.0 (was: 0.13.1) > Flink gen bucket index of mor table not

[jira] [Updated] (HUDI-4921) Fix last completed commit in CleanPlanner

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4921: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix last completed commit in CleanPlanner >

[jira] [Updated] (HUDI-4854) Deltastreamer does not respect partition selector regex for metadata-only bootstrap

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4854: Fix Version/s: 0.14.0 (was: 0.13.1) > Deltastreamer does not respect partition

[jira] [Updated] (HUDI-4852) Incremental sync not updating pending file groups under clustering

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4852: Fix Version/s: 0.14.0 (was: 0.13.1) > Incremental sync not updating pending file

[jira] [Updated] (HUDI-4818) Using CustomKeyGenerator fails w/ SparkHoodieTableFileIndex

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4818: Fix Version/s: 0.14.0 (was: 0.13.1) > Using CustomKeyGenerator fails w/

[GitHub] [hudi] bvaradar commented on a diff in pull request #8452: [HUDI-6077] Add more partition push down filters

2023-05-22 Thread via GitHub
bvaradar commented on code in PR #8452: URL: https://github.com/apache/hudi/pull/8452#discussion_r1201504641 ## hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java: ## @@ -96,11 +109,32 @@ public List getPartitionPathWithPathPrefixes(List

[jira] [Updated] (HUDI-4632) Remove the force active property for flink1.14 profile

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4632: Fix Version/s: 0.14.0 (was: 0.13.1) > Remove the force active property for flink1.14

[jira] [Updated] (HUDI-4643) MergeInto syntax WHEN MATCHED is optional but must be set

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4643: Fix Version/s: 0.14.0 (was: 0.13.1) > MergeInto syntax WHEN MATCHED is optional but

[jira] [Updated] (HUDI-4704) bulk insert overwrite table will delete the table and then recreate a table

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4704: Fix Version/s: 0.14.0 (was: 0.13.1) > bulk insert overwrite table will delete the

[jira] [Updated] (HUDI-4542) Flink streaming query fails with ClassNotFoundException

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4542: Fix Version/s: 0.14.0 (was: 0.13.1) > Flink streaming query fails with

[jira] [Updated] (HUDI-4573) Fix HoodieMultiTableDeltaStreamer to write all tables in continuous mode

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4573: Fix Version/s: 0.14.0 (was: 0.13.1) > Fix HoodieMultiTableDeltaStreamer to write all

[jira] [Updated] (HUDI-4541) Flink job fails with column stats enabled in metadata table due to NotSerializableException

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4541: Fix Version/s: 0.14.0 (was: 0.13.1) > Flink job fails with column stats enabled in

[jira] [Updated] (HUDI-4457) Make sure IT docker test return code non-zero when failed

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4457: Fix Version/s: 0.14.0 (was: 0.13.1) > Make sure IT docker test return code non-zero

[jira] [Updated] (HUDI-4430) Incorrect type casting while reading HUDI table created with CustomKeyGenerator and unixtimestamp paritioning field

2023-05-22 Thread Yue Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-4430: Fix Version/s: 0.14.0 (was: 0.13.1) > Incorrect type casting while reading HUDI

  1   2   3   4   5   >