[I] [SUPPORT] Process Fallout: Unable to upsert due to continues Failure in Compaction [hudi]

2023-10-04 Thread via GitHub
ketkidev opened a new issue, #9822: URL: https://github.com/apache/hudi/issues/9822 **Describe the problem you faced** Unable to process data into the Hudi tables as inline Compaction is failing. **To Reproduce** Previous **hoodie.compact.inline.max.delta.commits**

Re: [PR] [HUDI-6916] Improve performance of Custom Key Generators [hudi]

2023-10-04 Thread via GitHub
hudi-bot commented on PR #9821: URL: https://github.com/apache/hudi/pull/9821#issuecomment-1748048326 ## CI report: * 56ca25f24d40cdf1fb1897662072976470c9aebc Azure:

[jira] [Reopened] (HUDI-2388) Add test nodes for Spark SQL in integration test suite

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha reopened HUDI-2388: - > Add test nodes for Spark SQL in integration test suite >

[jira] [Resolved] (HUDI-2388) Add test nodes for Spark SQL in integration test suite

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha resolved HUDI-2388. - > Add test nodes for Spark SQL in integration test suite >

[jira] [Closed] (HUDI-2737) Use earliest instant by default for compaction and clustering job

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha closed HUDI-2737. --- Resolution: Fixed > Use earliest instant by default for compaction and clustering job >

[jira] [Reopened] (HUDI-2737) Use earliest instant by default for compaction and clustering job

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha reopened HUDI-2737: - > Use earliest instant by default for compaction and clustering job >

[jira] [Closed] (HUDI-2388) Add test nodes for Spark SQL in integration test suite

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha closed HUDI-2388. --- Resolution: Fixed > Add test nodes for Spark SQL in integration test suite >

[jira] [Resolved] (HUDI-2737) Use earliest instant by default for compaction and clustering job

2023-10-04 Thread Bhavani Sudha (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavani Sudha resolved HUDI-2737. - > Use earliest instant by default for compaction and clustering job >

Re: [PR] [HUDI-6916] Improve performance of Custom Key Generators [hudi]

2023-10-04 Thread via GitHub
hudi-bot commented on PR #9821: URL: https://github.com/apache/hudi/pull/9821#issuecomment-1748012790 ## CI report: * 56ca25f24d40cdf1fb1897662072976470c9aebc Azure:

[jira] [Updated] (HUDI-6916) Fix excessive object creation in custom key generator

2023-10-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6916: - Labels: pull-request-available (was: ) > Fix excessive object creation in custom key generator >

Re: [PR] [HUDI-6916] Improve performance of Custom Key Generators [hudi]

2023-10-04 Thread via GitHub
hudi-bot commented on PR #9821: URL: https://github.com/apache/hudi/pull/9821#issuecomment-1748007570 ## CI report: * 56ca25f24d40cdf1fb1897662072976470c9aebc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Closed] (HUDI-6153) Change the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason closed HUDI-6153. Fix Version/s: 0.14.0 (was: 0.14.1) Resolution: Fixed > Change the

[PR] [HUID-6916] Improve performance of Custom Key Generators [hudi]

2023-10-04 Thread via GitHub
the-other-tim-brown opened a new pull request, #9821: URL: https://github.com/apache/hudi/pull/9821 ### Change Logs Fixes an issue in the custom key generators where we are creating objects per record/row instead of reusing them. This leads to excess object creation which in turn

[jira] [Updated] (HUDI-1887) Make schema post processor's default as disabled

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1887: - Fix Version/s: 0.14.1 (was: 0.14.0) > Make schema post processor's default

[jira] [Updated] (HUDI-5552) Too slow while using trino-hudi connector while querying partitioned tables.

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5552: - Fix Version/s: 0.14.1 (was: 0.14.0) > Too slow while using trino-hudi

[jira] [Updated] (HUDI-4123) HoodieDeltaStreamer throws exception due to SqlSource return null checkpoint

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4123: - Fix Version/s: 0.14.1 (was: 0.14.0) > HoodieDeltaStreamer throws exception

[jira] [Updated] (HUDI-5490) Investigate test failures w/ record level index for existing tests

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5490: - Fix Version/s: 0.14.1 (was: 0.14.0) > Investigate test failures w/ record

[jira] [Updated] (HUDI-3026) HoodieAppendhandle may result in duplicate key for hbase index

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3026: - Fix Version/s: 0.14.1 (was: 0.14.0) > HoodieAppendhandle may result in

[jira] [Updated] (HUDI-4701) Support bulk insert without primary key and precombine field

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4701: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support bulk insert without primary

[jira] [Updated] (HUDI-4585) Optimize query performance on Presto Hudi connector

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4585: - Fix Version/s: 0.14.1 (was: 0.14.0) > Optimize query performance on

[jira] [Updated] (HUDI-6163) Add PR size labeler to Hudi repo

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6163: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add PR size labeler to Hudi repo >

[jira] [Updated] (HUDI-4294) Introduce build action to actually perform index data generation

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4294: - Fix Version/s: 0.14.1 (was: 0.14.0) > Introduce build action to actually

[jira] [Updated] (HUDI-6139) Add support for Transformer schema validation in DeltaStreamer

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6139: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add support for Transformer schema

[jira] [Updated] (HUDI-5323) Decouple virtual key with writing bloom filters to parquet files

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5323: - Fix Version/s: 0.14.1 (was: 0.14.0) > Decouple virtual key with writing

[jira] [Updated] (HUDI-2754) Performance improvement for IncrementalRelation

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2754: - Fix Version/s: 0.14.1 (was: 0.14.0) > Performance improvement for

[jira] [Updated] (HUDI-541) Replace variables/comments named "data files" to "base file"

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-541: Fix Version/s: 0.14.1 (was: 0.14.0) > Replace variables/comments named "data

[jira] [Updated] (HUDI-3541) Array of struct or Struct of Array AvroConversion issue

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3541: - Fix Version/s: 0.14.1 (was: 0.14.0) > Array of struct or Struct of Array

[jira] [Updated] (HUDI-4954) Shade avro in all bundles where it is included

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4954: - Fix Version/s: 0.14.1 (was: 0.14.0) > Shade avro in all bundles where it

[jira] [Updated] (HUDI-96) Use Command line options instead of positional arguments when launching spark applications from various CLI commands

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-96?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-96: --- Fix Version/s: 0.14.1 (was: 0.14.0) > Use Command line options instead of

[jira] [Updated] (HUDI-3676) Enhance tests for triggering clean every Nth commit

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3676: - Fix Version/s: 0.14.1 (was: 0.14.0) > Enhance tests for triggering clean

[jira] [Updated] (HUDI-5423) Flaky test: ColumnStatsTestCase(MERGE_ON_READ,true,true)

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5423: - Fix Version/s: 0.14.1 (was: 0.14.0) > Flaky test:

[jira] [Updated] (HUDI-3617) MOR compact improve

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3617: - Fix Version/s: 0.14.1 (was: 0.14.0) > MOR compact improve >

[jira] [Updated] (HUDI-6317) Streaming read should skip compaction and clustering instants to avoid duplicates

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6317: - Fix Version/s: 0.14.1 (was: 0.14.0) > Streaming read should skip

[jira] [Updated] (HUDI-1628) [Umbrella] Improve data locality during ingestion

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1628: - Fix Version/s: 0.14.1 (was: 0.14.0) (was: 1.1.0) >

[jira] [Updated] (HUDI-3555) re-use spark config for parquet timestamp format instead of having our own config

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3555: - Fix Version/s: 0.14.1 (was: 0.14.0) > re-use spark config for parquet

[jira] [Updated] (HUDI-4854) Deltastreamer does not respect partition selector regex for metadata-only bootstrap

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4854: - Fix Version/s: 0.14.1 (was: 0.14.0) > Deltastreamer does not respect

[jira] [Updated] (HUDI-6105) Partial update for MERGE INTO

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6105: - Fix Version/s: 0.14.1 (was: 0.14.0) > Partial update for MERGE INTO >

[jira] [Updated] (HUDI-1864) Support for java.time.LocalDate in TimestampBasedAvroKeyGenerator

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1864: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support for java.time.LocalDate in

[jira] [Updated] (HUDI-5464) Fix instantiation of a new partition in MDT re-using the same instant time as a regular commit

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5464: - Fix Version/s: 0.14.1 (was: 0.14.0) > Fix instantiation of a new partition

[jira] [Updated] (HUDI-6126) Fix test testInsertDatasetWIthTimelineTimezoneUTC to not block CI

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6126: - Fix Version/s: 0.14.1 (was: 0.14.0) > Fix test

[jira] [Updated] (HUDI-5997) Support DFS Schema Provider with S3/GCS EventsHoodieIncrSource

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5997: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support DFS Schema Provider with

[jira] [Updated] (HUDI-2681) Make hoodie record_key and preCombine_key optional

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2681: - Fix Version/s: 0.14.1 (was: 0.14.0) > Make hoodie record_key and

[jira] [Updated] (HUDI-1120) Support spotless for scala

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1120: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support spotless for scala >

[jira] [Updated] (HUDI-2151) Make performant out-of-box configs

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2151: - Fix Version/s: 0.14.1 (was: 0.14.0) > Make performant out-of-box configs >

[jira] [Updated] (HUDI-6112) Improve Doc generatiion to generate config tables for basic and advanced configs

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6112: - Fix Version/s: 0.14.1 (was: 0.14.0) > Improve Doc generatiion to generate

[jira] [Updated] (HUDI-6075) Improve config generation script and docs

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6075: - Fix Version/s: 0.14.1 (was: 0.14.0) > Improve config generation script and

[jira] [Updated] (HUDI-6116) Optimize log block reading by removing seeks to check corrupted blocks

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6116: - Fix Version/s: 0.14.1 (was: 0.14.0) > Optimize log block reading by

[jira] [Updated] (HUDI-4569) [RFC-59] Multiple event_time fields latest verification in a single table

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4569: - Fix Version/s: 0.14.1 (was: 0.14.0) (was: 1.1.0) >

[jira] [Updated] (HUDI-5175) Improving FileIndex load performance in PARALLELISM mode

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5175: - Fix Version/s: 0.14.1 (was: 0.14.0) > Improving FileIndex load performance

[jira] [Updated] (HUDI-3068) Add support to sync all partitions in hive sync tool

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3068: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add support to sync all partitions

[jira] [Updated] (HUDI-5131) Bundle validation: upgrade/downgrade

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5131: - Fix Version/s: 0.14.1 (was: 0.14.0) > Bundle validation: upgrade/downgrade

[jira] [Updated] (HUDI-2517) Simplify the amount of configs that need to be passed in for Delta Streamer

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2517: - Fix Version/s: 0.14.1 (was: 0.14.0) > Simplify the amount of configs that

[jira] [Updated] (HUDI-4937) Fix HoodieTable injecting HoodieBackedTableMetadata not reusing underlying MT readers

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4937: - Fix Version/s: 0.14.1 (was: 0.14.0) > Fix HoodieTable injecting

[jira] [Updated] (HUDI-2832) [Umbrella] [RFC-40] Integrated Hudi with Snowflake

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2832: - Fix Version/s: 0.14.1 (was: 0.14.0) > [Umbrella] [RFC-40] Integrated Hudi

[jira] [Updated] (HUDI-5079) Optimize rdd.isEmpty within DeltaSync

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5079: - Fix Version/s: 0.14.1 (was: 0.14.0) > Optimize rdd.isEmpty within

[jira] [Updated] (HUDI-2687) [UMBRELLA] A new Trino connector for Hudi

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2687: - Fix Version/s: 0.14.1 (was: 1.0.0) (was: 0.14.0)

[jira] [Updated] (HUDI-5092) Querying Hudi table throws NoSuchMethodError in Databricks runtime

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5092: - Fix Version/s: 0.14.1 (was: 0.14.0) > Querying Hudi table throws

[jira] [Updated] (HUDI-4287) Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4287: - Fix Version/s: 0.14.1 (was: 0.14.0) > Optimize Flink checkpoint meta

[jira] [Updated] (HUDI-3957) Evaluate Support for spark2 and scala12

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3957: - Fix Version/s: 0.14.1 (was: 0.14.0) > Evaluate Support for spark2 and

[jira] [Updated] (HUDI-4105) Identify out of the box performance config flips for spark-ds

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4105: - Fix Version/s: 0.14.1 (was: 0.14.0) > Identify out of the box performance

[jira] [Updated] (HUDI-4449) Spark: Support DataSourceV2 Read

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4449: - Fix Version/s: 0.14.1 (was: 0.14.0) > Spark: Support DataSourceV2 Read >

[jira] [Updated] (HUDI-956) Test MOR : Presto Realtime Query with metadata bootstrap

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-956: Fix Version/s: 0.14.1 (was: 0.14.0) > Test MOR : Presto Realtime Query with

[jira] [Updated] (HUDI-1258) Small file handling Merges can be handled without actual merging

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1258: - Fix Version/s: 0.14.1 (was: 0.14.0) > Small file handling Merges can be

[jira] [Updated] (HUDI-6282) Config conflict with Deltastreamer CustomKeyGenerator for PartitionPath

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6282: - Fix Version/s: 0.14.1 (was: 0.14.0) > Config conflict with Deltastreamer

[jira] [Updated] (HUDI-4967) Improve docs for meta sync with TimestampBasedKeyGenerator

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4967: - Fix Version/s: 0.14.1 (was: 0.14.0) > Improve docs for meta sync with

[jira] [Updated] (HUDI-992) For hive-style partitioned source data, partition columns synced with Hive will always have String type

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-992: Fix Version/s: 0.14.1 (was: 0.14.0) > For hive-style partitioned source data,

[jira] [Updated] (HUDI-6136) Add idempotency support to spark datasource writes

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6136: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add idempotency support to spark

[jira] [Updated] (HUDI-1280) Add tool to capture earliest or latest offsets in kafka topics

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1280: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add tool to capture earliest or

[jira] [Updated] (HUDI-2516) Upgrade to Junit 5.8.2

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2516: - Fix Version/s: 0.14.1 (was: 0.14.0) > Upgrade to Junit 5.8.2 >

[jira] [Updated] (HUDI-5018) Make user-provided copyOnWriteRecordSizeEstimate first precedence

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5018: - Fix Version/s: 0.14.1 (was: 0.14.0) > Make user-provided

[jira] [Updated] (HUDI-1549) Programmatic way to fetch earliest commit retained

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1549: - Fix Version/s: 0.14.1 (was: 0.14.0) > Programmatic way to fetch earliest

[jira] [Updated] (HUDI-1936) Introduce a optional property for conditional upsert

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1936: - Fix Version/s: 0.14.1 (was: 0.14.0) > Introduce a optional property for

[jira] [Updated] (HUDI-3582) Introduce Secondary Index to Improve HUDI Query Performance

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3582: - Fix Version/s: 0.14.1 (was: 0.14.0) > Introduce Secondary Index to Improve

[jira] [Updated] (HUDI-3560) Add docker image for spark3 hadoop3 and hive3

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3560: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add docker image for spark3 hadoop3

[jira] [Updated] (HUDI-1588) Support multiple ordering fields via payload class config

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1588: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support multiple ordering fields via

[jira] [Updated] (HUDI-6122) Call clean/compaction support custom options

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6122: - Fix Version/s: 0.14.1 (was: 0.14.0) > Call clean/compaction support custom

[jira] [Updated] (HUDI-5951) Replace format "org.apache.hudi" with short name "hudi" in Spark Datasource

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5951: - Fix Version/s: 0.14.1 (was: 0.14.0) > Replace format "org.apache.hudi"

[jira] [Updated] (HUDI-4626) Partitioning table by `_hoodie_partition_path` fails

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4626: - Fix Version/s: 0.14.1 (was: 0.14.0) > Partitioning table by

[jira] [Updated] (HUDI-4674) change the default value of inputFormat for the MOR table

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4674: - Fix Version/s: 0.14.1 (was: 0.14.0) > change the default value of

[jira] [Updated] (HUDI-2740) Support for snapshot querying on MOR table

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2740: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support for snapshot querying on MOR

[jira] [Updated] (HUDI-4324) Remove useJdbc config from meta sync tools

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4324: - Fix Version/s: 0.14.1 (was: 0.14.0) > Remove useJdbc config from meta sync

[jira] [Updated] (HUDI-1872) Move HoodieFlinkStreamer into hudi-utilities module

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-1872: - Fix Version/s: 0.14.1 (was: 0.14.0) > Move HoodieFlinkStreamer into

[jira] [Updated] (HUDI-4613) Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4613: - Fix Version/s: 0.14.1 (was: 0.14.0) > Avoid the use of regex expressions

[jira] [Updated] (HUDI-5575) Support any record key generation along w/ any partition path generation for row writer

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5575: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support any record key generation

[jira] [Updated] (HUDI-2034) Support explicit partition compaction strategy for flink batch compaction

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2034: - Fix Version/s: 0.14.1 (was: 0.14.0) > Support explicit partition

[jira] [Updated] (HUDI-4990) Parallelize deduplication in CLI tool

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4990: - Fix Version/s: 0.14.1 (was: 0.14.0) > Parallelize deduplication in CLI

[jira] [Updated] (HUDI-4106) Identify out of the box default performance flips for spark-sql

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4106: - Fix Version/s: 0.14.1 (was: 0.14.0) > Identify out of the box default

[jira] [Updated] (HUDI-3954) Don't keep the last commit before the earliest commit to retain

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3954: - Fix Version/s: 0.14.1 (was: 0.14.0) > Don't keep the last commit before

[jira] [Updated] (HUDI-6464) Implement Spark SQL Merge Into for tables without primary key

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6464: - Fix Version/s: 0.14.1 (was: 0.14.0) > Implement Spark SQL Merge Into for

[jira] [Updated] (HUDI-5101) Adding spark structured streaming tests to integ tests

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-5101: - Fix Version/s: 0.14.1 (was: 0.14.0) > Adding spark structured streaming

[jira] [Updated] (HUDI-6552) Restructure FAQ on the website to categorize tuning and troubleshooting related questions separately.

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6552: - Fix Version/s: 0.14.1 (was: 0.14.0) > Restructure FAQ on the website to

[jira] [Updated] (HUDI-3291) Flip Default record paylod to DefaultHoodieRecordPayload

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3291: - Fix Version/s: 0.14.1 (was: 0.14.0) > Flip Default record paylod to

[jira] [Updated] (HUDI-304) Bring back spotless plugin

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-304: Fix Version/s: 0.14.1 (was: 0.14.0) > Bring back spotless plugin >

[jira] [Updated] (HUDI-4345) Incorporate partition pruning for COW incremental query

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4345: - Fix Version/s: 0.14.1 (was: 0.14.0) > Incorporate partition pruning for

[jira] [Updated] (HUDI-2296) flink support ConsistencyGuard plugin

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-2296: - Fix Version/s: 0.14.1 (was: 0.14.0) > flink support ConsistencyGuard

[jira] [Updated] (HUDI-4878) Fix incremental cleaning for clean based on LATEST_FILE_VERSIONS

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4878: - Fix Version/s: 0.14.1 (was: 0.14.0) > Fix incremental cleaning for clean

[jira] [Updated] (HUDI-4688) Decouple lazy rollback of failed writes from clean action in multi-writer

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-4688: - Fix Version/s: 0.14.1 (was: 0.14.0) > Decouple lazy rollback of failed

[jira] [Updated] (HUDI-6115) Harden expected corrupt record column in chained transformer when error table settings are on/off

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6115: - Fix Version/s: 0.14.1 (was: 0.14.0) > Harden expected corrupt record

[jira] [Updated] (HUDI-6149) Add a tool to fetch table size for hudi tables

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-6149: - Fix Version/s: 0.14.1 (was: 0.14.0) > Add a tool to fetch table size for

[jira] [Updated] (HUDI-3819) upgrade spring cve-2022-22965

2023-10-04 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3819: - Fix Version/s: 0.14.1 (was: 0.14.0) > upgrade spring cve-2022-22965 >

  1   2   3   4   5   6   >