[jira] [Updated] (HUDI-6887) Add test for Record Index and MIT queries

2023-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6887: - Labels: pull-request-available (was: ) > Add test for Record Index and MIT quer

[jira] [Updated] (HUDI-6878) Hudi Table was not initialized corretly when write multi tables in a single job

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6878: - Labels: pull-request-available (was: ) > Hudi Table was not initialized corretly when wr

[jira] [Updated] (HUDI-6877) Fix unqualified namespace issues in Spark3.1

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6877: - Labels: pull-request-available (was: ) > Fix unqualified namespace issues in Spark

[jira] [Updated] (HUDI-6881) Hudi configured spark.scheduler.allocation.file should include scheme since Spark3.2

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6881: - Labels: pull-request-available (was: ) > Hudi configured spark.scheduler.allocation.file sho

[jira] [Updated] (HUDI-6806) Support Spark 3.5.0

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6806: - Labels: pull-request-available (was: ) > Support Spark 3.

[jira] [Updated] (HUDI-6874) Move configs for reading a file group to hudi-common module

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6874: - Labels: pull-request-available (was: ) > Move configs for reading a file group to hudi-com

[jira] [Updated] (HUDI-6882) Clustering Planning uses replacecommit for last cluster even though multiple operations use replacecommit

2023-09-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6882: - Labels: pull-request-available (was: ) > Clustering Planning uses replacecommit for last clus

[jira] [Updated] (HUDI-6866) When invalidate the table in the spark sql query cache, verify if the hive-async database exists

2023-09-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6866: - Labels: pull-request-available (was: ) > When invalidate the table in the spark sql query ca

[jira] [Updated] (HUDI-6861) Update SQL Pages for 0.14.0

2023-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6861: - Labels: pull-request-available (was: ) > Update SQL Pages for 0.1

[jira] [Updated] (HUDI-6870) [BigQuerySyncTool] Pass target project id when running job.

2023-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6870: - Labels: pull-request-available (was: ) > [BigQuerySyncTool] Pass target project id when runn

[jira] [Updated] (HUDI-6869) fix schema evol docs to move OOB to first section

2023-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6869: - Labels: pull-request-available (was: ) > fix schema evol docs to move OOB to first sect

[jira] [Updated] (HUDI-6867) Upgrade thrift's version to 0.13.0

2023-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6867: - Labels: pull-request-available (was: ) > Upgrade thrift's version t

[jira] [Updated] (HUDI-6865) Fix InternalSchema schemaId when column is dropped

2023-09-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6865: - Labels: pull-request-available (was: ) > Fix InternalSchema schemaId when column is drop

[jira] [Updated] (HUDI-6863) Revert "Auto-tune dedup parallelism"

2023-09-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6863: - Labels: pull-request-available (was: ) > Revert "Auto-tune dedup par

[jira] [Updated] (HUDI-6862) Replace directory connector markers in TestSqlStatement

2023-09-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6862: - Labels: pull-request-available (was: ) > Replace directory connector markers in TestSqlStatem

[jira] [Updated] (HUDI-6784) Clean Merger API and its invocations

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6784: - Labels: pull-request-available (was: ) > Clean Merger API and its invocati

[jira] [Updated] (HUDI-6858) Fix checkpoint reading in Spark structured streaming

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6858: - Labels: pull-request-available (was: ) > Fix checkpoint reading in Spark structured stream

[jira] [Updated] (HUDI-6857) Update Docs For BigQuerySyncTool

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6857: - Labels: pull-request-available (was: ) > Update Docs For BigQuerySyncT

[jira] [Updated] (HUDI-6856) [DOCS] Add info about partially failed writes handling w/ hudi

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6856: - Labels: pull-request-available (was: ) > [DOCS] Add info about partially failed writes handl

[jira] [Updated] (HUDI-6855) Exclude .hoodie_partition_metadata file in base file group

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6855: - Labels: pull-request-available (was: ) > Exclude .hoodie_partition_metadata file in base f

[jira] [Updated] (HUDI-6853) ArchiveCommitsProcedure should throw an exception when the archive operation executes failed

2023-09-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6853: - Labels: pull-request-available (was: ) > ArchiveCommitsProcedure should throw an exception w

[jira] [Updated] (HUDI-6852) [DOCS] Create separate page for spark streaming

2023-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6852: - Labels: pull-request-available (was: ) > [DOCS] Create separate page for spark stream

[jira] [Updated] (HUDI-6851) Fix spark quick start guide for key less

2023-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6851: - Labels: pull-request-available (was: ) > Fix spark quick start guide for key l

[jira] [Updated] (HUDI-6850) Add tests and docs for ported Bloom Filter classes

2023-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6850: - Labels: pull-request-available (was: ) > Add tests and docs for ported Bloom Filter clas

[jira] [Updated] (HUDI-6830) Fix downgrade from version six for partially failed commits

2023-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6830: - Labels: pull-request-available (was: ) > Fix downgrade from version six for partially fai

[jira] [Updated] (HUDI-6826) Port BloomFilter related classes from Hadoop library to remove dependency

2023-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6826: - Labels: pull-request-available (was: ) > Port BloomFilter related classes from Hadoop library

[jira] [Updated] (HUDI-6825) Use UTF_8 to encode String to byte array in all places

2023-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6825: - Labels: pull-request-available (was: ) > Use UTF_8 to encode String to byte array in all pla

[jira] [Updated] (HUDI-6847) improve the incremental clean fallback logic

2023-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6847: - Labels: pull-request-available (was: ) > improve the incremental clean fallback lo

[jira] [Updated] (HUDI-6846) fix a bug of consistent bucket index clustering

2023-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6846: - Labels: pull-request-available (was: ) > fix a bug of consistent bucket index cluster

[jira] [Updated] (HUDI-1623) Solid completion time on timeline

2023-09-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1623: - Labels: pull-request-available (was: ) > Solid completion time on timel

[jira] [Updated] (HUDI-6842) Fix flaky testHoodieAsyncClusteringJobWithScheduleAndExecute

2023-09-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6842: - Labels: pull-request-available (was: ) > Fix fl

[jira] [Updated] (HUDI-6845) Upgrade org.apache.pulsar:pulsar-client to 2.10.2

2023-09-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6845: - Labels: pull-request-available (was: ) > Upgrade org.apache.pulsar:pulsar-client to 2.1

[jira] [Updated] (HUDI-6838) Fix file writers to honor bloom filter configs

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6838: - Labels: pull-request-available (was: ) > Fix file writers to honor bloom filter conf

[jira] [Updated] (HUDI-6839) Github Actions Workflow Improvements

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6839: - Labels: pull-request-available (was: ) > Github Actions Workflow Improveme

[jira] [Updated] (HUDI-6836) Shutdown metrics for metadata table writer in deltastreamer

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6836: - Labels: pull-request-available (was: ) > Shutdown metrics for metadata table writer

[jira] [Updated] (HUDI-6834) Time travel query for an instant not in active timeline should throw exception

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6834: - Labels: pull-request-available (was: ) > Time travel query for an instant not in active timel

[jira] [Updated] (HUDI-6835) Adjust spark sql core flow test scenarios

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6835: - Labels: pull-request-available (was: ) > Adjust spark sql core flow test scenar

[jira] [Updated] (HUDI-6753) Fix parquet inline reading flaky test

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6753: - Labels: pull-request-available (was: ) > Fix parquet inline reading flaky t

[jira] [Updated] (HUDI-6832) When one of table paths is incorrect, ensure that other table services are not affected.

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6832: - Labels: pull-request-available (was: ) > When one of table paths is incorrect, ensure that ot

[jira] [Updated] (HUDI-6831) Add back missing project_id to query statement in BigQuerySyncTool

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6831: - Labels: pull-request-available (was: ) > Add back missing project_id to query statement

[jira] [Updated] (HUDI-6336) Support TimelineBased Checkpoint Metadata for flink

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6336: - Labels: pull-request-available (was: ) > Support TimelineBased Checkpoint Metadata for fl

[jira] [Updated] (HUDI-6833) Add field for tracking log files from failed commit in rollback metadata

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6833: - Labels: pull-request-available (was: ) > Add field for tracking log files from failed commit

[jira] [Updated] (HUDI-6823) writeTimer in emitCommitMetrics need to be initialized before using

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6823: - Labels: pull-request-available (was: ) > writeTimer in emitCommitMetrics need to be initiali

[jira] [Updated] (HUDI-6820) Fix Azure CI timeout for UT FT other modules

2023-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6820: - Labels: pull-request-available (was: ) > Fix Azure CI timeout for UT FT other modu

[jira] [Updated] (HUDI-6780) Replace classnames by modes/enums in table properties

2023-08-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6780: - Labels: pull-request-available (was: ) > Replace classnames by modes/enums in table propert

[jira] [Updated] (HUDI-6795) Implement generation of record_positions for updates and deletes on write path

2023-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6795: - Labels: pull-request-available (was: ) > Implement generation of record_positions for upda

[jira] [Updated] (HUDI-6702) Extend merge API to support all merging operations

2023-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6702: - Labels: pull-request-available (was: ) > Extend merge API to support all merging operati

[jira] [Updated] (HUDI-6776) Unify commit metadata content in json for completed and avro for pending commits

2023-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6776: - Labels: pull-request-available (was: ) > Unify commit metadata content in json for completed

[jira] [Updated] (HUDI-6805) Print detailed error messages in clustering

2023-08-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6805: - Labels: pull-request-available (was: ) > Print detailed error messages in cluster

[jira] [Updated] (HUDI-6804) Fix hive read schema evolution table

2023-08-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6804: - Labels: pull-request-available (was: ) > Fix hive read schema evolution ta

[jira] [Updated] (HUDI-6495) Finalize the RFC-61/Non-blocking Concurrency Control design

2023-08-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6495: - Labels: pull-request-available (was: ) > Finalize the RFC-61/Non-blocking Concurrency Cont

[jira] [Updated] (HUDI-6734) Add back HUDI-5409 in Hudi 0.12.x branch

2023-08-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6734: - Labels: pull-request-available (was: ) > Add back HUDI-5409 in Hudi 0.12.x bra

[jira] [Updated] (HUDI-6773) Add Test cases to show case insert into behaviour with different mergers and payload.

2023-08-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6773: - Labels: pull-request-available (was: ) > Add Test cases to show case insert into behaviour w

[jira] [Updated] (HUDI-6725) Support efficient completion time queries on the timeline

2023-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6725: - Labels: pull-request-available (was: ) > Support efficient completion time queries on

[jira] [Updated] (HUDI-6712) Implement optimized keyed lookup on parquet files

2023-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6712: - Labels: pull-request-available (was: ) > Implement optimized keyed lookup on parquet fi

[jira] [Updated] (HUDI-6763) WriteStats are extracted twice in BaseSparkCommitActionExecutor

2023-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6763: - Labels: pull-request-available (was: ) > WriteStats are extracted twice

[jira] [Updated] (HUDI-3727) Add metrics for async indexer

2023-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3727: - Labels: pull-request-available (was: ) > Add metrics for async inde

[jira] [Updated] (HUDI-6481) Implement MultipleServiceRunner to run services on multiple tables through single job

2023-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6481: - Labels: pull-request-available (was: ) > Implement MultipleServiceRunner to run services

[jira] [Updated] (HUDI-6760) Add SelfDescribingInputFormatInterface for hive FileInputFormat

2023-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6760: - Labels: pull-request-available (was: ) > Add SelfDescribingInputFormatInterface for h

[jira] [Updated] (HUDI-6455) Prepped Delete Unit Test For Flink

2023-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6455: - Labels: pull-request-available (was: ) > Prepped Delete Unit Test For Fl

[jira] [Updated] (HUDI-6397) Cleanup tests and code around MDT disable through configs

2023-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6397: - Labels: pull-request-available (was: ) > Cleanup tests and code around MDT disable thro

[jira] [Updated] (HUDI-6759) Some of the valid instants in MDT are ignored

2023-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6759: - Labels: pull-request-available (was: ) > Some of the valid instants in MDT are igno

[jira] [Updated] (HUDI-6758) Avoid duplicated log blocks on the LogRecordReader

2023-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6758: - Labels: pull-request-available (was: ) > Avoid duplicated log blocks on the LogRecordRea

[jira] [Updated] (HUDI-6757) Compaction execution terminated in async threads in flink bounded streaming scene

2023-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6757: - Labels: pull-request-available (was: ) > Compaction execution terminated in async threads

[jira] [Updated] (HUDI-6738) Apply object filter before checkpoint batching in GcsEventsHoodieIncrSource

2023-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6738: - Labels: pull-request-available (was: ) > Apply object filter before checkpoint batching

[jira] [Updated] (HUDI-6711) Write a RFC for Multi Table Txns

2023-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6711: - Labels: pull-request-available (was: ) > Write a RFC for Multi Table T

[jira] [Updated] (HUDI-6754) Fix NullPointerException w/ AbstractRealTimRecordReader

2023-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6754: - Labels: pull-request-available (was: ) > Fix NullPointerException w/ AbstractRealTimRecordRea

[jira] [Updated] (HUDI-6736) Change the order of rollback transitioning to complete and actual commit timeline files deletion

2023-08-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6736: - Labels: pull-request-available (was: ) > Change the order of rollback transitioning to compl

[jira] [Updated] (HUDI-6562) AWSDmsAvroPayload is failing for Delete events when CDC enabled

2023-08-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6562: - Labels: CDC pull-request-available (was: CDC) > AWSDmsAvroPayload is failing for Delete eve

[jira] [Updated] (HUDI-6708) Support Record Index with the Async Indexer

2023-08-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6708: - Labels: pull-request-available (was: ) > Support Record Index with the Async Inde

[jira] [Updated] (HUDI-6741) Timeline server cannot handle multiple base paths when metadata table is enabled

2023-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6741: - Labels: pull-request-available (was: ) > Timeline server cannot handle multiple base paths w

[jira] [Updated] (HUDI-6739) Avoid checking timeline for successful commits for spark structured streaming when offset is 0

2023-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6739: - Labels: pull-request-available (was: ) > Avoid checking timeline for successful commits

[jira] [Updated] (HUDI-6740) Add 0.13.x to Spark 3 support matrix doc

2023-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6740: - Labels: pull-request-available (was: ) > Add 0.13.x to Spark 3 support matrix

[jira] [Updated] (HUDI-6549) Add support for comma separated read path format in CloudObjectsSelectorCommon

2023-08-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6549: - Labels: pull-request-available (was: ) > Add support for comma separated read path format

[jira] [Updated] (HUDI-4115) Replace Configuration in Flink with HoodieConfig

2023-08-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4115: - Labels: pull-request-available (was: ) > Replace Configuration in Flink with HoodieCon

[jira] [Updated] (HUDI-6735) Add support for SnapshotQueryLoadSplit interface

2023-08-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6735: - Labels: pull-request-available (was: ) > Add support for SnapshotQueryLoadSplit interf

[jira] [Updated] (HUDI-6732) Handle wildcards for partition paths passed in via spark-sql

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6732: - Labels: pull-request-available (was: ) > Handle wildcards for partition paths passed in

[jira] [Updated] (HUDI-6731) Allow MoR Read-Optimized BigQuery Sync

2023-08-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6731: - Labels: pull-request-available (was: ) > Allow MoR Read-Optimized BigQuery S

[jira] [Updated] (HUDI-6730) Enable hoodie configuration using the --conf option with the "spark." prefix.

2023-08-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6730: - Labels: pull-request-available (was: ) > Enable hoodie configuration using the --conf opt

[jira] [Updated] (HUDI-6621) Add a downgrade step from 6 to 5 to detect new delete blocks

2023-08-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6621: - Labels: pull-request-available (was: ) > Add a downgrade step from 6 to 5 to detect new del

[jira] [Updated] (HUDI-6729) Fix get partition values from path for non-string type partition column

2023-08-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6729: - Labels: pull-request-available (was: ) > Fix get partition values from path for non-string t

[jira] [Updated] (HUDI-6728) Add Schema Evolution Support to BigQuery Sync

2023-08-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6728: - Labels: pull-request-available (was: ) > Add Schema Evolution Support to BigQuery S

[jira] [Updated] (HUDI-6726) Fix connection leaks related to file reader and iterator close

2023-08-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6726: - Labels: pull-request-available (was: ) > Fix connection leaks related to file reader

[jira] [Updated] (HUDI-6724) Initializing prevInstance to HoodieTimeline.INIT_INSTANT_TS to avoid partial reading of first commit

2023-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6724: - Labels: pull-request-available (was: ) > Initializing prevInstance

[jira] [Updated] (HUDI-6719) Fix data inconsistency issues caused by concurrent clustering and delete partition.

2023-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6719: - Labels: pull-request-available (was: ) > Fix data inconsistency issues caused by concurr

[jira] [Updated] (HUDI-6718) Concurrent cleaner commit same instance conflict

2023-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6718: - Labels: pull-request-available (was: ) > Concurrent cleaner commit same instance confl

[jira] [Updated] (HUDI-6717) Fix downgrade handler for 0.14.0

2023-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6717: - Labels: pull-request-available (was: ) > Fix downgrade handler for 0.1

[jira] [Updated] (HUDI-4756) Clean up usages of "assume.date.partition" config within hudi

2023-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4756: - Labels: pull-request-available (was: ) > Clean up usages of "assume.date.partition

[jira] [Updated] (HUDI-6704) Fix Flink metadata table update

2023-08-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6704: - Labels: pull-request-available (was: ) > Fix Flink metadata table upd

[jira] [Updated] (HUDI-6703) StreamWriteOperatorCoordinator should refresh the last txn metadata firstly for recommit

2023-08-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6703: - Labels: pull-request-available (was: ) > StreamWriteOperatorCoordinator should refresh the l

[jira] [Updated] (HUDI-6697) Add doc for Flink Hudi Catalog

2023-08-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6697: - Labels: pull-request-available (was: ) > Add doc for Flink Hudi Cata

[jira] [Updated] (HUDI-6695) Create an AWS credentials provider to support assuming a role, and use HoodieAWSCredentialsProviderFactory for Glue.

2023-08-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6695: - Labels: aws-glue pull-request-available (was: aws-glue) > Create an AWS credentials provider

[jira] [Updated] (HUDI-6694) Fix log file CLI around command blocks

2023-08-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6694: - Labels: pull-request-available (was: ) > Fix log file CLI around command blo

[jira] [Updated] (HUDI-6692) If table with recordkey doesn't have recordkey in spark ds write, it will bulk insert by default

2023-08-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6692: - Labels: pull-request-available (was: ) > If table with recordkey doesn't have recordkey

[jira] [Updated] (HUDI-6683) Added kafka key as part of hudi metadata columns for Json & Avro KafkaSource

2023-08-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6683: - Labels: pull-request-available (was: ) > Added kafka key as part of hudi metadata columns

[jira] [Updated] (HUDI-6690) Generate test jars for hudi-utilities and hudi-hive-sync modules

2023-08-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6690: - Labels: pull-request-available (was: ) > Generate test jars for hudi-utilities and hudi-h

[jira] [Updated] (HUDI-6689) Add record index validation in metadata table validator

2023-08-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6689: - Labels: pull-request-available (was: ) > Add record index validation in metadata table valida

[jira] [Updated] (HUDI-6688) Fix partition validation to only consider commits in metadata table validator

2023-08-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6688: - Labels: pull-request-available (was: ) > Fix partition validation to only consider commits

[jira] [Updated] (HUDI-6686) Handling empty commit for s3 Incr job

2023-08-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6686: - Labels: pull-request-available (was: ) > Handling empty commit for s3 Incr

[jira] [Updated] (HUDI-6685) Fix code typo in quick start guide under pyspark "Insert Overwrite" section.

2023-08-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6685: - Labels: pull-request-available (was: ) > Fix code typo in quick start guide under pysp

<    5   6   7   8   9   10   11   12   13   14   >