[
https://issues.apache.org/jira/browse/HUDI-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-4021:
----------------------------
Component/s: compaction
table-service
> Support deferring compaction when there is an inflight delta commit
> -------------------------------------------------------------------
>
> Key: HUDI-4021
> URL: https://issues.apache.org/jira/browse/HUDI-4021
> Project: Apache Hudi
> Issue Type: Improvement
> Components: compaction, multi-writer, table-service, writer-core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
>
> in multi-writer scenario, if there is an inflight delta commit by writer1 and
> if writer2 i looking to schedule compaction, the delta commit might
> eventually fail. This may not be a good experience for end user. Think about
> regular writes are happening via deltastreamer and a separate async process
> is taking care of scheduling and executing compaction. Deltastreamer
> continuous mode might get aborted whenever such overstepping happens. We
> should add a support where in, compaction scheduling is deferred when an
> inflight delta commit is found in the timeline.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
