[
https://issues.apache.org/jira/browse/HUDI-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17640509#comment-17640509
]
lei w edited comment on HUDI-2833 at 11/29/22 9:41 AM:
-------------------------------------------------------
hi ,[~zhangyue19921010], this feature is to merage all candidateFiles(small
archive instant) to a new logFile, then delete the candidateFiles? I have a
question is what time of the new logFile will be cleaned up?
was (Author: lei w):
hi ,[~zhangyue19921010], this feature is to merage all candidateFiles(small
archive instant) to a new logFile, then delete the candidateFiles? I have a
question is when the new logFile will be cleaned?
> Clean up unused archive files instead of expanding indefinitely
> ---------------------------------------------------------------
>
> Key: HUDI-2833
> URL: https://issues.apache.org/jira/browse/HUDI-2833
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Yue Zhang
> Priority: Major
> Labels: core-flow-ds, pull-request-available, sev:high
> Fix For: 0.11.0
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> As we know, most of storage do not support append action, so that hoodie will
> create a new archive file under archived dictionary when archiving.
> As time goes by, there may be thousands of archive files, which most of them
> is not useful anymore.
> Maybe it is meaningful to have a function to clean these unused archive files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
