[
https://issues.apache.org/jira/browse/HUDI-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liujinhui updated HUDI-3765:
----------------------------
Description:
When writing to the mor table using structured streaming, when the asynchronous
clean service is enabled, the clean will only be triggered when the task is
restarted. Through debugging, it is found that in the process of continuous
operation, the generation of clean will not actually be triggered. ...
{code:java}
// Async cleaner has been spawned. Waiting for it to finish |
org.apache.hudi.client.BaseHoodieWriteClient.autoCleanOnCommit(BaseHoodieWriteClient.java:541)
2022-03-31 19:26:06,677 | INFO | [stream execution thread for [id =
ce830c81-00c6-4d83-8a60-1970b8a6a1c9, runId =
d57ad29c-c63e-441d-b183-0e5eb028acc5]] | Waiting for async clean service to
finish |
org.apache.hudi.async.AsyncCleanerService.waitForCompletion(AsyncCleanerService.java:73)
2022-03-31 19:26:06,677 | INFO | [stream execution thread for [id =
ce830c81-00c6-4d83-8a60-1970b8a6a1c9, runId =
d57ad29c-c63e-441d-b183-0e5eb028acc5]] | Async cleaner has finished |
org.apache.hudi.client.BaseHoodieWriteClient.autoCleanOnCommit(BaseHoodieWriteClient.java:543)
{code}
was:When writing to the mor table using structured streaming, when the
asynchronous clean service is enabled, the clean will only be triggered when
the task is restarted. Through debugging, it is found that in the process of
continuous operation, the generation of clean will not actually be triggered.
...
> Can not clean asynchronously
> ----------------------------
>
> Key: HUDI-3765
> URL: https://issues.apache.org/jira/browse/HUDI-3765
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: liujinhui
> Priority: Major
>
> When writing to the mor table using structured streaming, when the
> asynchronous clean service is enabled, the clean will only be triggered when
> the task is restarted. Through debugging, it is found that in the process of
> continuous operation, the generation of clean will not actually be triggered.
> ...
>
>
> {code:java}
> // Async cleaner has been spawned. Waiting for it to finish |
> org.apache.hudi.client.BaseHoodieWriteClient.autoCleanOnCommit(BaseHoodieWriteClient.java:541)
> 2022-03-31 19:26:06,677 | INFO | [stream execution thread for [id =
> ce830c81-00c6-4d83-8a60-1970b8a6a1c9, runId =
> d57ad29c-c63e-441d-b183-0e5eb028acc5]] | Waiting for async clean service to
> finish |
> org.apache.hudi.async.AsyncCleanerService.waitForCompletion(AsyncCleanerService.java:73)
> 2022-03-31 19:26:06,677 | INFO | [stream execution thread for [id =
> ce830c81-00c6-4d83-8a60-1970b8a6a1c9, runId =
> d57ad29c-c63e-441d-b183-0e5eb028acc5]] | Async cleaner has finished |
> org.apache.hudi.client.BaseHoodieWriteClient.autoCleanOnCommit(BaseHoodieWriteClient.java:543)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
