big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1654900164
> > how should these log files be cleaned up. Duplicate bucket id files
cause tasks to fail to start all the time
>
> The log expected to be cleaned when the instant is committed
big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1654860685
>
Maybe you are right, if the partial failover will not be rolled back, how
should these log files be cleaned up. Duplicate bucket id files cause tasks to
fail to start all the
big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1653037352
StreamWriteOperatorCoordinator#handleBootstrapEvent() -> initInstant() ->
startInstant() -> this.writeClient.startCommit() ->
tableServiceClient.rollbackFailedWrites() ->
big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1652846394
> > failover causes the bootstrap event not to be sent
>
> Why we must send the bootstrap event in this case, the bootstrap event
itself is a empty event without any metadata.
big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1651615360
> > How does this affect metadata cleaning?
>
> It removes the preceeding partial metadata if there is any.
Before the checkpoint is completed, BucketStreamWrite flush buffer
big-doudou commented on PR #9182:
URL: https://github.com/apache/hudi/pull/9182#issuecomment-1650926469
> Each failed attempt of a subtask would trigger invocation of
`StreamWriteOperatorCoordinator#subtaskFailed`, the original write metadata
would got cleaned,
The
