[ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lei w updated HUDI-5565: ------------------------ Description: [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start up to avoid data lose. But when task parallelism is changed(increase parallelism), data lose may happen too. Should we recommit instant when writeStatus is not empty in WriteMetadataEvent. (was: [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start up avoid data lose. But when task parallelism is changed, data lose may happen too. Should we recommit instant when writeStatus is not empty in WriteMetadataEvent.) > Application restart may cause data lose when task parallelism is changed > ------------------------------------------------------------------------- > > Key: HUDI-5565 > URL: https://issues.apache.org/jira/browse/HUDI-5565 > Project: Apache Hudi > Issue Type: Bug > Components: core > Reporter: lei w > Priority: Major > > [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted > write metadata when start up to avoid data lose. But when task parallelism is > changed(increase parallelism), data lose may happen too. Should we recommit > instant when writeStatus is not empty in WriteMetadataEvent. -- This message was sent by Atlassian Jira (v8.20.10#820010)