[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Priority: Blocker (was: Critical) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/09/05 (was: 2022/08/22) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/09/19 (was: 2022/09/05) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: (was: 2022/09/19) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaojing Yu updated HUDI-4631: -- Fix Version/s: 0.13.0 (was: 0.12.1) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/10/03 > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Story Points: 3 > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-4631: - Labels: pull-request-available (was: ) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Status: Patch Available (was: In Progress) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Status: In Progress (was: Open) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18 (was: 2022/10/04) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Fix Version/s: 0.12.1 > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Major > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Priority: Critical (was: Major) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Priority: Critical > Fix For: 0.12.1 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/15 (was: 2022/10/04, 2022/10/18, 2022/11/01) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/10/04, 2022/10/18, 2022/11/01) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/15) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/12/06 (was: 2022/10/04, 2022/10/18, 2022/11/01) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4631: -- Epic Link: HUDI-1456 > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 1.0.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4631: -- Fix Version/s: 1.0.0 > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 1.0.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4631: -- Status: Patch Available (was: In Progress) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: Sagar Sumit >Priority: Blocker > Labels: pull-request-available > Fix For: 1.0.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-4631: -- Status: In Progress (was: Open) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: Sagar Sumit >Priority: Blocker > Labels: pull-request-available > Fix For: 1.0.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Status: Open (was: Patch Available) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-4631: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/12/12, 0.13.0 Final Sprint (was: 2022/10/04, 2022/10/18, 2022/11/01, 2022/12/12) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4631: - Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/12/12, 2023-01-09 (was: 2022/10/04, 2022/10/18, 2022/11/01, 2022/12/12, 0.13.0 Final Sprint) > Enhance retries for failed writes w/ write conflicts in a multi writer > scenarios > > > Key: HUDI-4631 > URL: https://issues.apache.org/jira/browse/HUDI-4631 > Project: Apache Hudi > Issue Type: Improvement > Components: multi-writer >Reporter: sivabalan narayanan >Assignee: sivabalan narayanan >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds > w1. and user restarts w2 and for next 5 mins, lets say there are no other > overlapping writers. So the same write from w2 will now succeed. so, whenever > there is a write conflict and pipeline fails, all user needs to do is, just > restart the pipeline or retry to ingest the same batch. > > Ask: can we add retries within hudi during such failures. Anyways, in most > cases, users just restart the pipeline in such cases. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
