[ https://issues.apache.org/jira/browse/SPARK-30800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sarth Frey updated SPARK-30800: ------------------------------- Description: Issue is only present if this PR gets merged: [https://github.com/apache/spark/pull/27395] Currently the `BarrierTaskContext.barrier` method and the `BarrierTaskContext.allGather` method both call the `BarrierTaskContext.runBarrier` method to perform the majority of the computation. A refactor to reduce complexity would be to change the `BarrierTaskContext.barrier` method to simply call `BarrierTaskContext.allGather` with an empty string as input and return `Unit` to the user. In this way, we can remove the `requestMethod` field from `RequestToSync` and do not have to case on `BarrierRequestToSync` and `AllGatherRequestToSync` in the `BarrierCoordinator` code. was: Issue is only present if this PR gets merged: [https://github.com/apache/spark/pull/27395] Currently the `BarrierTaskContext.barrier` method and the `BarrierTaskContext.allGather` method both call the `BarrierTaskContext.runBarrier` method to perform the majority of the majority of the computation. A refactor to reduce complexity would be to change the `BarrierTaskContext.barrier` method to simply call `BarrierTaskContext.allGather` with an empty string as input and return `Unit` to the user. In this way, we can remove the `requestMethod` field from `RequestToSync` and do not have to case on `BarrierRequestToSync` and `AllGatherRequestToSync` in the `BarrierCoordinator` code. > Refactor BarrierTaskContext barrier method to use allGather directly > -------------------------------------------------------------------- > > Key: SPARK-30800 > URL: https://issues.apache.org/jira/browse/SPARK-30800 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 3.1.0 > Reporter: Sarth Frey > Priority: Minor > > Issue is only present if this PR gets merged: > [https://github.com/apache/spark/pull/27395] > > Currently the `BarrierTaskContext.barrier` method and the > `BarrierTaskContext.allGather` method both call the > `BarrierTaskContext.runBarrier` method to perform the majority of the > computation. A refactor to reduce complexity would be to change the > `BarrierTaskContext.barrier` method to simply call > `BarrierTaskContext.allGather` with an empty string as input and return > `Unit` to the user. In this way, we can remove the `requestMethod` field from > `RequestToSync` and do not have to case on `BarrierRequestToSync` and > `AllGatherRequestToSync` in the `BarrierCoordinator` code. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org