FMX commented on code in PR #3144:
URL: https://github.com/apache/celeborn/pull/3144#discussion_r2000415124
##########
client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java:
##########
@@ -344,6 +344,20 @@ public static void cancelShuffle(int shuffleId, String
reason) {
.hiddenImpl(TaskSetManager.class, "taskInfos")
.defaultAlwaysNull()
.build();
+ private static final DynFields.UnboundField<
+ scala.collection.mutable.HashMap<
+ Integer, scala.collection.mutable.HashMap<Integer,
TaskSetManager>>>
+ TASK_SETS_BY_STAGE_ID_AND_ATTEMPT_FIELD =
+ DynFields.builder()
+ .hiddenImpl(TaskSchedulerImpl.class,
"taskSetsByStageIdAndAttempt")
Review Comment:
```
// TaskSetManagers are not thread safe, so any access to one should be
synchronized
// on this class. Protected by `this`
private val taskSetsByStageIdAndAttempt = new HashMap[Int, HashMap[Int,
TaskSetManager]]
```
It looks like this variable is not thread-safe, will this be safe if you get
the variable by reflection?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
