[
https://issues.apache.org/jira/browse/SPARK-49133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated SPARK-49133:
---------------------------------
Description:
MemoryConsumer doesn't provide internal thread-safety so user should add their
own locking scope when it's used in multi-threaded environment.
Thinking of multiple threads are allocating memory in the same task (although
it's a special case regarding Spark's memory model), to protect the
thread-safety of MemoryConsumer, user has to lock the API invocations of it. In
this case, if one memory consumer spills another concurrently, there's a risk
of ABBA deadlock. E.g., consumer A locks itself to acquire memory from task
memory manager, while consumer B locks task memory manager then spills A.
To fix this problem, Spark could ensure MemoryConsumer's thread-safety so user
doesn't have to add their own lock in most cases.
was:
TaskMemoryManager is thread-safe however MemoryConsumer is not.
Thinking of multiple threads are allocating memory in the same task (it's a
corner case but could be adopted for certain purposes from user side), it leads
to practice that developer should add a lock in the caller code of
MemoryConsumer. However since it's common that one memory consumer spills
another, using lock in user code could easily causes ABBA dead lock, e.g.,
consumer A locks itself to acquire memory from task memory manager, while
consumer B locks task memory manager then spills A.
A solution could be done in the Spark side to ensure basic thread safety of the
MemoryConsumer, so that users don't have to lock the entire memory consumer
which could lead to the ABBA dead lock.
> Make MemoryConsumer#used atomic to avoid deadlock
> -------------------------------------------------
>
> Key: SPARK-49133
> URL: https://issues.apache.org/jira/browse/SPARK-49133
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.5.1
> Reporter: Hongze Zhang
> Priority: Major
>
> MemoryConsumer doesn't provide internal thread-safety so user should add
> their own locking scope when it's used in multi-threaded environment.
> Thinking of multiple threads are allocating memory in the same task (although
> it's a special case regarding Spark's memory model), to protect the
> thread-safety of MemoryConsumer, user has to lock the API invocations of it.
> In this case, if one memory consumer spills another concurrently, there's a
> risk of ABBA deadlock. E.g., consumer A locks itself to acquire memory from
> task memory manager, while consumer B locks task memory manager then spills A.
> To fix this problem, Spark could ensure MemoryConsumer's thread-safety so
> user doesn't have to add their own lock in most cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
