[
https://issues.apache.org/jira/browse/SPARK-10072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das resolved SPARK-10072.
-----------------------------------
Resolution: Fixed
Fix Version/s: 1.5.0
> BlockGenerator can deadlock when the queue block queue of generate blocks
> fills up to capacity
> ----------------------------------------------------------------------------------------------
>
> Key: SPARK-10072
> URL: https://issues.apache.org/jira/browse/SPARK-10072
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
> Fix For: 1.5.0
>
>
> Generated blocks are inserted into an ArrayBlockingQueue, and another thread
> pulls stuff from the ArrayBlockingQueue and pushes it into BlockManager. Now
> if that queue fills up to capacity (default is 10 blocks), then the inserting
> into queue (done in the function updateCurrentBuffer) get blocked inside a
> synchronized block. However, the thread that is pulling blocks from the queue
> uses the same lock to check the current (active or stopped) while pulling
> from the queue. Since the block generating threads is blocked (as the queue
> is full) on the lock, this thread that is supposed to drain the queue gets
> blocked. Ergo, deadlock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
