[
https://issues.apache.org/jira/browse/SPARK-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-19486:
---------------------------------
Labels: bulk-closed (was: )
> Investigate using multiple threads for task serialization
> ---------------------------------------------------------
>
> Key: SPARK-19486
> URL: https://issues.apache.org/jira/browse/SPARK-19486
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler
> Affects Versions: 2.1.0
> Reporter: Shivaram Venkataraman
> Priority: Major
> Labels: bulk-closed
>
> This is related to SPARK-18890, where all the serialization logic is moved
> into the Scheduler backend thread. As a follow on to this we can investigate
> using a thread pool to serialize a number of tasks together instead of using
> a single thread to serialize all of them.
> Note that this may not yield sufficient benefits unless the driver has enough
> cores and we don't run into contention across threads. We can first
> investigate potential benefits and if there are sufficient benefits we can
> create a PR for this.
> cc [~kayousterhout]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
