Shivaram Venkataraman created SPARK-19485: ---------------------------------------------
Summary: Launch tasks async i.e. dont wait for the network Key: SPARK-19485 URL: https://issues.apache.org/jira/browse/SPARK-19485 Project: Spark Issue Type: Improvement Components: Scheduler Affects Versions: 2.1.0 Reporter: Shivaram Venkataraman Currently the scheduling thread in CoarseGrainedSchedulerBackend is used to both walk through the list of offers and to serialize, create RPCs and send messages over the network. For stages with large number of tasks we can avoid blocking on RPCs / serialization by moving that to a separate thread in CGSB. As a part of this JIRA we can first investigate the potential benefits of doing this for different kinds of jobs (one large stage, many independent small stages etc.) and then propose a code change. cc [~kayousterhout] -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org