Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49264857 If a TaskSet only contains no-pref tasks, there won't be delay because the only valid level is ANY, so everything gets scheduled right away. If a TaskSet contains process-local + no-pref tasks (I suppose it should be process-local + node-local + no-pref actually since process-local task is also node-local?), there'll be delay on the no-pref tasks, even when all the process-local tasks have been launched. This is because valid levels are not re-computed as task finishes. Not sure if there'll be too much overhead to do so... (it should also be re-computed when executor is lost)
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---