wangminfeng created SPARK-21171:
-----------------------------------
Summary: Speculate task scheduling block dirve handle normal task
when a job task number more than one hundred thousand
Key: SPARK-21171
URL: https://issues.apache.org/jira/browse/SPARK-21171
Project: Spark
Issue Type: Improvement
Components: Block Manager
Affects Versions: 2.1.1
Environment: We have more than two hundred high-performance machine to
handle more than 2T data by one query
Reporter: wangminfeng
If a job have more then one hundred thousand tasks and spark.speculation is
true, when speculable tasks start, choosing a speculable will waste lots of
time and block other tasks. We do a ad-hoc query for badiu data analyse, we
can't tolerate one job wasting time even it is a large job
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
