liupengcheng created SPARK-31105:
------------------------------------
Summary: Respect sql execution id when scheduling taskSets
Key: SPARK-31105
URL: https://issues.apache.org/jira/browse/SPARK-31105
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.0.0
Reporter: liupengcheng
Currently, spark will sort taskSets by jobId and stageId and then schedule them
in order for FIFO schedulingMode. In OLAP senerios, especially under high
concurrency, the taskSets are always from different sql queries and several
jobs can be submitted for execution at one time for one query for adaptive
execution. But now we order those taskSets without considering the execution
group, which may causes the query being delayed.
So I propose to consider the sql execution id when scheduling jobs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
