Anurag Mantripragada created SPARK-40475:
--------------------------------------------
Summary: Allow job status tracking with jobGroupId
Key: SPARK-40475
URL: https://issues.apache.org/jira/browse/SPARK-40475
Project: Spark
Issue Type: Improvement
Components: Spark Core, Web UI
Affects Versions: 3.3.0
Reporter: Anurag Mantripragada
Spark let's us group jobs together by setting a job group id. This is useful to
check the job group in the web UI. For example
{{spark.sparkContext().setJobGroup("mygroup_id")}}
We have a use-case where we would like to have a long running Spark application
and have jobs submitted to it. We would like to programmatically check the
status of the jobs created by this group id. For example,
[SQLStatusStore|#L41]] has `executionList()` which returns a map of jobs to the
status. There is no way to filter this based on jobGroupId.
This Jira is to add ability to get fine grained job statues by jobGroupId.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
