Ruifeng Zheng created SPARK-44862:
-------------------------------------
Summary: Adding metric tracking number of jobs running on a cluster
Key: SPARK-44862
URL: https://issues.apache.org/jira/browse/SPARK-44862
Project: Spark
Issue Type: New Feature
Components: Web UI
Affects Versions: 4.0.0
Reporter: Ruifeng Zheng
We frequently come across issues where in there are considerable number of
jobs/notebooks running on one interactive cluster. This leads to either
degraded performance due to cluster being overloaded or leads to driver OOM. It
would be helpful to have a metric identifying number of notebooks/jobs running
on a cluster as this would be important datapoint for customers to balance out
their workloads.
We propose to add a metric which keeps track of number of jobs running on a
cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
