GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/23002
[SPARK-26003] Improve SQLAppStatusListener.aggregateMetrics performance
## What changes were proposed in this pull request?
In `SQLAppStatusListener.aggregateMetrics`, we use the `metricIds` only to
filter the relevant metrics. And this is a Seq which is also sorted. When there
are many metrics involved, this can be pretty inefficient. The PR proposes to
use a Set for it.
## How was this patch tested?
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-26003
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23002.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23002
----
commit 7e790412ed6409fdda96216dde7f4f408bb04a57
Author: Marco Gaido <marcogaido91@...>
Date: 2018-11-10T16:04:25Z
[SPARK-26003] Improve SQLAppStatusListener.aggregateMetrics performance
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
