Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/22609
> I tried that a lot while writing this code, but the disk store is just
too slow
That is unfortunate ... I was actually hoping that would be a good
compromise if expanded api needs to be preserved.
> if UI is disabled, any task related metrics were reported as zero:
I was incorrect in this regard - they were not zero, but
JobProgressListener had stage/job counters which maintained this.
This was limited to job and stage level though - keeping the actual
overhead fairly minimal and honoring
spark.ui.retainedStages/spark.ui.retainedJobs.
Given this, current PR would be a functionality regression - since we end
up loosing information we would otherwise expose.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
