Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2944#issuecomment-60856334
Alright, I've updated this to send the dumps as part of a separate
fire-and-forget RPC and removed the new code from the heartbeat code paths
(which should make things cleaner).
One final thing that I may experiment with is compression of the stack
traces. There's bound to be a lot of redundancy in these stacktraces, so they
should be highly compressible.
I'm also going to add an option to disable sending stacktraces so that we
have an escape-hatch in case this turns out to be a problem (or if a user would
rather not expose stacktraces in the UI for some reason).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
