Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20532
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20532
I agree with @jiangxb1987 ... we already have issues with event logs being
too big, as it the driver gets backlogged even writing them out, and then the
history server takes a long time to parse
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20532
I'm still wondering whether event log is supposed to work this way, that as
the source for customized analysis. I really feel we shall need some
event/metrics logging framework that serves for
Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/20532
Thanks @jerryshao. Changed to simply add a new configuration instead
sampling logging.
---
-
To unsubscribe, e-mail:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20532
I would suggest to do it like what we have already done for block update
event. Since we already opened a door for block update event, it is also
acceptable to leave room for another event. User
Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/20532
Thanks everyone. So just close it? Or easily leave an enabled switch like
blockUpdated dose? I am all OK.
---
-
To
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20532
I can see why you want this sometimes, but I'm trying to figure out if its
really valuable for users in general. You could always add a custom listener
to log this info. It would go into separate
Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/20532
@jerryshao and @jiangxb1987 , thanks for your advice. In 2.1.x, the two
Update events (BlockUpdated & ExecutorMetricsUpdate) are all dumb. And in
2.2.x, only BlockUpdated event has a
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20532
I agree with @jiangxb1987 . @LantaoJin would you please elaborate the usage
scenario of dumping executor metrics to event log? Seems history server doesn't
leverage such information necessarily.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20532
I'm also worried that if we want to sample more events in the future, we
have to add more configs following this way, which doesn't sound like a perfect
choice.
---
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20532
Emmm... in case we want to sample more events, does that means we shall add
a new config for each event sampling?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20532
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/20532
@jerryshao Could you have a time to help to review?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20532
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
14 matches
Mail list logo
