dongjoon-hyun commented on PR #60164: URL: https://github.com/apache/spark/pull/60164#issuecomment-6689269761
Re: "Failing post-status-update listeners no longer get their own error log entry" in the review above. No, dropping it was not intended, and the structured-logging concern is real. I rendered a failing listener with the real `SparkLayout.json` (`JsonTemplateLayout`) and with the default `%m%n%ex` pattern: - Pattern layout lists `Suppressed: java.lang.IllegalStateException: push failed` with the listener's stack frames. - The JSON layout only has `TaskCompletionListenerException` with `msg: "push failed"`. The listener exception class and its stack frames are missing, since the template does not render suppressed exceptions. Fixed in f1449e8: each failure is logged as `Error in PostStatusUpdateListener` with its own exception in the catch, best effort like `invokeTaskInterruptListeners`, so a failure to log is added as suppressed to the listener's exception and cannot mark the task failed. Two tests cover it: every failing listener is logged with its exception, and a failing log call does not mark the task failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
