Github user squito commented on the issue: https://github.com/apache/spark/pull/22109 this looks reasonable, but now I'm wondering whether this will only affect the driver. Couldn't it also effect the executors? Executors might get created as soon as there is a [`schedulerBackend`](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L493) and they can register as soon as there is a blockManagerMaster, which is done as part of the [SparkEnv creation](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L424) -- both happen before the EventLoggingListener is created. so we'd need to do something different for that case, probably have the listenerBus buffer up events until we tell it all "primimary" listeners are registered. I suppose that's an extremely unlikely race, so I might be overthinking this. I do recall seeing cases where the executors didn't have info either, but I doubt this was the cause.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org