Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13599 @holdenk @HyukjinKwon @ueshin I have updated the PR, and now it also works when executor is restarted and even dynamic allocation is enabled. The only overhead is on the driver side when executor ask for configuration (changes in https://github.com/apache/spark/pull/13599/files#diff-7d99a7c7a051e5e851aaaefb275a44a1), Now I will constructing new properties when executor ask for properties instead of like before that creating properties before hand and never change even when SparkConf is changed. But I believe the overhead is trivial, so the improvement could be done in a separated PR.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org