Github user chemikadze commented on the issue: https://github.com/apache/spark/pull/7786 @vanzin If those would be implemented, would it have any change to get merged? We use preemption quite a lot and current behavior is not the best we can get: logs sometimes getting overfilled with preemption side effects (RPC errors, etc), getting logs hard to read and confusing some users. I agree that depending on task size, effect might be both positive and negative (longer ones anyway won't be able to complete be wasting resources, but lots of shorter ones will not get chance to run). Does it just mean it should be configurable behavior (spark.yarn.releasePreemptedContainers=true/false)?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org