Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22752#discussion_r226420000 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala --- @@ -64,4 +64,11 @@ private[spark] object config { .bytesConf(ByteUnit.BYTE) .createWithDefaultString("1m") + val IN_PROGRESS_ABSOLUTE_LENGTH_CHECK = + ConfigBuilder("spark.history.fs.inProgressAbsoluteLengthCheck.enabled") --- End diff -- How much overhead are we talking about? That thread is not really in any performance critical path, and in general people won't have so many running apps that this should become a problem... but that kinda depends on how bad this new call is.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org