Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/22752#discussion_r226416951 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala --- @@ -64,4 +64,11 @@ private[spark] object config { .bytesConf(ByteUnit.BYTE) .createWithDefaultString("1m") + val IN_PROGRESS_ABSOLUTE_LENGTH_CHECK = + ConfigBuilder("spark.history.fs.inProgressAbsoluteLengthCheck.enabled") --- End diff -- This new check adds overhead to the checkForLogs thread, made it disabled by default since most of the users may not want to see the history ui for the running applications, they can enable it explicitly if they want to see progress apps in history ui. I can remove this config if you think not much useful.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org