Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226737882 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -274,11 +282,20 @@ private[history] class FsHistoryProvider(conf: SparkConf, clock: Clock) pool.scheduleWithFixedDelay( getRunner(() => checkForLogs()), 0, UPDATE_INTERVAL_S, TimeUnit.SECONDS) - if (conf.getBoolean("spark.history.fs.cleaner.enabled", false)) { + if (conf.get(CLEANER_ENABLED)) { // A task that periodically cleans event logs on disk. pool.scheduleWithFixedDelay( getRunner(() => cleanLogs()), 0, CLEAN_INTERVAL_S, TimeUnit.SECONDS) } + + driverLogFs.foreach { _ => --- End diff -- Here it's clearer to use `if (driverLogFs.isDefined && ...)`. But really this is a bit confusing and I think my previous suggestion is clearer: don't keep `driverLogFs` in a field, just get it when running the cleaner task. And do this initialization based on the config entries, not `driverLogFs`.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org