Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19061#discussion_r142569029 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -598,6 +598,15 @@ object SparkSubmit extends CommandLineUtils with Logging { } } + // In case of shells, spark.ui.showConsoleProgress can be true by default or by user. + if (isShell(args.primaryResource)) { + if (!sparkConf.contains("spark.ui.showConsoleProgress")) { + sysProps("spark.ui.showConsoleProgress") = "true" + } + } else { + sysProps("spark.ui.showConsoleProgress") = "false" --- End diff -- This'll prevent user's mistakes by overriding user's `spark.ui.showConsoleProgress=true` with `SparkSubmit` CLI. In case of `SparkSubmit`, only shells will accept user's `spark.ui.showConsoleProgress`.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org