Satha created SPARK-61830:
-----------------------------
Summary: ConsoleProgressBar fails when COLUMNS is empty
Key: SPARK-61830
URL: https://issues.apache.org/jira/browse/SPARK-61830
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 4.3.0
Reporter: Satha
ConsoleProgressBar reads the terminal width with
\{{sys.env.getOrElse("COLUMNS", "80").toInt}}. The default only covers the
unset case.
If COLUMNS is set but empty (we see this on headless hosts), SparkContext fails
to start:
{noformat}
$ COLUMNS= ./bin/spark-shell
ERROR SparkContext: Error initializing SparkContext.
java.lang.NumberFormatException: For input string: ""
{noformat}
A negative value parses, but then every job that draws the bar fails when the
bar is cleared:
{noformat}
$ COLUMNS=-1 ./bin/spark-shell
scala> sc.parallelize(1 to 4, 4).map \{ x => Thread.sleep(1500); x }.count()
java.lang.IllegalArgumentException: count is negative: -1
at java.lang.String.repeat(String.java:4651)
at org.apache.spark.ui.ConsoleProgressBar.clear(ConsoleProgressBar.scala:108)
at
org.apache.spark.ui.ConsoleProgressBar.finishAll(ConsoleProgressBar.scala:118)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2807)
{noformat}
The progress bar is cosmetic, so it should just fall back to 80 when COLUMNS is
not a positive integer.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]