[ 
https://issues.apache.org/jira/browse/SPARK-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-19466:
------------------------------------

    Assignee:     (was: Apache Spark)

> Improve Fair Scheduler Logging
> ------------------------------
>
>                 Key: SPARK-19466
>                 URL: https://issues.apache.org/jira/browse/SPARK-19466
>             Project: Spark
>          Issue Type: Improvement
>          Components: Scheduler
>    Affects Versions: 2.1.0
>            Reporter: Eren Avsarogullari
>            Priority: Minor
>
> Fair Scheduler Logging for the following cases can be useful for the user.
> 1- If *valid* spark.scheduler.allocation.file property is set, user can be 
> informed so user can aware which scheduler file is processed when 
> SparkContext initializes.
> 2- If *invalid* spark.scheduler.allocation.file property is set, currently, 
> the following stacktrace is shown to user. In addition to this, more 
> meaningful message can be shown to user by emphasizing the problem at 
> building level of fair scheduler and covering other potential issues at this 
> level.
> {code:xml}
> Exception in thread "main" java.io.FileNotFoundException: INVALID_FILE (No 
> such file or directory)
>       at java.io.FileInputStream.open0(Native Method)
>       at java.io.FileInputStream.open(FileInputStream.java:195)
>       at java.io.FileInputStream.<init>(FileInputStream.java:138)
>       at java.io.FileInputStream.<init>(FileInputStream.java:93)
>       at 
> org.apache.spark.scheduler.FairSchedulableBuilder$$anonfun$buildPools$1.apply(SchedulableBuilder.scala:76)
>       at 
> org.apache.spark.scheduler.FairSchedulableBuilder$$anonfun$buildPools$1.apply(SchedulableBuilder.scala:75)
> {code}
> 3- If spark.scheduler.allocation.file property is not set and *default* fair 
> scheduler file(fairscheduler.xml) is found in classpath, it will be loaded 
> but currently, user is not informed so logging can be useful.
> 4- If spark.scheduler.allocation.file property is not set and default fair 
> scheduler file does not exist, currently, user is not informed so logging can 
> be useful.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to