Repository: spark Updated Branches: refs/heads/branch-1.6 cb869a143 -> 1f031635f
[SPARK-13441][YARN] Fix NPE in yarn Client.createConfArchive method ## What changes were proposed in this pull request? Instead of using result of File.listFiles() directly, which may throw NPE, check for null first. If it is null, log a warning instead ## How was the this patch tested? Ran the ./dev/run-tests locally Tested manually on a cluster Author: Terence Yim <tere...@cask.co> Closes #11337 from chtyim/fixes/SPARK-13441-null-check. (cherry picked from commit fae88af18445c5a88212b4644e121de4b30ce027) Signed-off-by: Sean Owen <so...@cloudera.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1f031635 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1f031635 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1f031635 Branch: refs/heads/branch-1.6 Commit: 1f031635ffb4df472ad0d9c00bc82ebb601ebbb5 Parents: cb869a1 Author: Terence Yim <tere...@cask.co> Authored: Thu Feb 25 13:29:30 2016 +0000 Committer: Sean Owen <so...@cloudera.com> Committed: Thu Feb 25 13:29:41 2016 +0000 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/deploy/yarn/Client.scala | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/1f031635/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---------------------------------------------------------------------- diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala index f0590d2..7631aa3 100644 --- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala +++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala @@ -539,9 +539,14 @@ private[spark] class Client( sys.env.get(envKey).foreach { path => val dir = new File(path) if (dir.isDirectory()) { - dir.listFiles().foreach { file => - if (file.isFile && !hadoopConfFiles.contains(file.getName())) { - hadoopConfFiles(file.getName()) = file + val files = dir.listFiles() + if (files == null) { + logWarning("Failed to list files under directory " + dir) + } else { + files.foreach { file => + if (file.isFile && !hadoopConfFiles.contains(file.getName())) { + hadoopConfFiles(file.getName()) = file + } } } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
