Kousuke Saruta created SPARK-32773:
--------------------------------------
Summary: The behavior of listJars and listFiles is not consistent
with YARN and other cluster managers
Key: SPARK-32773
URL: https://issues.apache.org/jira/browse/SPARK-32773
Project: Spark
Issue Type: Bug
Components: YARN
Affects Versions: 3.1.0
Reporter: Kousuke Saruta
Assignee: Kousuke Saruta
Jars/files specified with --jars/--files options are listed by sc.listJars and
listFiles except when we run apps on YARN.
If we run apps not on YARN, those files are served by the embedded file server
in the driver and listJars/listFiles list the served files.
But with YARN, such files specified by the options are not served by the
embedded file server so listJars and listFiles don't list them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
